Our truth
In a time of deep fakes, how can the Earth observation community push back against artificial representations of Planetary events?
The sky has always engendered a sense of the mystic. Mysteries of the circular movement of points across an infinite black. An assembling of shapes, appearance of tones, and motes of regular reflections all capture by an upwards tilt of the head. We’ve had a glimpse of the astral workings of the mechanical clock of our universe. Humans would see patterns that foretold seasons and catastrophes.
Today, we still consult with the sky, asking questions about what it sees. While many still look out, our geospatial and Earth observation community, use orbital sensors to look back down at ourselves. These sensors are the bringers of truth. Looking down from the heavens on us and the world we have sculpted. That truth is only demeaned by humanity's crass interpretations of those pixels. While those sensors, in the silence of orbital velocity, continue to capture, record, and transmit… Capture, record, and transmit…
There is art in bringing the stories of our landscapes to people. Those stories are of colours we can and can’t see. Of measuring the beauty of nature through the electromagnetic reflection of our sun by dirt, rock, or leaf. Of the steaming emisivities of our planet and its industry. And, there is science in measuring the pulsing vitality of our home.
But I fear our community of planetary voyeurs has not yet even touched the subject of truth. Across the media spectrum, we are immersed in imagery of various sorts, and we are increasingly challenged with the question of what is real, what is true, and what is artificial.
Of course, what is real and what is true are questions more of philosophy and interpretation. Truth, as a concept, is never reached. There is always an interpretation in the definition of truth. Interestingly, while truth is impossible to achieve, falsehoods are easy. The negative is much easier to determine than the positive. This is true of business strategy as it is of morals. It’s often easier to know what not to do, and with the negative as a guide, we can chart a path towards the positive.
A phenomenon‘s reality is yet another interpretation. How would we determine reality? As humans, witnessing that phenomenon would be the most visceral method. Can you see it? Can you smell, touch, or hear it? Can we use devices to measure it? I can’t see a muon, but I can build a muon detector, so they must exist… right?
Maybe, but do we trust our devices? Are we willing to trust the nuance of scientific instrumentation? Usually. If I go running with my family, our GPS watches will each have different readings. This is not a digital maliciousness insisting that I run an additional 253m to get to the same distance as the others. Instead, it’s the technical mirage of geographic accuracy. 10 Km for one watch is not 10 km for another. Of course, we can now augment our realities with information overlaid on our senses to provide entertainment or information. But are we augmenting with information, or opinion? So reality can be measured within the context of sensory perception and accuracy.
What we have not tackled as a geospatial community is the artificial. Most of our community uses artificial intelligence for interpretation purposes. We are familiar with the creation of neural networks and now we are highly motivated by foundation models and embeddings. But, what I fear is an inevitability is the emergence of deep fakes from space? When will we consult the sky and have to wonder who those devices are serving, and if the array of pixels presented reflects an existing or artificial landscapes?
The provenance of a pixel is mostly untested. But this subject will become increasingly complex as our global community starts to push against the constraints of accepted international borders. Who gets to write the histories in real time? In a very real sense, our geospatial community is empowered and responsible for reporting on the changes we see. While not journalists, we are custodians of a digital journal, reflections of time, ultimately a flip-book of human activity. When future communities ask “what happened?” our community will have provided the eyewitness account.
So, when we are challenged with an alternative image, how can we know what or who is right? Can AI be used to write a different history? Certainly. How can we defend against that inevitability? What is effective digital provenance.
Provenance can happen in several ways. We could digitally encrypt at the source, or use some kind of blockchain to create a chain of custody. In some ways this kind of activity would be operationally beneficial too. How often is a pixel abused in the preprocessing of an image. It can be squashed and stretched in geographic reprojection, and radiometrically teased during atmospheric correction. While these processes make the product more functional and representative, they also represent a edit to the original image.
Another consideration is that any argument or interpretation benefits from listening to more than one opinion. With that simple concept in mind, perhaps more than one source of data should be sought when trying to interpret a landscape phenomena. This simple sounding statement hides a multitude of technical complexities and cost. But these complexities must be meaningfully addressed by our community anyway.
We will need to address harmonization of engineering and data. Having a common understanding of time and space would be a good start. I have often observed that the EO sector has always been too sensor rather than location obsessed. The net result of this is that comparing images taken at different times from the same sensor rarely “line up” in geographic or radiometric spaces. If we were modelling the planet and had a common understanding of its shape (perhaps some sort of discrete grid system), the changes would be more easily consumed. If we can’t even have imagery from the same sensor line up, think of the challenges of different sensors, different engineering teams, different countries and even different phenologies of data.
Time also is a critical component of this model. Sensors are passing locations at different times so there will almost certainly be discontinuities in the content of each image or data source.
And, then there is the simple barrier of cost. More images imply a higher cost. Even if the data sources are open, there is a processing and storage cost to be considered.
Another consideration is that each image or data source is typically considered in isolation. I suppose this is a remnant of the sensor-approach, suggested earlier. But if we consider that every image is an image of a place, and that place has both a history and a future, then a particular image can be proven or disproven by the history and future of that location. Of course some events are deeply temporal, such as conflicts, protests, or sporting events, but even so, these events can often leave a signature on their landscapes.
So time itself has some built-in provenance if we consider the geospatial community as custodians of a history of our Planet’s surface. So, the final technical opportunity is to build a living digital twin of our planet, fed with new Earth observation data.
This, then is the hope I have for geospatial foundation models: in a time of deep fakes we can develop a series of models, based on agreed representations of our Planet, that will consume new data sources as they become available to act as one source of information on global human activity.
Because right next to the word truth, is the word trust.