Deterministic Control Points
How should we think about deterministic data in a non-deterministic world?
Some things just are. Or were, I suppose.
For instance, my house is where it is, and it has been there since the seventies. It has changed in that time, but the bones of it are somewhat fixed. In fact, physical and political geography chart the fixture of all places through time. At a particular time, a place is generally a place.
Now, different organizations may disagree on attributes of that place, Taiwan, being an obvious example of a disputed sovereignty. Nevertheless, a few people disagree on the presence of the island in question. My point is that certain things are deterministic: they existed at a particular place and time.
My question for today is how we can ensure that knowledge of space and time is properly handled by “less-deterministic” AI technologies. For those who have spent any time trying to orthorectify aerial imagery, this reminds me of adding control points to the polynomial, allowing a modelled representation of the ground to be calculated. It was possible to add incorrect control points, which would blow up your root-mean-square error. The game being to reduce that error to close to zero. That optimization problem is far better done by machines now.
But the mental image of control points is useful: places that, at times in history, hold together our human stories. The bar where you met your partner. The zoo where you first saw a penguin. The Battle of Bannockburn. The coronation of Queen Elizabeth 1st. My postbox. All these examples have location, linked to an event history, some important, some entirely peripheral, all deterministic.
When I think about places in a frontier model world, I am concerned that the deterministic nature of geography is not adequately accounted for. So, how can we help to build the deterministic control points to support a model in its desperate attempt to represent or interpret reality?
Now, you could say that event-driven determinism leaves the door open to models being misinformed about past events. And I believe you’d be right in saying so. Today, though, my concern is that a model’s understanding of geography is generally acquired implicitly, through text and images, rather than explicitly through an understanding of our geoid and human distribution. This implicity breeds a non-determinism of geography that is jarring to those of us who think of space and time as “fixed-features” of our shared human experience.
When I see the phrase:
“You were right to push back, those locations don’t exist,” I am left troubled.
So, what to do? Well, there are some geographic benchmarks to draw from:
GeoAnalystBench, looks at the quality of geographic code creation. GeoBench effectively assesses how well models perform on GeoGuessr. And then there is the other side of the geospatial coin: we have geospatial foundation models, what can be done with deep stacks of imagery, and the development of embeddings. Interestingly, because of the nature of Earth observation, these almost fall back into the non-deterministic bucket, as they typically require substantial context to derive analytics. A pixel can mean many different things depending on the context of an analysis.
My feeling is that we need to develop a series of open benchmarks to encourage frontier models to treat geography as fixed points of information around which non-deterministic interpretations can be made. When I think about this, I continually come back to the idea of SpaceTimeIDs, which aim to track boundary evolution over time (consider the Taiwan example above). But ultimately, SpaceTimeIDs fix a place at a time. I also think about the work my team at Sparkgeo has been doing for the Overture Maps Foundation on living point-of-interest (POI) data, using social signals to indicate whether a place has changed. I remember a discussion with Marc Prioleau a long time ago, where he suggested that building a POI dataset once was ‘relatively straightforward’ when compared to keeping it up to date. He’s the kind of guy who would know.
The point here is that knowing a place exists isn’t necessarily straightforward, and knowing what something was at a particular time can be even harder. But these things, in the most part, should be deterministic. They are fixed points in space and time - control points of reality.