This is the script from a presentation I gave at this year’s (2026) Cloud Native Geospatial Forum
I’m not a very metaphysical person. But if you will allow me to get a bit woo-woo for a second, I would point out that we are all sharing a common experience.
How we feel about this experience is up to ourselves, and I am willing to accept your reviews. But this experience is happening at a particular place in our universe, in this hotel. And it’s happening… right now.
As a simple creature, I tend to think that those two attributes, space and time, are somewhat fixed. They happened. Perhaps our universes diverge from this point, but now, and just then, they were common. In fact, this hotel has been here for some time, Claude told me, since 1974; Snowbird itself thinks it opened in 1973. Details. The point is, this place has existed for some time.
My name is Will Cadell, and I have been running a company called Sparkgeo since 2010. We build cloud native geospatial data pipelines. Prior to that, I worked in agricultural research and operational forestry. I moved to Canada 20 years ago to climb mountains, and I’ve not really gone home yet.
I have been writing the Strategic Geospatial Substack for the last five years. I’ve been providing advice to enterprises, startups, and venture funds for much longer.
The nature of a large language model is that answers are not always consistent. In fact, that’s often their superpower: the ability to review ideas and come up with new solutions. Annoyingly, that’s why they are good at code. Software development is also non-deterministic: there are many ways to answer a particular question with software. The same is true of prose. You or I can try to describe the same idea in writing, and we could each do it a hundred different ways.
There is something about monkeys and Shakespeare in here. If I could just find the words to articulate it.
However, if you speak to the hotel’s architect, or the HVAC guy, or indeed the concierge in this hotel, I’m quite sure they will all agree that this hotel is, in fact, here. Undisputed.
So, location is determinate. What irks me is that I can ask a probabilistic system for a determinate fact and be given a plausible answer that’s wrong. Those are Claude’s words, not mine.
I made an agent that can pass the GISP exam.
It’s based on Claude, but with a variety of additional skills, geospatial awareness and a deep knowledge of “the shapefile.” My agent is hard to communicate with, so I have to be very explicit. Often, my agent misunderstands me, asking for particular specificity about seemingly irrelevant details, and then offers me a map of the wrong subject. Given that most GIS offices communicate via email, which is essentially a text-based prompt, all this should be somewhat reminiscent and a little concerning to those who identify as GIS people.
But my bigger problem with this workflow was the moment I asked my agent to mark two points indicating local wildfires, and it enthusiastically placed them in the completely wrong places. When I called it out, the response was “good catch.” If I hadn’t known they were wrong, we would have all carried on merrily in complete ignorance.
Here is an illustration.
I ask my agent for a cloud-free image of London, UK. We’ve already created a skill for looking up both Microsoft’s Planetary Computer and the UK Space Agency’s Earth Observation Data Hub. For whatever reason, this time it went to Planetary Computer. And I am given a view of parched South East England. England’s brown and dusty land.
Feeling optimistic, I ask for a point to mark the Tower of London.
You can see that I am assured that on a different map (I guess I wasn’t explicit enough) I will be provided with a point indicating the Tower of London in crisp detail, against a crystal-clear image.
Well, here it is. It’s the same image; to be fair, it’s a beautiful Sentinel image. At first glance, the point is close, except it’s actually on Tower Bridge. And if we zoom in, most of us might push back on the word “crisp.”
This started us on a journey. How good are the Frontier Labs at location?
Firstly, a reminder about location and specifically, navigation. Partial success is not really acceptable; you either get somewhere, or you don’t. Now, humans are thinking computers, so if I’m on Tower Bridge, there is a fair chance that I will get myself the rest of the way to the Tower of London. But it’s not guaranteed. This is what I mean by:
90% right is 100% wrong.
Apple discovered this with the first release of their maps product. Since then, I would argue they have become very good at location and navigation, but it feels like they might have initially underestimated the problem’s scale. Potentially not considering that spatial is a bit special.
So, our non-deterministic robot friends face a problem: They have a per-question answer budget within a probabilistic computing framework, in which not answering is unacceptable. So, like the most enthusiastic intern in the world, they blurt out the best answer they have within their budget.
Everyone in this room knows that, while I state location is not probabilistic, points of interest do change over time. A Starbucks can become a Pete’s or a Subway.
Indeed, maps should be living data spaces that represent our thriving cities.
That said, you would be forgiven for pointing out that the Tower of London’s foundations were, in fact, laid at the direction of William the Conqueror in 1066. Potentially, over the intervening 960 years, we could have shaken out some of the intricacies of natural-language gazetteers.
Nevertheless, places do change, and these programmatic features can conspire to create horrific circumstances. While I often think the work we do is only incrementally useful. And, at the risk of being lynched at the bar later (I wasn’t), potentially a bit niche. At the same time, it is of the utmost importance.
Getting it wrong can result in tragedy. Location matters, a lot.
Today, there are benchmarks for every facet of AI, and several for geospatial activities. Some for geospatial foundation models, some for geoprocessing algorithms, some for a model’s performance in GeoGuessr. But I was surprised to find there were very few for measuring the performance of what you might call ‘consumer location questions’, questions like “Where is the Tower of London?” For instance. So, at Sparkgeo, we have been developing ideas and comparative tools to measure the performance of different models in understanding location.
We wanted to overtly solve the consumer side of the problem. So, not how well a Frontier model can buffer or intersect data. Not how well they can find data products or reproject them. Not how well a geoprocessing script created by Claude works. Instead, how well a Frontier model can resolve a real human problem using natural language and its understanding of the world. The kind of question you might ask Google or Apple Maps.
In doing so, we broke the components down into “Data Quality,” “Spatial Context,” and “Model Reasoning.”
We then bucketed different question types into nine categories for comparison. We felt these describe the most common navigational or situational-awareness use cases.
They are reasonable questions, but we can all see how these might stretch an AI, because we know there is nuance in these questions and a reliance on good location data and some context.
But critically, we also wanted to create question types against personas. Our personas represent the analyst, the professional, and the consumer. The analyst provides their questions with a great deal of operational detail, the professional provides some specificity but is more conversational, and the consumer is extremely terse in their requests, expecting a great deal of inferred context.
Each persona asks their question differently. Here is an example:
Each question is run on a fresh model in a new account on a separate Amazon Workspace. No inferred prior knowledge of any of the personas or those people running the benchmark.
We ran these questions over 6 North American cities. You will notice a commonality with places that are local to my team. This is because we wanted to know how “good” our own benchmark was. To do that, the best way is to get ground data. So yes, this is very North America-centric, and we will figure out how to make it a global solution in time. I suspect the Overture Maps Foundation might be a component.
Now, I am running out of time quickly, and the work we have done is, so far, preliminary. But I wanted to show some teasers. Our first run was over 500 questions. We expect this to grow enormously as we develop this process.
Let’s start with restaurants within a kilometre.
A kilometre is more than a foot but less than a mile for those in the audience unfamiliar. In this example, we asked the frontier models to list restaurants within a kilometre of a particular point, knowing that the correct answer was eight (lower-left pane). The answers vary widely, with Gemini returning over 30 locations and getting only 4 of the actual restaurants right. We see this as a problem of data and spatial reasoning; it largely ignores one of the key criteria: distance. ChatGPT, on the other hand, seemed more willing to provide wrong or old data. Grok and Claude seemed to do best with this question.
Then onto areas.
A park is an area; every model was confused by the concept of an area versus a point, failing to find one of two parks within a five-minute walk of a particular point. This is a failure of spatial reasoning and, indeed, the lack of a spatial model beyond points and lines.
I’m not even thinking about the nuance of where the various entrances to the park might be. Or, indeed, the concept of routing to a park’s edge.
We also found some interesting spatial reasoning challenges
This is the Classic Canadian dilemma: Where is my nearest Tim Hortons?
But this question is more about spatial nuance in the word “nearest”. The confounding factor here is a bridge that has been impassable since 2024. So, nearest becomes a question of interpretation and context.
Grok figured this out every time, and Gemini and ChatGPT in most cases. Claude chose the wrong one twice, once via a connector across the river that doesn’t exist. Gemini once sent the driver to a store “800m” away, which was, in fact, over 3km wrong.
Finally, I wanted to show our data accuracy results.
Ultimately, Gemini and Grok score the highest and are interchangeable at a city level. Claude and ChatGPT score somewhat lower.
We will be publishing these results and our full method in the coming weeks. But overall, you can see a significant opportunity for refinement in both spatial reasoning and data quality.
I think there are interesting opportunities to harness the non-determinism of frontier models for reasoning with more accurate data. I visualize this as adding deterministic control points to solidify a non-deterministic model, similar to how ground control pins a polynomial down for geographic correction.
We could consider applying a POI-based and/or spatial-reasoning MCP to pin down our robot friends. But is a consumer likely to connect to a POI MCP service before finding a place to get coffee? Seems unlikely. The real answer is to encourage our robot friend to take space and time as foundational, while constructing their ideas on those foundations.
In the end, if you are planning a high-consequence event, check your AI’s work. It might be close, but it’s probably not winning any cigars.