Assist.me
I built an agent. You can too. What happens when global imagery or data does not require specialist anything?
I’ve started building again. Not in the way I used to, but it’s not wildly different either. In fact, I can now build alongside my day job.
I am building an assistant. At some point, it will be a fully fledged Geospatial Analyst. Today, it's just a tool I can use to ask questions and quickly source and view our changing planet. This is something I have been wanting to do for some time. I will do this somewhat openly, so you can play along too. As we do this, I’ll be evaluating a series of tools and practices. Hopefully, we’ll be left with a sense of what is working these days and what might still be flaky.
Something that I know is flaky is how people go about these projects. There is typically minimal planning, and feature bloat tends to follow a dopamine-fueled, token-burning few hours. So, it’s worth taking a few moments to review a few goals. Of course, I think geographically, so this assistant will generally serve a few geographic purposes. Firstly, I want it to interact with open data catalogues and quickly render vector images of places at specific times. That means it needs to understand SpatioTemporal Asset Catalogues (STAC) and have a rendering capability.
I have a sense that AI will provide excellent glue, but I would like my assistant to be able to source data and then assemble components. I will try to encourage it to use the ESA APEx web components to assemble interfaces. Finally, for now, I want to wire in the strategic geospatial articles to provide specific strategic and analytical capabilities around the geospatial market.
For this project, I will start with Claude. I am comfortable with Claude and Claude Code. However, I am somewhat curious what Grok would look like with geographic data, given its purportedly more physics-based approach. That could be a tremendous value-add for geographic analysis. But perhaps in a text-based, code-driven environment, it won’t matter at all? An open question for me, something I will look into in the future.
So, today, I asked Claude to build an agent for me.
During this adventure, you will see a variety of my typos. You will also find I am unusually polite to AIs.
Claude helped me set up a Model Context Protocol (MCP) chat environment and a container (in OrbStack) so I can push this to the public later and get the basic geospatial stack ready. At this point, my agent will be local. Once I’m happy with the functionality, I'll publish it, and we can all watch it evolve. In my basic stack, I needed something to parse a STAC, something to store vectors if needed (PostGIS), and a visualization environment. For this, Claude picked Mapbox, interestingly.
Quick Note. An MCP environment provides the context for a series of chat requests, enabling a more conversational dialogue than a typical stateless Application Programming Interface (API) environment. It’s not wildly more complex, but for an agent, it does matter. Given the market, wherever you have seen an API for the last decade, you should probably now be seeing MCP.
Many geospatial data providers have had a hard time getting to an API; so, getting them MCP-ready might be a stretch. In fact, many Earth Observation providers will tell you they have an STAC implementation, but I would argue many have more of a stuck catalogue! Notably, though, Mapbox does have an MCP environment for its search and navigation services.
So, I worked on my agent.
Ultimately, I have built something that can easily request data from multiple STACs and render them in a fit-for-purpose manner. Additionally, I wired in two interesting features.
The ESA APEx framework provides access to a series of robust geospatial tools and components for assembling ad hoc interfaces.
The history of this substack. Because I wanted to ask this agent to have some market awareness and opinion.
In time, I will release the full repo. It’s not enormous, but I have a few notes on the process.
First, I ended up with four skills. Skills are handy blobs of capability which can be compartmentalized and shared:
Dashboard builder - to build ad-hoc interfaces.
Eodh-connect - to connect to the Earth Observation Data Hub.
Stac-imagery-workflow - to connect with Microsoft’s planetary computer.
Strategic-geospatial-advisor - to interpret market questions through the lens of this publication.
There is certainly room for many, many more skills, as well as some consolidation even here. But the skills are, notionally, handy. The presence of redundancy here implies significant potential for bloat in non-deterministic programming. Indeed, if I look harder, I see more than one way to call a STAC. Hold on, I suppose all programming is non-deterministic, like language is. However, having multiple methods for the same thing is just poor practice. This could become a problem in the future. So, one needs to keep a close eye on the developed code.
Second, it's hard to interrupt Claude when you know it’s pursuing an idea or a debug pathway in the wrong direction. As someone who has written a lot of code and spent a lot of time with geospatial technology and data, I could have helped debug much more quickly. There is a component of the human-ai teaming that I will work on. With a person, I could say “stop, look at this instead…” but interrupting the robot seems harder. That said, the astonishing speed of development is impossible to ignore. I will be digging into the idea of optimal teaming in future discussions.
Third, AI tends to try to please. It then ends up guessing important things like locations. I will address this in future versions. I want control over location disambiguation.
Finally, for now, it's becoming clearer to me that STAC implementations are absolutely essential for scalable data access. Those, combined with a sensible MCP, allow data to be traversed and pulled into ad hoc interfaces. To me, this is a workflow very reminiscent of the traditional GIS, seek and discover data. I expect the jury is out on whether this workflow would be conducted by an analyst and their job accelerated, or by another individual, making the analyst superfluous. In the near term, that will ultimately depend on the other contextual components of the analyst’s responsibilities, such as specific domain knowledge.
For reference, the readme for my geoaptial analyst looks like this:
# geo-agents
AI agents for satellite imagery and vector GIS workflows, built on the [Claude Agent SDK](https://code.claude.com/docs/en/agent-sdk/python).
## Stack
- **Python + uv** — dependency management (`pyproject.toml`)
- **Docker (OrbStack) + GDAL base image** — reproducible native geospatial deps
- **rasterio / rioxarray / pystac-client / planetary-computer** — raster & satellite imagery
- **geopandas / shapely / pyproj** — vector GIS
- **claude-agent-sdk** — agent runtime, custom tools via `@tool` + `create_sdk_mcp_server`
## Setup
Local (uv, using your Homebrew GDAL):
```bash
uv sync
cp .env.example .env # fill in ANTHROPIC_API_KEY, etc.
```
Containerized (recommended once native deps get heavier):
```bash
docker compose build
docker compose run --rm app bash
```
## Run the example agent
```bash
uv run python -m agents.geo_agent
```
## Test
```bash
uv run pytest
```
## Web chat UI
```bash
uv run uvicorn ui.server:app --host 127.0.0.1 --port 8420
```
Open `http://localhost:8420`. Chat interface with inline satellite imagery previews and
interactive Mapbox GL maps rendered directly from tool results.
## Running permanently (macOS)
The chat UI, PostGIS, and Dekart can all run as always-on background services that
survive crashes and restart at login.
**Chat UI** — installed as a macOS LaunchAgent (`deploy/com.geoagents.ui.plist`):
```bash
cp deploy/com.geoagents.ui.plist ~/Library/LaunchAgents/
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.geoagents.ui.plist
launchctl kickstart -k gui/$(id -u)/com.geoagents.ui
```
Logs: `~/Library/Logs/geo-agents-ui.log` / `.error.log`. To stop permanently:
`launchctl bootout gui/$(id -u)/com.geoagents.ui`.
**PostGIS** — `docker-compose.yml` sets `restart: unless-stopped`, so once started
(`docker compose up -d postgis`) it survives Docker/OrbStack restarts.
**Dekart** — `dekart local up` already runs with `--restart unless-stopped`.
**OrbStack itself** — enable "start at login" so the containers above have a Docker
daemon to restart into: `orb config set app.start_at_login true`.
## Layout
- `agents/` — agent entry points (Claude Agent SDK wiring)
- `tools/` — custom tool implementations (imagery search, vector ops, etc.)
- `ui/` — web chat UI (Starlette + WebSocket + Mapbox GL)
- `deploy/` — LaunchAgent plist for running the UI permanently
- `tests/` — unit tests for tools
- `notebooks/` — exploratory analysis
- `data/` — local scratch data (gitignored)