Porpoiseful - your dynamic data strategy
Your data isn't static, why is your data strategy?
It’s a true-ism that I’ve been sharing for years:
That is not new, nor terribly clever. It is, however, enduringly true. Data consistently pays dividends, while you’ll be writing new user interfaces every couple of years just to keep up. Now, when I say “data,” I also mean access to data. A big blob of information is not useful if it sits on an external hard drive, inaccessible to broader use, waiting for physical failure. Waiting for someone to retire and for the corporation to entirely forget that any data was even on the drive, or where the drive is, for that matter. I don’t want to tell you how many discussions I’ve had with organizations that have involved them recalling “that time we lost half a million dollars of survey data.” It’s almost as frequent as the founder stories that invariably involve them reflecting on “that time we spent a million bucks on AWS, by mistake.”
The point being, a data strategy goes well beyond the data. In fact, a data strategy is much more about how data can be used than it is about the data itself.
So, how do we build a data strategy? As always, it goes back to purpose.
Purpose Porpiose
Interestingly, I don’t mean the purpose of a dataset, that's useful metadata, but I’m thinking of the purpose of an organization. Different organizations will assemble data products differently depending on the purpose of that organization. A civil government organization might want to develop an open data strategy. A defence organization may be far more limited by security and compartmentalization needs. A professional services team may need to organize data by project or customer. A resources or logistics company might want data available more corporately, accessible by different user groups.
In each case, the purpose of an organization defines the foundation of their strategy. But there is another key feature, that the data is always changing. In fact, if the data are good and representative of our changing environment (physical or otherwise), then the data should be changing.
So, two key themes of a data strategy are appropriate accessibility and living data. Indeed, a strategy should answer the question: how can one leverage living datasets to meet an enterprise's evolving needs? A dynamic data strategy is the answer. Data will change over time, and the quality of that data should change as sensors, practices, and processes change. Data, while captured at a particular time, may be relevant for longer or instantly out of date. How that data is stored and then accessed becomes a question of purpose, opportunity, and cost.
What is useful?
While data captured for one organization is specifically useful, it is also possible that the same data product may be generally useful for another organization. Those organizations might exist within the same enterprise. The inability of these organizations to share data is known as a silo. This is even more relevant for geographic data. A particular pixel, which is a measurement of the reflectance of a particular surface at a particular time, could be used for impervious surface modelling, fire fuel modelling, land cover mapping, as a basemap for a web experience, or in agricultural analysis, among numerous other applications. The point being, a data point can be generally useful within a single organization. Even if it’s just old reference data, its use is demonstrable. But its use depends on its discoverability.
Discoverability should be the technical basis of a strategy. I have started using teh phase “protect at all cost, share carefully.” This embodies the idea that data should be nurtured and managed, but shared with a robust authenticationa nd authorization framework. Organizations need to explicitly choose who can access their data. This could mean being open, partially open, or selectively available within different organizational units, club goods, or other sharing models. Critically, this needs to be done with intent. With many traditional data management systems, it's easy to sleepwalk into uncomfortable situations. Being strategically intentional clarifies the situation. Mistakes can still happen, and probably will, but with strategic intent, those mistakes can be quickly identified and resolved.
A second technical note is the use of programmatic Application Programming Interface (API) access. I am a huge fan of the SpatioTemporal Asset Catalogue as a commonly understood community and now OGC standard. These days, machine access is as important as people's access. So, perhaps as important as the API is a Model Context Protocol (MCP) environment on top. This will support a more agentic, conversational experience. Again, data access is a critical theme of the Dynamic Data Strategy, so being progressive in access strategies makes good sense.
Dynamic Data Strategies
This has been a short note to encourage some thinking about dynamic data strategies. Data is changing, and captured data needs to be refreshed in line with the organization's purpose. Purpose provides an organizational intent, leading to clarity around data use, access, and its optimal refresh rate. From which fall access, storage, and cloud-optimization strategies. At all levels, appropriate data discovery enables data to be fully exploited, ensuring the highest credible return on investment.
Everything starts with purpose. But whatever your organizational purpose, the external hard drive is almost always the wrong answer!
If you want help with dynamic data strategies, data distribution technology, or just want to talk about data, DM or reply to this email. At Sparkgeo, we have been advising large and small organizations on big and small data problems since the cloud was just a few specks of water vapour. We have opinions and can back those up with strategies, data, technology, and code: a data infrastructure company that can execute.