New data frontend strategy [map]: lightweight data layers for every entry point with location component (focus on non-binary static files) #48
Description
- Some data formats related to GIS
- GeoJSON Related
- https://datatracker.ietf.org/doc/html/rfc7946
- https://geojson.org/
- https://geojson.org/geojson-ld/
- Very relevant for our use cases (we're already mapping things to RDF)
- https://geojson.org/geojson-ld/
- A Uniform Resource Identifier for Geographic Locations ('geo' URI)
- Indirectly related:
- Other links related to organize entire libraries of datasets by location
Fact: data exchange often have some location component. It might not be easy to relate prepare data, and may not be the main focus of what user want, but is possible to not only key in data by the used numerical taxonomy Numerordĭnātĭo, but by location.
The idea here would be, in addition to the tabular formats which both can work as plain CSVs (but also via frictionless can be loaded into databases, as per #37) is something which could be loaded on tools that typically would use map. I think that some tools that work with graph (maybe plugins for Protege) would already do something, but this would be on #41.
Challenges
(Likely major issue at standards level) interlink data related to location without replicate geometry on every file
I might be wrong, but all I'm seeing (maybe because most GIS tools are strongly focused on Desktop and strong numerical precision) that they tend to allow attach data to administrative locations, BUT... the way most of then do it is duplicating the geometries when related data comes from several sources!
Interoperability to change location geometry references and data for same topic easily
Assuming the issue of allow (at client-level, likely these desktop programs or, with some documentation, web interfaces) to optionally not duplicate geometries every time static file contains related data, we come to the second point: allow end users change both.
There's reasons (both for changes on precision of geometries, or maybe because new data might have small variation) that users might not using exact same geometry, yet still relevant to interlink the data.
Potential first approach
Unless we resort to XML files (or would use geopackage, but this is binary format, not really what we want for things that need to allow user change parts) one close alternative would be... Geojson.
GeoJSON, GeoJSON-LD, ...
GeoJSON is by far the most well supported non XML format / non tabular format. The main complain (and is the fact TopoJSON was created) is that tend to have higher file size and take more memory from the user than binary formats.
However, while we might test if can work in practice, I think we can at least start generating geojson and mark the properties associated with each feature in some way that it could be understand as stricter RDF (not mere text). We already been doing this, so it would make it on a JSON-like format.
Maybe create "dummy" points and mark the real geometries with extensions to GeoJSON
GeoJSON itself does not allow to reuse geometries from other places (but we might use something based on JSON Schema or RDF to signal this), but we could at least, for clients that would not be able to understand, create dummy points, like the centroind of an administrative area.
One advantage of this approach is that GeoJSON with only single points would take very low extra weight, since most of the file size would actually be what user really want as metadata. This extra heigth also would be such that we would likely not have relevant benefit of using topojson (which I think mostly use arcs to simplify things, but no metadata is changed).
Strategies on how the "dummy" points could be replaced:
- Either command like instruction (which could be generated as part of user documentation) or online tools could contatenate both the real geometries (most likely used for geometries which would be wasteful to repeat on every dataset, such as administrative regions).
- This approach also allows for the benefict of users know how to merge several related data from different subjects on a "final" file.
- At client-side, tools be aware of the exchanged data
- Web interfaces this would means implement this (like with javascript or something)
- One obvious advantage (even for tools that would allow import several geojson layers) is much lower memory usage
- Desktop tools learn how to interlink the files
- Not something sort term (but could make sense in long run) if already have considerable amount of data
- Web interfaces this would means implement this (like with javascript or something)
Not focus here
On a quick look, it seems that there's very complex and detailed all-in-one servers, like open source geonode https://geonode.org/ or MapServer https://www.mapserver.org/, which would allow to deploy pretty much everything. The analogy would be a CKAN, but strongly focused on maps. They do use documented protocols, but unless we find ways to make very simplistic automated generation of static files to emulate then, is out of scope we try to create production-level server just as frontend for the data.
We can, however, automate or document how to ingest data. But at this point the start here is just make things work at client-side and server simply have static files in predictable ways. This is why we cannot rely too much on a public data warehouse for everyone.