Skip to content

Conversation

@hobu
Copy link
Contributor

@hobu hobu commented Dec 31, 2025

The GDAL Virtual File System is the extremely powerful indirection that allows users to abstract data location in complex data processing pipelines with GDAL. While it works generically with HTTP-based endpoints with vsicurl, GDAL's specializations such as vsis3 and vsis3_streaming for S3, vsiaz and vsiaz_streaming for Azure, and vsigs vsigs_streaming for Google bring these object stores to GDAL. vsis3, vsigs, and vsiaz are super convenient, flexible, and performant for users of huge geospatial data on the cloud. It's the reason why each cloud shows off GDAL in their documentation that demonstrates how to process geospatial data with their storage platform.

This convenience comes at a significant maintenance cost to the project. Maintenance burden in the form of keeping up with cloud API changes and additions, normal bug and performance fixes, additional software and API complexity for GDAL, and user support load related to cloud-based VSI layers all add up. Lack of resources to keep up with the maintenance burden of these capabilities was indeed one of the factors that instigated the creation of the GDAL Sponsorship Program, and for a number of years the large clouds were financially supporting their upkeep through it. This is no longer the case going forward, however.

GDAL's support of cloud-specific VSI capability is great for users (fast and convenient) and great for the clouds (more usage sells more storage/network/accesses), but it's not so great for the GDAL software project itself. The project is caught in that it can't remove this capability or its users will be harmed, but maintaining them in perpetuity is expensive opportunity cost over other project priorities.

So what do we do about it? For now, let's start by favoring promotion of cloud APIs in our examples and docs of the organizations that actually sponsor the project. We can show how convenient and powerful it is to use the VSI cloud API specializations of organizations that actually sponsor their upkeep without impacting our users.

Do you have a sales contact with a cloud that once sponsored GDAL? Do you pump 100s of terrabytes of content through a cloud that once sponsored GDAL? Please let these organizations know the GDAL VSI specializations matter a lot to you, and their sponsorship enables you to buy more of their product.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants