[Proposal] Hosting Update

[hyperupcall]

Description of the feature / enhancement.

Me and @madskristensen have been discussing how to improve the hosting. We're looking to move towards hosting that more leverages GitHub to improve reliability. There are a few proposals:

Proposal 1

In catalog.json, swap all https://json.schemastore.org/* with https://raw.githubusercontent.com/SchemaStore/schemastore/master/src/schemas/json/* URLs.

Proposal 2

Create extra GitHub repositories for every subdomain needed (limitation of GitHub). For each of these repositories, publish to GitHub actions and configure the schemastore.org DNS servers (through a CNAME record) to:

1. "Direct" json.schemastore.org requests to the GitHub Pages of SchemaStore/schemastore. On every commit on master, we can run an action that publishes to GitHub Pages. Must copy schemas so each one is available at, for example /ava.json and /ava

2. "Direct" www.schemastore.org requests to the GitHub Pages of (the new) SchemaStore/schemastore-www. The catalog is hosted at www.schemastore.org/api/json/catalog.json, so for every change in catalog.json on master in SchemaStore/schemastore, it will need to be copied (and committed) over

3. "Direct" schemastore.org requests to the GitHub Pages of (the new) SchemaStore/schemastore-apex. This repository either:

• A. Has the same content as www.schemastore.org
• B. Has HTML redirects for all the routes under www.schemastore.org
• C. Has an HTML redirect for only schemastore.org/index.html
• D. Same as C, but also a copy (no redirect) of api/json/catalog.json

It's not uncommon to omit the www part of URLs (I certainly do). The old site automatically added www to everything, but that can't be done server-side anymore. I'm learning towards C or D. This apex URL is used in the wild and users currently have the expectation that going to schemastore.org will show the up-to-date site. catalog.json can also be copied over, but I poked around at a few projects with links to the catalog, and they all use URLs with the www subdomain.

Implementation Timeline

Phase A:

• Create schemastore-{www,apex} GitHub repositories
• For all three repositories, set Settings > Pages > Build and deployment > Source to "GitHub Actions"
• Add configuration/code to all three repositories to get the desired behavior

Phase B:

• For all three repositories, hook up GitHub pages with DNS for a testing domain
• Edit DNS records for schemastore.org, wait at least the TTL, then tell GitHub to use schemastore.org

Proposal 3

1. Do Proposal 1

2. But also "Direct" json.schemastore.org requests to the GitHub Pages of SchemaStore/schemastore (no extra repositories)

Summary

Proposal 1 is much simpler. But proposal 2 covers nearly all uses of SchemaStore (including direct reference like "$schema": "https://json.schemastore.org/ava.json"). Proposal 2 won't break the implicit contract that json.schemastore.org links are always up to date. I also noticed that both will allow schemas to be transferred over HTTP/2 (thanks to GitHub).

Proposal 1 is simpler and follows the original intent of the catalog. Proposal 2 does a lot of work to be very backwards-compatible, but is too complex. Proposal 3 is a hybrid approach that attempts to follow the original intent of the catalog, but also keeps json.schemastore.org/* URLs up to date for current consumers that access it directly.

Are you making a PR for this?

Yes

[madskristensen]

What I like about proposal 1 is that it makes it clear that you can host your schema anywhere and doesn't have to be part of the SchemaStore repo. It makes it look like the catalog is primarily a registry of schemas from around the web, which was always the intent.

[hyperupcall]

@madskristensen Thanks for your feedback! I added a third proposal, which is based on proposal 1, but adds only the most important aspect of proposal 2

Now I'm thinking, after this issue has been closed, maybe the schema checks that we have here can be extracted to a separate npm package, "schemacheck", to encourage people to host their own schemas in their own repositories?

[madskristensen]

I like proposal 3 a lot. Thanks for thinking this through

[madskristensen]

@hyperupcall if we update all schema references in catalog.json to point to the raw GitHub URL, do we need to update all the schemas' id attribute to match that same URL? Or remove the id attributes from individual schema files altogether.

[hyperupcall]

if we update all schema references in catalog.json to point to the raw GitHub URL, do we need to update all the schemas' id attribute to match that same URL?

@madskristensen Nope that change shouldn't be needed at all!. A few years ago I made several PRs that added id (or $id) to all of the schemas and enforced that in cli.js. That was an intentional change that specifically made it so that we can do these sorts of URL changes worry-free, as long as $id is kept the same.

[hyperupcall]

But in the case of converting the $ids and $refs from json.schemastore.org/* to raw.githubusercontent.com/.../* (not in catalog.json), I'm not fully certain that it will lead to no breakages at all. Not to mention that it would make schemas more difficult to read if they have a lot of $refs.