Semantic Versioning

[cmwaters]

We need to set and follow a consistent version scheme that indicates to node operators the varying importance of upgrading. Noticeably there are two signals worth producing:

1. A coordinated network wide upgrade. There are upcoming breaking changes to the protocol and users must upgrade to the latest version as soon as possible. [Major Release]
2. Whether a non-breaking change is merely an optimisation or feature [Minor Release] and upgrading is optional to the node operator or that it is a fix [Patch Release] and should be upgraded immediately.

As a follow up discussion it may be worthwhile to further separate the versioning of Celestia's libraries and clients from Celestia's nodes (consensus and networking). We are already looking at creating separate go.mod's for popular packages that we believes others will use. This potentially runs orthogonal to the semantic versioning we apply to the binaries that are released.

This conversation should ideally involve celestia-node maintainers given the likelihood that the binary is eventually combined.

We should also be mindful of the amount of versions we plan to currently maintain and make sure this is correctly communicated to the public.

As a proposal, I would suggest remaining with the three numbers X.Y.Z [Major.Minor.Patch]. Major signifying breaking network compatibility change. Patch indicating non-breaking fix that node operators should immediately upgrade to and lastly, Minor, indicating optional non-breaking features and improvements.

Ideally every minor release would each be supported and updated whenever there was a fix i.e. I can update from v1.0.0 to v1.0.1 instead of needing to jump to v1.2.1. However, this implies a lot of potential maintenance work for the engineering teams. If we are to only support the latest minor release, and require users to constantly update, it begs the question if three numbers are even necessary. Perhaps we make the Minor and Patch numbers increment independent of one another (i.e. if you see a patch update you should immediately update but if you see a minor update you don't necessarily need to).

This also brings in to question how regular we plan to upgrade the binary.

[evan-forbes]

I like this and it makes sense to me. Curious to see what other teams think.

Since this differs slightly from normal semver, should we name it something else? Like "Network Semver" or something similar?

[cmwaters]

Yeah I have quite a loose interpretation of SemVer whereby API breaking refers to both from node to node (i.e. network or consensus breaking changes) as well as client/server breaking changes (i.e. gRPC or RPC)

I do think any of the libraries (i.e. in pkg) should use the go mod's versioning. There's the argument that we move this out into their own repository so we can cut library API changes different to node API changes

[MSevey]

we can cut library API changes different to node API changes
This is definitely helpful, and could turn an app major upgrade into just a library minor upgrade.

I guess my main sticking point is to avoid being specific as to which breaking change is a major version bump.
Any and all breaking changes should be major version bumps.
As a team we just need to make sure we understand all those interactions to properly version the software for a good devX for the community.

[staheri14]

Yeah I have quite a loose interpretation of SemVer whereby API breaking refers to both from node to node (i.e. network or consensus breaking changes) as well as client/server breaking changes (i.e. gRPC or RPC)

I do think any of the libraries (i.e. in pkg) should use the go mod's versioning. There's the argument that we move this out into their own repository so we can cut library API changes different to node API changes

Is the process similar to how we handle the nmt module, which has its own repository and versioning? When we implement a significant change to nmt which may affect its lib API, we release it with a major version increase. However, for the celestia-app, even if we update to this new nmt version, as long as it doesn't affect the node-to-node or client-server APIs (plus consensus rules), we can issue a release with just a minor version upgrade. That sounds like a feasible and clean approach.

[cmwaters]

Any and all breaking changes should be major version bumps.

Yes I agree. The point however is that if we couple all these different user groups into the same release it can become confusing to them whether their usage of the repo has broken (thus the argument for separation).

[MSevey]

The point however is that if we couple all these different user groups into the same release it can become confusing to them whether their usage of the repo has broken (thus the argument for separation).

This is true for current semver. While a major version bump indicates a breaking change, it doesn't mean that it is breaking for any and all users. Only those that were using the feature that was impacted by the breaking change.

Deviating from semver might make it easier to track network upgrades, but it makes it harder to track other breaking feature changes. So it is just swapping one challenge for another.

I think the handling of coordinating the network release with the major releases is something that can be handled by an internal team process. For example, most feature changes can be done in an additive way, so that they don't need to break existing functionality. By doing this, the team could then coordinate removing outdated functionality (breaking change) with network breaking changes. This keeps the complexity/nuance of the version within the team, vs in the public facing versioning.

Or we just KISS and follow traditional semver :-) simplicity scales.

[rootulp]

Since celestia-app can still be used as a Go dependency, the proposed network SemVer has the downside that consumers of the celestia-app Go dependency will get no guarantees about when a version is breaking. Therefore, I think celestia-app releases should strictly follow SemVer at least until celestia-app isn't a consumable Go dependency.

After that, I think we can revisit the proposal to adopt network SemVer but I don't see a compelling reason to include the network version or app version in the release version. As far as I can tell, that's less flexible than having release versions based on breaking changes which may happen more frequently than network version or app version bumps.

As an engineer, I am conditioned to treat X.Y.Z as a SemVer number. I think we should be extremely careful about repurposing that format for the proposed network SemVer because I expect consumers of celestia-app to make incorrect assumptions when they see a X.Y.Z release number. If maintainers feel strongly about adopting a network SemVer format for releases, I think we should deviate from the X.Y.Z format to distinguish it from the real SemVer.

Out of curiosity, are there any other projects that have adopted something like the proposed network SemVer?

[rootulp]

We agreed to use SemVer for celestia-app releases.