Releasing the same jar with different version numbers can confuse security software

[dermot-hardy]

The com.squareup.okhttp3:okhttp jar has been completely identical since 5.0.0-alpha.15.

These are the SHA1 digests for example:

• com.squareup.okhttp3:okhttp:5.0.0-alpha.14 → c59864766ffc0d0dd4394ec0af5e3b60bf83d10f
• com.squareup.okhttp3:okhttp:5.0.0-alpha.15 → 6891a4fd19bcab6f69fe091c2bc6077edec5258e
• com.squareup.okhttp3:okhttp:5.0.0-alpha.16 → 6891a4fd19bcab6f69fe091c2bc6077edec5258e
• com.squareup.okhttp3:okhttp:5.0.0-alpha.17 → 6891a4fd19bcab6f69fe091c2bc6077edec5258e
• com.squareup.okhttp3:okhttp:5.0.0 → 6891a4fd19bcab6f69fe091c2bc6077edec5258e
• com.squareup.okhttp3:okhttp:5.1.0 → 6891a4fd19bcab6f69fe091c2bc6077edec5258e

If you lookup the digest on https://search.maven.org/ or https://central.sonatype.com/ the latest version does not always come up first, or at all sometimes:

https://search.maven.org/

https://central.sonatype.com/

Security software can report than an alpha version is being used when it is actually one of the stable versions.

I realize this jar just contains some Kotlin metadata, but might it be possible to adjust the manifest to include the version number, or to make any change at all, so that new releases of the jar are not binary identical?

[JakeWharton]

This becomes a pretty significant request that I think I'm opposed to. I'll have to see what others think.

For me:

• OkHttp is by no means our only Kotlin multiplatform project. We have tens of them producing hundreds of these common jars. How do we ensure this is done for all projects and modules moving forward?
• Binaries which contain code but which haven't changed from release to release should also produce the same hash. And that's independent of the use of Kotlin multiplatform or even Kotlin in general.
• This actually seems like a desirable feature to me. Nothing has changed and our pipeline is capable of producing reproducible binaries. I am of the opinion that the final release candidate of a project should be bit-for-bit compatible with the stable version. That's a bit hard, especially when a project knows a bit about itself (such as its version). But it's more a criticism of the idea that a hash can be relied upon to reverse map back to a version.

For the search, https://search.maven.org/ is effectively dead. We should provide the feedback to https://central.sonatype.com/ that it should probably favor the most recent version. Of course, this means it will potentially show alphas if we were to release them again, but I think that's okay. Just so long as it's not favoring the oldest match.

[dermot-hardy]

Generally speaking JARs contain metadata in their manifest that include details like the version of the JAR (normally in the Implementation-Version entry). Common build tools like Maven and Gradle will automatically populate these and they will typically also embed even more metadata such as dependency information. I think that is why this is so unusual.

[JakeWharton]

Sure, I get that. But I think if you want to pursue this you may need to go upstream to Kotlin and its Gradle plugin and maybe convince them that it's their responsibility (and I'm still not convinced it really is).

The empty common jars are not the only which are susceptible to this. In Okio (an upstream dependency of ours), its version 3.10.1 and 3.10.2 versions contain bit-for-bit matching binaries (.klib) of its fake file system for iOS ARM 64 targets. Of course, those are not .jars.

The jar specification does not require Implementation-Version. Having the common build systems add it automatically is usually convenient. But build systems like Bazel, however, do not do this. If we someday switch to Bazel or some new shiny thing, we potentially re-introduce this problem all over again.

[yschimke]

I'm not against adding this if it's useful.

I think I am against adding it, if security audit software is flagging it and causing spurious reports.

The onus should be on the analysis tools to get this right. We have similar things with spurious CVE reports on libraries that make no sense.

If we now have 2 incorrect automated processes that run, generate work for multiple open source projects, it's just a toil.

[dermot-hardy]

I'm not against adding this if it's useful.

My personal opinion is that this sort of metadata in JARs is useful, in the same way that having version resources embedded in executables or dlls is useful. In a way you could argue that JARs are the JVM equivalent of DLLs. And yes technically you could build a DLL or an EXE without a version resource, but I don't think anybody would see it as a good practice.

But it's up to you guys at the end of the day. Feel free to close this if you don't agree that it should be done or feel that it needs upstream tooling changes to be able to do it.

Thanks,
Dermot