Reproducible Builds fail with v1.4.2

[IzzySoft]

At IzzyOnDroid we support Reproducible Builds (see: Reproducible Builds, special client support and more at IzzyOnDroid). Trying for yours, I was able to successfully generate the APK using ./gradlew assembleRelease, but the resulting APKs were not identical:

  -rw-r--r--  0.0 unx     4925 b-     4925 stor 1981-01-01 01:01:02 5757ea25 assets/dexopt/baseline.prof
- -rw-r--r--  0.0 unx      405 b-      405 stor 1981-01-01 01:01:02 ae0ff18d assets/dexopt/baseline.profm
+ -rw-r--r--  0.0 unx      402 b-      402 stor 1981-01-01 01:01:02 a2afe7e6 assets/dexopt/baseline.profm
  -rw-r--r--  0.0 unx  9490012 b-  3691127 defN 1981-01-01 01:01:02 fe53024c classes.dex

Comparing the two profiles, you can see they have the same contents just in different order:

 profile_idx=0
+  profile_key='classes3.dex'
+  num_type_ids=6650
+  num_class_ids=0
+profile_idx=1
   profile_key='classes2.dex'
   num_type_ids=10230
   num_class_ids=0
-profile_idx=1
+profile_idx=2
   profile_key='classes.dex'
   num_type_ids=10081
   num_class_ids=464
-profile_idx=2
-  profile_key='classes3.dex'
-  num_type_ids=6650
-  num_class_ids=0

This is something easy to fix, but needs to be fixed on your end.

We'd appreciate if you could help making your build reproducible again. We've also prepared some hints on reproducible builds for that.

Looking forward to your reply!

[puutaro]

In other words, Should I version up after re-build release code ?

[IzzySoft]

Yes, please make that part of a new release then (not replacing and existing one). A ping once that's ready is much appreciated then 😃

[obfusk]

This is something easy to fix, but needs to be fixed on your end.

I would recommend upgrading Android Gradle Plugin to 8.1.0 or later instead of using the workaround.

[IzzySoft]

Hope you don't forget before making the next release? 😉

[puutaro]

Please tell me the significance of Reproducible Builds.
Also, why hasn't this been raised as an issue until now?

And, although you answer like bellow

Yes, please make that part of a new release then

Is it not necessary to update the versionCode ?
In other words, is it okay to just put the built aliased apk?

[IzzySoft]

Please tell me the significance of Reproducible Builds.

Longer explanation behind the links provided in the opening comment. But in short: it shows the app was really built from the code it claims to be built from – with nothing "injected" or taken out. So it's a security feature.

Also, why hasn't this been raised as an issue until now?

No funding 😝 Honestly: I'm drowning in tasks and cannot be everywhere simultaneously, apologies.

Yes, please make that part of a new release then
Is it not necessary to update the versionCode ?

Don't the two go together? You should always increase versionCode when making a new release – even if it's just a "maintenance release". As Android uses versionCode to tell versions apart, only APKs with an increased versionCode will be considered updates.

In other words, is it okay to just put the built aliased apk?

For a cross-check (if all worked out) we indeed would only need the APK (and the commit it was built from). For distribution, we'd need a new release (with increased versionCode) as we cannot replace the existing one (wouldn't work as update for those already having installed the latest release – and except in rare cases (e.g. "broken APK"), one does not replace what's already distributed but provides a new release, so everyone receives it.

[puutaro]

I upload CommandClick-1.4.2-release-reproductive.apk.
Ah, thank you very much.

[IzzySoft]

That was built from which commit? For RB, we must match that.

[puutaro]

What is mean?

That was built from which commit?

Because, I built from not certain commit, but repository itself. I don't know how to build certain commit.
I also assume that you agree with my questions below.

In other words, Should I version up after re-build release code ?

[IzzySoft]

Because, I built from not certain commit, but repository itself. I don't know how to build certain commit.

You create a tag and a release, right? May I suggest a workflow like this (example for the current release):

# adjust versionCode to 1.4.2 here and increase versionCode, then
# commit the latest changes; then, from a clean tree without further modifications:

git tag v1.4.2
./gradle assembleRelease  # or whatever you use to build; running it at this point means
                          # building from the just created tag, which points to the proper commit

# now you head to Github to create the release bound to the v1.4.2 tag you created above

Should I version up after re-build release code ?

Does the above example answer this? Best practice is AFAIK to adjust the version right before releasing, as the example does.

[puutaro]

My master head is ac46857.
I create from above commit.
However, at present, there may be more support than this, so please wait until the next release.
Becuase, I don't really understand the utility of reporoductive builds, and I can't issue a new release just for that purpose.　　

With that in mind, the question is, what is the problem with not being able to do a reproductive build?　　
Also, in order to do a reproductive release, it seems that when issuing a tag, set the commit for the build as the head, but when developing a feature, the commits are distributed across multiple commit. Physically putting it all together is cumbersome and difficult.
Specifically, I think there are files etc. that you would like included in the head, could you tell me about them?

[IzzySoft]

and I can't issue a new release just for that purpose.

Be welcome to ping us when you're ready to release (i.e. have the commit ready which you will tag then, and an APK built from that commit) so we can test it before you tag. If the test succeeds, then just create the tag at that commit and attach that APK, and it should be fine.

what is the problem with not being able to do a reproductive build?

The confidence that your app was really built from the code it claims to be built from is "a matter of belief" then – while with a successful RB, there's hardly any doubt as it was proven.

Physically putting it all together is cumbersome and difficult.

I don't think so. When you're about to release, just stick to the workflow:

• build your last test release to make sure everything works as intended
• make sure all changes have been committed and the tree is clean
• build the release APK (maybe test again and, should it fail, back to step 1)
• tag that commit
• create the release at that tag and attach that APK.

This ensures the APK matches the commit you tagged as release – what we call the "first basic rule" in our hints on reproducible builds.

Specifically, I think there are files etc. that you would like included in the head, could you tell me about them?

Sorry, but I don't understand the question: what head? Where Git's HEAD is is irrelevant for RB as we check out the commit the tag points to. The question is not what is at HEAD, but what is at the commit the APK was built from (which should be the commit the tag points to). And yes, all files needed should be committed to the repo; files only existing in your local tree would make RB impossible.

That said:

My master head is ac46857.
I create from above commit.

That commit is 10h old. The APK provided above is 2 days old, so it cannot have been built from that commit – which means I cannot use that combination to verify RB. To verify RB, both sides must build from the very same commit (which usually is the one the tag points to, as outlined).

[IzzySoft]

A pointer, @puutaro: if you build from a dirty tree (like with the current release: you've committed ffbf018, then changed versionPatch in build.gradle.kts, then built without committing your changes before and only committed them after the build with 63f08c2 – then we have no chance to get the app RB again. Just a bunch of work in trying to manually fix the recipe:

git checkout 63f08c2ba56091caebc28580904894768fa131c4
git reset --soft ffbf0185148ccb6cb34b6e4c0050ad0ec1825113
./gradlew assembleRelease

Even if that works now (update: which it luckily did! So that was all it took, just commit before building and it would work!), it breaks the next build and we'll have to adjust again. So may I kindly ask to remember this for the next release? Thanks a lot in advance!

[IzzySoft]

PS, while I'm here:

! repo/com.puutaro.commandclick_84.apk contains signature block blobs: 0x504b4453 (DEPENDENCY_INFO_BLOCK; GOOGLE)

This can easily be avoided with a minor addition to your build.gradle:

android {
    dependenciesInfo {
        // Disables dependency metadata when building APKs.
        includeInApk = false
        // Disables dependency metadata when building Android App Bundles.
        includeInBundle = false
    }
}

For some background: that BLOB is supposed to be just a binary representation of your app's dependency tree. But as it's encrypted with a public key belonging to Google, only Google can read it – and nobody else can even verify what it really contains. More details can be found e.g. here: Ramping up security: additional APK checks are in place with the IzzyOnDroid repo.

Thanks in advance, @puutaro!

[puutaro]

Altough I think for a long time, I'm thinking of leaving your repository for now.
That is because, CommandClick is chaos state now.
The specifications of my app are undergoing rapid changes.
So, I feel like it's becoming difficult to respond to your requests.
Once the development has calmed down, we would like to consider posting it on various repositories, including yours. What do you think?
Thank you for your pointint out.