Changes regarding (debug) symbols in 3.8.x?

[tobiasbrunner]

Building against 3.8.1 required enabling backtraces via libbfd to whitelist functions in our own leak detective (instead of just using dladdr(), which worked before).

A second thing I had to change was whitelisting botan_privkey_load.cold (in addition to botan_privkey_load for older versions) even though I don't see the cold attribute getting used anywhere in the Botan code.

Can you point me to the changes between 3.7.1 and 3.8.1 that caused either or both of these things?

[randombit]

Can you point me to the changes between 3.7.1 and 3.8.1 that caused either or both of these things?

Any change that caused a behavioral difference in this way was not intentional. There were between 3.7.1 and 3.8.1 no change in compiler flags used (with the sole exception relating to LoongArch SIMD instructions, and only used on that architecture). Generating .cold functions seems to be under the control of GCC's option -freorder-blocks-and-partition which (per GCC info pages) is enabled at -O2 and higher on x86. And botan_privkey_load itself had no changes in 3.7.1..3.8.1

In comparing the behavior of 3.7.1 vs 3.8.1 are you using the same version of GCC, same options, etc?

[tobiasbrunner]

Thanks for looking into it.

In comparing the behavior of 3.7.1 vs 3.8.1 are you using the same version of GCC, same options, etc?

I think so. Here is a failing build with 3.8.1 that used image 20250427.1.0, where you can see the missing symbols. And here is a test build I ran with 3.7.1 that uses the same image, but here the symbols were still resolved without compiling with --enable-bfd-backtraces, and there was no .cold suffix, so it succeeded.

[tobiasbrunner]

@randombit By the way, for a more direct comparison, here is a successful build with 3.7.1, and here a failing build with 3.8.1 with only two additional commits, one changes botan_privkey|pubkey_export to corresponding view functions (to avoid deprecation warnings) and the other is the version bump to 3.8.1. The build image was the same as above.

Note that in the "latest (all, clang, yes, no)" case Botan is still built with GCC, only strongSwan is built with clang. The build log for Botan shows the same GCC version: success, failure (some differences can be seen, e.g. 3.7.1 reporting Assuming target x86_64 is little endian, or 3.8.1 disabling sha2_64_x86, but these are probably intended changes). Also, this "all"-build has --enable-bfd-backtraces enabled, so the non-whitelisted symbol with .cold suffix can be seen in the failing build's log.

Compared to that, in the "crypto-plugins (ubuntu-latest, botan, yes)" case, the above option is disabled, so symbols are not resolved in the failing case and whitelisting didn't work either (again, the GCC version is the same for the successful and failing runs).

[tobiasbrunner]

@randombit Any updates?

[tobiasbrunner]

I've tested 3.9.0 and I see the same issues (here without BFD-based backtraces, then with but without .cold symbol whitelisted, then successful after adding both workarounds).

Do you think it's possible to determine the cause somehow? It's really weird that this just started to happen without changes in compiler flags (same compiler, different codebase, i.e. just 3.7.x -> 3.8.x+).

We also have a Debian-based regression test environment where we have several scenarios that use Botan. So far I haven't seen these issues there (neither with bookworm and GCC 12.2, nor trixie and GCC 14.2). It only happens in these GitHub Action images with GCC 13.3. The configure options are basically the same, the only difference is that we build with --amalgamation on GitHub Actions (this reduced the build time by almost 50% back in 2018, not sure if that would still be the case nowadays). But I tried disabling it and it didn't make a difference.