Undefined Symbols for Crate libafl-libfuzzer (with nix)

[nausicaea]

Preface / IMPORTANT

1. You have verified that the issue to be present in the current main branch

Yes, I can confirm the issue on the current main branch.
I've provided a fully reproducible environment and a PoC at my repository: fuzz-ciborium-poc.

Thank you for making LibAFL better!

Thank YOU for creating LibAFL in the first place! <3
Note that the error may be on my side. If I can provide more information, please let me know.

Describe the Bug

Any attempts to build the fuzzing target (see my PoC above) fail with "Undefined symbols" errors. The following symbols are undefined:

• _LLVMFuzzerCustomCrossOver
• _LLVMFuzzerCustomMutator
• _libafl_main

Additional context clues see below.

To Reproduce

1. Download the code from the linked repository: fuzz-ciborium-poc
2. Install the nix package manager
3. Navigate to the repository code and run nix develop to instantiate the development environment
4. Either build or run the fuzzing target with
   cargo fuzz build --fuzz-dir=. parse_cbor or cargo fuzz run --fuzz-dir=. parse_cbor

Expected behavior

I expect the build process to succeed.

Additional context

System Information

I am running an Apple MacBook Air M1 from 2020.

$ sw_vers
ProductName:            macOS
ProductVersion:         15.7.3
BuildVersion:           24G419

Nix Version

$ nix --version
nix (Nix) 2.31.2

Build Output

$ cargo fuzz build --fuzz-dir=. parse_cbor
   Compiling proc-macro2 v1.0.104
   Compiling quote v1.0.42
   Compiling unicode-ident v1.0.22
   Compiling libc v0.2.179
   Compiling serde_core v1.0.228
   Compiling shlex v1.3.0
   Compiling find-msvc-tools v0.1.6
   Compiling equivalent v1.0.2
   Compiling hashbrown v0.16.1
   Compiling rustversion v1.0.22
   Compiling winnow v0.7.14
   Compiling zerocopy v0.8.31
   Compiling indexmap v2.12.1
   Compiling toml_writer v1.0.6+spec-1.1.0
   Compiling toml_parser v1.0.6+spec-1.1.0
   Compiling cfg-if v1.0.4
   Compiling serde v1.0.228
   Compiling arbitrary v1.4.2
   Compiling syn v2.0.113
   Compiling ciborium-io v0.2.2
   Compiling serde_spanned v1.0.4
   Compiling toml_datetime v0.7.5+spec-1.1.0
   Compiling jobserver v0.1.34
   Compiling toml v0.9.10+spec-1.1.0
   Compiling cc v1.2.51
   Compiling libfuzzer-sys v0.4.10
   Compiling libafl_libfuzzer v0.16.0 (https://github.com/AFLplusplus/LibAFL.git#541f2ab6)
   Compiling zerocopy-derive v0.8.31
   Compiling serde_derive v1.0.228
   Compiling half v2.7.1
   Compiling ciborium-ll v0.2.2
   Compiling ciborium v0.2.2
   Compiling fuzz v0.0.0 (/Users/user/fuzz-ciborium)
error: linking with `cc` failed: exit status: 1
  |
  = note:  "cc" "/Users/user/fuzz-ciborium/target/aarch64-apple-darwin/release/deps/rustc0F986c/symbols.o" "-Wl,-rpath,<sysroot>/lib/rustlib/aarch64-apple-darwin/lib" "-lrustc-nightly_rt.asan" "<2 object files omitted>" "/Users/user/fuzz-ciborium/target/aarch64-apple-darwin/release/deps/{libciborium-3802971a3dbfc522,libciborium_ll-1e428eb778db642d,libhalf-4d15cf6bcef657b0,libcfg_if-377cfced8ebf246a,libzerocopy-63c0177f1b8a37d7,libciborium_io-63e217a9a3e83d77,libserde-81112f74d620b63f,libserde_core-0ce0f116188b8bc4,liblibafl_libfuzzer-cbf6165bc4b5726e,liblibfuzzer_sys-bb2e3fceb8cc50fb,libarbitrary-ab00bc0412e9afeb}.rlib" "<sysroot>/lib/rustlib/aarch64-apple-darwin/lib/{libstd-*,libpanic_unwind-*,libobject-*,libmemchr-*,libaddr2line-*,libgimli-*,libcfg_if-*,librustc_demangle-*,libstd_detect-*,libhashbrown-*,librustc_std_workspace_alloc-*,libminiz_oxide-*,libadler2-*,libunwind-*,liblibc-*,librustc_std_workspace_core-*,liballoc-*,libcore-*,libcompiler_builtins-*}.rlib" "-lc++" "-lSystem" "-lc" "-lm" "-arch" "arm64" "-mmacosx-version-min=14.0.0" "-L" "/Users/user/fuzz-ciborium/target/aarch64-apple-darwin/release/build/libafl_libfuzzer-5a9ac33ce1560295/out/libafl_libfuzzer/target" "-L" "<sysroot>/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/Users/user/fuzz-ciborium/target/aarch64-apple-darwin/release/deps/parse_cbor-f47b4ddfbca59b2b" "-Wl,-dead_strip" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: Undefined symbols for architecture arm64:
            "_LLVMFuzzerCustomCrossOver", referenced from:
                _libafl_targets_has_libfuzzer_custom_crossover in liblibafl_libfuzzer-cbf6165bc4b5726e.rlib(ea708c7824d36062-libfuzzer.o)
                _libafl_targets_libfuzzer_custom_crossover in liblibafl_libfuzzer-cbf6165bc4b5726e.rlib(ea708c7824d36062-libfuzzer.o)
            "_LLVMFuzzerCustomMutator", referenced from:
                _libafl_targets_has_libfuzzer_custom_mutator in liblibafl_libfuzzer-cbf6165bc4b5726e.rlib(ea708c7824d36062-libfuzzer.o)
                _libafl_targets_libfuzzer_custom_mutator in liblibafl_libfuzzer-cbf6165bc4b5726e.rlib(ea708c7824d36062-libfuzzer.o)
            "_libafl_main", referenced from:
                _main in liblibafl_libfuzzer-cbf6165bc4b5726e.rlib(ea708c7824d36062-libfuzzer.o)
          ld: symbol(s) not found for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)


error: could not compile `fuzz` (bin "parse_cbor") due to 1 previous error
Error: failed to build fuzz script: ASAN_OPTIONS="detect_odr_violation=0" RUSTFLAGS=" -Cpasses=sancov-module -Cllvm-args=-sanitizer-coverage-level=4 -Cllvm-args=-sanitizer-coverage-inline-8bit-counters -Cllvm-args=-sanitizer-coverage-pc-table -Cllvm-args=-sanitizer-coverage-trace-compares --cfg fuzzing -Cllvm-args=-simplifycfg-branch-fold-threshold=0 -Zsanitizer=address -Cdebug-assertions -Ccodegen-units=1" "cargo" "build" "--manifest-path" "./Cargo.toml" "--target" "aarch64-apple-darwin" "--release" "--config" "profile.release.debug=\"line-tables-only\"" "--bin" "parse_cbor"

[s1341]

I applied the following patch, and was able to get it to run fine on nixos (not nix on darwin):

diff --git a/flake.nix b/flake.nix
index 158ff54..ff9964f 100644
--- a/flake.nix
+++ b/flake.nix
@@ -23,6 +23,8 @@
         devShells.default = pkgs.mkShell {
           nativeBuildInputs = with pkgs; [
             rustEnv
+            llvmPackages.compiler-rt-libc
+            clang
           ];
 
           buildInputs = with pkgs; [
@@ -30,6 +32,7 @@
             cargo-fuzz
           ];
 
+          LIBCLANG_PATH = "${pkgs.llvmPackages.libclang.lib}/lib";
           RUST_BACKTRACE = "errors_backtrace";
         };
       }

[nausicaea]

Thanks for your suggestion. I tried your patch, and I can confirm that this doesn't work with nix on darwin.

[atharva-potdar]

@nausicaea could you check if this works on your end? I don't own a macOS device so my only way to check it was the CI/CD.

[nausicaea]

@atharva-potdar Of course! Thanks for your attempt at fixing it.

Using libfuzzer-sys = { git = "https://github.com/atharva-potdar/LibAFL", branch = "fix-libfuzzer-macos", package = "libafl_libfuzzer" } as dependency, and with the nix flake changes provided by @s1341, I get the following compiler output:

$ cargo fuzz build --fuzz-dir=. parse_cbor
...
Compiling fuzz v0.0.0 (/Users/user/fuzz-ciborium)
error: linking with `cc` failed: exit status: 1
  |
  = note:  "cc" "/Users/user/fuzz-ciborium/target/aarch64-apple-darwin/release/deps/rustczvhCU3/symbols.o" "-Wl,-rpath,<sysroot>/lib/rustlib/aarch64-apple-darwin/lib" "-lrustc-nightly_rt.asan" "<2 object files omitted>" "/Users/user/fuzz-ciborium/target/aarch64-apple-darwin/release/deps/{libciborium-439c285a319260b4,libciborium_ll-77873ed0d754f1cd,libhalf-37263ff7b89e9f3f,libcfg_if-377cfced8ebf246a,libzerocopy-6b881de2d021396d,libciborium_io-63e217a9a3e83d77,libserde-40b97479a7ba90df,libserde_core-0ce0f116188b8bc4,liblibafl_libfuzzer-1cfc0d75426d610f,liblibfuzzer_sys-1a6378d92851a78c,libarbitrary-ab00bc0412e9afeb}.rlib" "<sysroot>/lib/rustlib/aarch64-apple-darwin/lib/{libstd-*,libpanic_unwind-*,libobject-*,libmemchr-*,libaddr2line-*,libgimli-*,libcfg_if-*,librustc_demangle-*,libstd_detect-*,libhashbrown-*,librustc_std_workspace_alloc-*,libminiz_oxide-*,libadler2-*,libunwind-*,liblibc-*,librustc_std_workspace_core-*,liballoc-*,libcore-*,libcompiler_builtins-*}.rlib" "-lc++" "-lSystem" "-lc" "-lm" "-arch" "arm64" "-mmacosx-version-min=14.0.0" "-L" "/Users/user/fuzz-ciborium/target/aarch64-apple-darwin/release/build/libafl_libfuzzer-f04816a00c398b25/out/libafl_libfuzzer/target" "-L" "<sysroot>/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/Users/user/fuzz-ciborium/target/aarch64-apple-darwin/release/deps/parse_cbor-266ef4c64ecae661" "-Wl,-dead_strip" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: Undefined symbols for architecture arm64:
            "_LLVMFuzzerCustomCrossOver", referenced from:
                _libafl_targets_has_libfuzzer_custom_crossover in liblibafl_libfuzzer-1cfc0d75426d610f.rlib(ea708c7824d36062-libfuzzer.o)
                _libafl_targets_libfuzzer_custom_crossover in liblibafl_libfuzzer-1cfc0d75426d610f.rlib(ea708c7824d36062-libfuzzer.o)
            "_LLVMFuzzerCustomMutator", referenced from:
                _libafl_targets_has_libfuzzer_custom_mutator in liblibafl_libfuzzer-1cfc0d75426d610f.rlib(ea708c7824d36062-libfuzzer.o)
                _libafl_targets_libfuzzer_custom_mutator in liblibafl_libfuzzer-1cfc0d75426d610f.rlib(ea708c7824d36062-libfuzzer.o)
            "_libafl_main", referenced from:
                _main in liblibafl_libfuzzer-1cfc0d75426d610f.rlib(ea708c7824d36062-libfuzzer.o)
          ld: symbol(s) not found for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)


error: could not compile `fuzz` (bin "parse_cbor") due to 1 previous error
Error: failed to build fuzz script: ASAN_OPTIONS="detect_odr_violation=0" RUSTFLAGS=" -Cpasses=sancov-module -Cllvm-args=-sanitizer-coverage-level=4 -Cllvm-args=-sanitizer-coverage-inline-8bit-counters -Cllvm-args=-sanitizer-coverage-pc-table -Cllvm-args=-sanitizer-coverage-trace-compares --cfg fuzzing -Cllvm-args=-simplifycfg-branch-fold-threshold=0 -Zsanitizer=address -Cdebug-assertions -Ccodegen-units=1" "cargo" "build" "--manifest-path" "./Cargo.toml" "--target" "aarch64-apple-darwin" "--release" "--config" "profile.release.debug=\"line-tables-only\"" "--bin" "parse_cbor"

[atharva-potdar]

Okay I think it builds correctly now! I looked at common.h and found how to correctly do the weak linking. Created a CI job that tests it like so using your PoC for the fuzz target and it builds just fine, no more unresolved symbols.

cargo fuzz init
cargo fuzz add parse_cbor
# Configuring the Cargo.toml here
cd fuzz && cargo add ciborium && cargo add serde --features derive
# Adding your fuzz target here
cd fuzz && cargo +nightly fuzz build -v --target aarch64-apple-darwin parse_cbor

Link to the job

Edit: Note that this was the Cargo.toml after all configuration:

[package]
name = "build_id2-fuzz"
version = "0.0.0"
publish = false
edition = "2024"

[package.metadata]
cargo-fuzz = true


[dependencies.build_id2]
path = "../crates/build_id2"

[[bin]]
name = "fuzz_target_1"
path = "fuzz_targets/fuzz_target_1.rs"
test = false
doc = false
bench = false

[[bin]]
name = "parse_cbor"
path = "fuzz_targets/parse_cbor.rs"
test = false
doc = false
bench = false

[dependencies]
libfuzzer-sys = { package = "libafl_libfuzzer", path = "../crates/libafl_libfuzzer" }
libafl_targets = { path = "../crates/libafl_targets", features = ["libfuzzer"] }
ciborium = "0.2.2"
serde = { version = "1.0.228", features = ["derive"] }

[workspace]