Add simple fuzz-harness

[nathaniel-brough]

This PR is based on the proposal in #1538 inlined below, to integrate yoga with oss-fuzz

Hey yoga team,

I've recently become interested in yoga. I'd like to suggest and champion an effort to set up some basic fuzz-testing and combine it with google/oss-fuzz for continuous fuzzing. I'm fully aware that you are very busy people and I don't want to overload your review/maintenance capacity. Is this a bad time to discuss potential security/reliability improvements?

If you're not familiar with fuzzing or oss-fuzz I've included a few brief notes below.

Benefits of Fuzz-Testing

• Dynamic Code Testing: Fuzz-testing challenges systems with unexpected data, aiming to identify vulnerabilities or bugs. It’s akin to an exhaustive stress-test for the code.
• Detecting Hidden Vulnerabilities: It can uncover potential weaknesses that may not be evident in routine tests.
• Continuous and Automated Testing: With tools like Google’s OSS-Fuzz, fuzz-testing can be automated, running continuously on distributed systems, ensuring daily resilience checks.

Google/oss-fuzz for Continuous Fuzzing

• Automated Fuzzing: OSS-Fuzz undertakes comprehensive fuzz-testing daily on a distributed cluster.
• Detailed Reporting: OSS-Fuzz offers exhaustive reports in case of detected anomalies, enabling effective action.

I’d be more than happy to lead the effort in integrating fuzz testing with the yoga and assist in any way required.

Prior integrations

There have been a number of previous integrations completed with facebook repositories and google/oss-fuzz including;

• facebook/time
• facebook/zstd
• facebookexperimental/starlark-rust (this was me)
• facebook/proxygen
• facebook/hermes
• facebook/rocksdb

As a proof of concept I created a couple of super simple fuzz harnesses in #1537.

NOTE: Adding fuzz-testing and integrating with google/oss-fuzz was previously suggested here #1055 and was rejected. I think I've addressed the concerns raised in the first PR. While the original PR contained what was probably a higher performance fuzzer, the new fuzzer should be easier to integrate and doesn't introduce multiple sources of truth.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
yoga-website-next	⬜️ Ignored (Inspect)	Visit Preview		Jan 13, 2024 1:38am

[NickGerleman]

Thanks for the contribution! I read through #1538 and looked through this code, and can definitely see the value here.

My main ask, if that if we want code which fuzzes Yoga to live in the Yoga repo, the Yoga repo should also have automation to build (and ideally run) the fuzzer. E.g. add GitHubs actions workflows exercising what this PR adds.

Otherwise, the changes here look good. If you could inline your proosal, or link to it, in the PR body, that would be helpful when looking back at commits.

[NickGerleman]

I read more about google/oss-fuzz, and see that they would be running the fuzzer continuously.

So, I think we would just want some build automation for this, so we don't accidentally break it without noticing. I think that would look like adding a job to https://github.com/facebook/yoga/blob/main/.github/workflows/validate-cpp.yml

[nathaniel-brough]

Looks like I need to fix some formatting.

I read more about google/oss-fuzz, and see that they would be running the fuzzer continuously.
So, I think we would just want some build automation for this, so we don't accidentally break it without noticing. I think that would look like adding a job to https://github.com/facebook/yoga/blob/main/.github/workflows/validate-cpp.yml

There is an oss-fuzz specific github action, which can be useful for this sort of thing. It'll build/run the fuzzers for a short period on every pull request. The goal being to catch shallow bugs before they merge, with the additional benefit that it'll prevent regressions as it shares the corpus from the main oss-fuzz cluster.

[nathaniel-brough]

I've gone with a simple CI script as you've suggested for now, we can look at using the oss-fuzz github action in future.

[NickGerleman]

@silvergasp I hit the button to run, and looks like the action currently runs into a linker error.

[nathaniel-brough]

@silvergasp I hit the button to run, and looks like the action currently runs into a linker error.

Hmm that's odd I was just running the fuzzer on my machine yesterday. I'll experiment a bit with the CI on my branch so that I can hit the CI button myself. I'll ping you again here once I've fixed the problem.

[nathaniel-brough]

Ok it took a little fiddling but I think I've got the incantation correct now. The issue was the the compiler runtime that the fuzzer uses is (at least on ubuntu) linked against libstdc++ but yoga is linked against libc++ on the CI, this lead to some conflicts and missing symbols when linking. The things I tried;

• Switching everything over to use stdc++ instead of libc++. This lead to compiler errors in gtest.
• Building libfuzzer again from source and linking against libc++ instead of libstdc++. This isn't difficult, but adds build time and complexity unnecessarily.
• Switching over to stdlibc++ and then only building the fuzzer's not any of the unit tests. This is the solution that I settled on.

TLDR; this is green now :)

[NickGerleman]

Thanks for the explanation.

In https://github.com/facebook/yoga/blob/6a72a7a20b05cf646ae43d63994c9f6dbb5c123c/.github/actions/setup-cpp/action.yml we usually either setup the machines as gcc and stdlib++ or clang and libc++.

Does this work if we keep to that pattern, and use GCC as OS default? Or does it need to be Clang?

[nathaniel-brough]

Thanks for the explanation.

In https://github.com/facebook/yoga/blob/6a72a7a20b05cf646ae43d63994c9f6dbb5c123c/.github/actions/setup-cpp/action.yml we usually either setup the machines as gcc and stdlib++ or clang and libc++.

Hmm where is the stdlib++ build setup, I wasn't able to get a working complete build using stdlib++, i.e. without gtest compiler failures.

Does this work if we keep to that pattern, and use GCC as OS default? Or does it need to be Clang?

As far as I'm aware libfuzzer only works with clang, as it is an llvm project. There are other fuzzing engines that support gcc e.g. AFL. But google/oss-fuzz essentially provides adapters for libfuzzer based fuzzers and it'll link in AFL/Centifpede/honggfuzz under the same interface of libfuzzer. As far as I'm aware oss-fuzz only uses clang.

[NickGerleman]

Ah, thanks for the context.

Hmm where is the stdlib++ build setup, I wasn't able to get a working complete build using stdlib++, i.e. without gtest compiler failures.

We use stdlibc++ alongside the GCC reference build, but set Clang build to use libc++, since it is more often tested with it. As you noticed, this can conflict with any libraries installed by the system (on Linux).

That change was actually made, when Ubuntu LTS bumped some version of Clang or libstdc++, so that importing the chrono header under C++ 20 was enough to produce a compile error. Ubuntu LTS is using some older compilers/libraries where C++ 20 was not as mature, but it seems like they don't really try hard to support the combination (e.g. why we see gtest failure).

Fuzz here runs into that kinda unsupported state right now, but it does build, and might be less error-prone if we use OSS fuzz action to build outside this infra.

[facebook-github-bot]

@NickGerleman has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[NickGerleman]

I missed this the first time around, but YGConfigRef and YGNodeRef are themselves just pointers. That also makes const apply to the pointer value, instead of the underlying node/config.

Needed to fix a quick license header to make internal linter happy, but I'm going to make a quick ninja-edit to make that clearer.

[NickGerleman]

Same for a couple other quick repo conventions around naming

[nathaniel-brough]

No worries. Let me know if there is anything you'd like me to do here :)

[facebook-github-bot]

@NickGerleman merged this pull request in 1a8b80a.