Proposal: Integrate with google/oss-fuzz for continuous fuzz testing?

[nathaniel-brough]

Hey yoga team,

I've recently become interested in yoga. I'd like to suggest and champion an effort to set up some basic fuzz-testing and combine it with google/oss-fuzz for continuous fuzzing. I'm fully aware that you are very busy people and I don't want to overload your review/maintenance capacity. Is this a bad time to discuss potential security/reliability improvements?

If you're not familiar with fuzzing or oss-fuzz I've included a few brief notes below.

Benefits of Fuzz-Testing

• Dynamic Code Testing: Fuzz-testing challenges systems with unexpected data, aiming to identify vulnerabilities or bugs. It’s akin to an exhaustive stress-test for the code.
• Detecting Hidden Vulnerabilities: It can uncover potential weaknesses that may not be evident in routine tests.
• Continuous and Automated Testing: With tools like Google’s OSS-Fuzz, fuzz-testing can be automated, running continuously on distributed systems, ensuring daily resilience checks.

Google/oss-fuzz for Continuous Fuzzing

• Automated Fuzzing: OSS-Fuzz undertakes comprehensive fuzz-testing daily on a distributed cluster.
• Detailed Reporting: OSS-Fuzz offers exhaustive reports in case of detected anomalies, enabling effective action.

I’d be more than happy to lead the effort in integrating fuzz testing with the yoga and assist in any way required.

Prior integrations

There have been a number of previous integrations completed with facebook repositories and google/oss-fuzz including;

• facebook/time
• facebook/zstd
• facebookexperimental/starlark-rust (this was me)
• facebook/proxygen
• facebook/hermes
• facebook/rocksdb

As a proof of concept I created a couple of super simple fuzz harnesses in #1537.

NOTE: Adding fuzz-testing and integrating with google/oss-fuzz was previously suggested here #1055 and was rejected. I think I've addressed the concerns raised in the first PR. While the original PR contained what was probably a higher performance fuzzer, the new fuzzer should be easier to integrate and doesn't introduce multiple sources of truth.

[NickGerleman]

Thank you for raising this, and contributing a PR implementing it. I think it is a good idea, so long as we make sure it isn't disrupted by any unrelated Yoga changes.

[nathaniel-brough]

As a follow up I'm going to start the integration with google/oss-fuzz if you are still keen on it. There is an application/integration process which I can complete on your behalf. To do this I'll need a couple of things from you;

• A google/gmail email address that I can use for the primary contact. As far as I'm aware meta/facebook emails meet this requirement.
• You to comment on the PR that I'll create that will form the integration/application saying that you approve of the integration.

Given the popularity of yoga I'm reasonably confident that it'll be accepted into oss-fuzz, but there is a non-zero chance that it could be rejected.

[nathaniel-brough]

So I've got a draft PR ready to go integrating with OSS-fuzz in google/oss-fuzz#11533. The only thing I need to do now is add a primary contact. I've added [email protected] which seems to be CC'd on most of the other facebook integrations.

[NickGerleman]

Thanks @silvergasp. Is there anything left to action on from our side?

[nathaniel-brough]

Yeah, all I need is the email address that you'd like to use to get access to the bug tracker and oss fuzz dashboard. Just as a note, it'll be stored as plain text on the oss fuzz git repo. After that everything should be good to go.

[NickGerleman]

Reusing oss-fuzz probably makes sense if it is already being used, though I’m not actually sure where it is funelling to. I can try to find that out.

[nathaniel-brough]

Easy do you want me to add your email to the config? That will give you personal access to the dashboard/bug tracker?

[NickGerleman]

I appreciate it. Could you use this address? https://github.com/facebook/yoga/commit/cc2a87d4f95d2deac3f4ffed02ca4c130aec70ba.patch

[nathaniel-brough]

Ah sorry I missed this notification. I've pulled the trigger on the PR over at oss-fuzz now. All that is needed is to wait for the oss-fuzz team to approve.