Rust CLI Benchmark for Noir Witness Generation

[sidthakidd]

Summary

We propose WBench, a Rust-based CLI tool for systematically benchmarking witness generation in Noir circuits. WBench will repeatedly execute Noir programs on given inputs and record timing data, reporting metrics (mean, median, p95, p99, etc.) in structured JSON and human-readable formats. By targeting both local use and CI (e.g. GitHub Actions), WBench fills a tooling gap: while Noir’s ecosystem includes a profiler and proving benchmarks, no dedicated tool exists for measuring witness generation performance. Efficient witness generation is crucial, as it often dominates end-to-end runtime (particularly on constrained devices) and can vary dramatically with inputs. For example, one benchmark found Noir witness times around 99–121 ms for SHA-256 and simple circuits. Existing work (e.g. zk_bench) compares Circom vs Noir runtimes but lacks structured outputs or CI integration. Our tool will leverage Noir’s nargo infrastructure (invoking witness generation via the Rust APIs or CLI) to remain compatible with future Noir releases, and will be written in Rust for speed and portability. WBench will output JSON summaries for automated tracking and also print nicely formatted tables for developers, enabling easy regression testing of witness performance in diverse environments.

Motivation

Noir’s promise as an accessible general-purpose ZK language rests on developer productivity and performance. A witness is essentially a program execution trace given public inputs, and generating it can be a major cost. Community examples highlight this: the noir_bigcurve library explicitly optimizes to reduce witness overhead, noting that “witness generation time [is] currently the bottleneck” under Brillig, and noir_bignum warns that heavy arithmetic “would be very expensive until noir witness computation times improve”. Independent benchmarks echo this: one study reported Noir witness times of ~99–121 ms on an M2 MacBook Air for small primitives (vs 132–278 ms in Circom). These costs matter for user experience and feasibility: e.g. mobile or browser environments (ARM, WASM) incur extra overhead – indeed, a ZKP pantheon benchmark noted that on ARM, WASM-based witness generation was slower than native C++.

However, despite these concerns, the Noir ecosystem currently lacks a general tool to measure them. The official awesome-noir list includes performance tools like a Barretenberg flamegraph profiler and a browser-based proving benchmark, but nothing for CLI-based witness benchmarking. An existing zk_bench tool can compare Noir vs Circom runtimes, yet it is a JS/Yarn project without structured JSON output or CI hooks. As a result, developers have no easy way to track performance regressions or optimize witness code. Given that the NRG-4 “Developer Tools” round specifically calls for infrastructure to aid Noir development, a Rust CLI witness benchmark directly addresses a clear gap. By enabling rigorous performance testing of Noir circuits (e.g. before/after code changes), WBench will improve reliability and signal bottlenecks that affect everything from library design to UX in light-client settings.

Methodology

WBench will be implemented in Rust (using crates like clap for argument parsing and serde_json for output). It will consume one or more Noir artifacts (e.g. target/circuit.json from nargo compile) and corresponding input files (JSON) for each circuit. For each [circuit, input] pair, WBench will run witness generation repeatedly (configurable runs, e.g. 10× or 100×), using either the nargo library API or invoking nargo as a subprocess. Each run’s duration (using std::time::Instant) is recorded. After collecting timings in a Vec<Duration>, the tool computes statistics:

• Mean, median, min, max
• Percentiles (e.g. p95, p99) computed from the sorted times.
• Stddev (optional) and total runs.

These are emitted as structured JSON. For example:

{
  "circuit": "path/to/circuit.json",
  "input": "input.json",
  "runs": 50,
  "mean_ms": 112.3,
  "median_ms": 110.7,
  "p95_ms": 130.9,
  "p99_ms": 145.2,
  "min_ms": 98.4,
  "max_ms": 152.1
}

This JSON format allows CI pipelines (e.g. GitHub Actions) to parse and log performance over time. Concurrently, WBench will print a human-readable report to the console, showing a table of metrics per circuit (using tabulated text or simple formatting). For example:

Circuit            Runs  Mean(ms)  Median(ms)  p95(ms)  p99(ms)
---------------------------------------------------------------
add.json           50    12.4      12.1        13.8     14.0
bigmult.json       50    87.9      85.3        102.4    110.7

Key design choices include:

• Rust & native execution: Ensures minimal overhead compared to the witness computation itself. For heavy circuits, the cost of spawning a process or loading can be negligible, but Rust’s speed is still beneficial.
• Integration with nargo: WBench will rely on Noir’s published compilation artifacts and (if feasible) reuse nargo’s Rust libraries for witness generation to stay compatible with Noir’s evolution. If no stable library exists, WBench can shell out to nargo witness --path <circuit> --json-input <file>. This loose coupling is documented so future nargo changes can be adopted
• Data collection: By default, WBench will ignore any caching effects by cleaning intermediate state as needed. It will allow a "warm-up" run (discarded) and then record a fixed number of timed runs
• Report formatting: JSON keys and CLI headers will be clearly labeled (mean, median, p95, p99). WBench can optionally output CSV or append to a Markdown table for easy inclusion in reports.
• Sample CLI usage: e.g. wbench witness --input input.json --runs 20 --json-out results.json will produce both a console summary and a file results.json.

Example Rust snippet:

let mut times = Vec::new();
for _ in 0..runs {
    let start = std::time::Instant::now();
    noir::execute_witness(&circuit, &inputs)?; // pseudo-code for running Noir
    times.push(start.elapsed().as_millis() as f64);
}
times.sort_by(|a, b| a.partial_cmp(b).unwrap());
let median = times[runs/2];
let p95 = times[(runs*95/100)];

Business Model

• Community Maintenance: By placing WBench on GitHub (ideally under the noir-lang org or a community repo), the broader community can contribute fixes and enhancements. User feedback (via issues/PRs) can guide future features (e.g. new metrics or CI integrations).
• Sponsorship / Grants: Given its relevance, WBench could be maintained by grants or sponsorship from stakeholders (Aztec, Arbitrum, Stellar, etc.) interested in Noir performance. The codebase can be light, requiring minimal ongoing costs.
• Integration into Noir Toolchain: If the core Noir team sees value, WBench could be adopted as an “official” performance tool, similar to how noir-profiler was included in Noirup. In that case, the team might help maintain it as part of the ecosystem (e.g. merging it into nargo or providing CI support).

Overall, WBench’s sustainability relies on its utility to the community. By lowering barriers to performance testing and regression analysis, it should attract contributions and possibly pipeline integration.

Timeline and Deliverables

• Week 1 – Design & Setup: Finalize requirements; scaffold the Rust project and CLI (clap arguments for circuits/inputs/runs); configure CI (GitHub Actions) for building and testing. Deliverables: Project repo with initial CLI parsing, and documented scope.
• Week 2 – Circuit Handling: Implement loading of Noir artifacts (e.g. reading compiled circuit JSON) and input files. Prototype witness invocation (e.g. call nargo as a subprocess or use internal APIs). Validate with a simple circuit to ensure correctness. Deliverables: Basic witness-run loop works for one circuit; sample logs.
• Week 3 – Metrics Computation: Implement timing loop and statistics. Compute mean, median, p95, p99. Add JSON output support (serde_json). Format CLI output. Write unit tests for statistics. Deliverables: JSON and CLI reports for test circuits, with correct values.
• Week 4 – CI Integration & Examples: Create example workflows (e.g. a GitHub Actions job that runs WBench on a test circuit). Write README/documentation, including usage and example JSON schema. Enhance error handling and performance options (e.g. warm-up runs flag). Deliverables: CI example (workflow YAML) and user guide.
• Week 5 – Optimization & Polishing: Profile WBench itself (to ensure minimal overhead). Improve output (add optional metrics, e.g. stderr summary). Solicit early feedback from Noir community (perhaps a preview release). Fix any issues. Deliverables: v1.0 prototype release; change-log of features; benchmarks showing minimal tool overhead.
• Week 6 – Finalization: Incorporate feedback, finalize docs, release 1.0 on crates.io (optional) or GitHub Releases. Prepare grant report and demos. Deliverables: Final source release, documentation website or repo README, NRG-4 final report.

Team

This project will be led by a solo developer who is new to the Noir ecosystem but deeply interested in zero-knowledge proofs and the promise of client-side proving. What drew them to Noir specifically was its focus on simplicity, general-purpose usability, and growing support for proving in constrained environments like the browser and mobile devices. While new to Noir, the developer has a solid background in programming and has been studying the technical foundations of zero-knowledge systems. They have several friends in the Noir community—some of whom are actively contributing to the language or tooling—and will seek their guidance as needed. Additionally, they plan to make full use of the open support channels, public GitHub discussions, and direct community engagement encouraged by the NRG-4 Developer Tools initiative.

This proposal is also a learning journey. By building WBench, the developer aims to create a useful public tool while deepening their understanding of the witness generation process, Noir internals, and performance profiling in ZK systems.

Questions

• Maintenance Interest: Would the Noir core team consider adopting WBench into the official tooling suite or supporting its maintenance after the grant? Is there interest in further integrating it (e.g. as a nargo bench-witness command)?
• CI Practices: Are there recommended practices for incorporating benchmark tools into Noir’s CI (e.g. performance thresholds, regression alerts)?
• Scope of Input Handling: Should WBench automatically discover and manage input files (like Prover.toml) or rely on explicit CLI arguments? What conventions do Noir projects follow for inputs in benchmarks?