Parallelised proof generation

[alexanderwestlake]

I've been trying to generate ZK proofs that recursively verify transitions between successive states. The original approach was batched linear recursion, where each "round" would prove a batch of moves with a proof and a successor state. I wanted to optimise this further, so I wrote a new ZkProgram to instead prove two states and a batch of moves, then prove that two proofs were consecutive. This would improve on the original's performance...assuming the rounds were able to be generated in parallel.

With the context out the way, is there currently support in o1js for this sort of parallelisation? I'm aware that multithreading's already used for individual proof generation using nifty tricks like SharedArrayBuffer and so on, but I'd specifically like to be able to delegate proof generation for each batch of inputs to a web worker and have them execute concurrently.

[dfstio]

When proving on the local computer, o1js already uses all the available CPU cores, so parallel proving will not be faster since the same CPUs will be utilized and just split between several prove tasks.

To make parallel proving faster, you need to calculate the proofs across different workers that use different CPUs, such as in a serverless environment where you can start many instances, each having its own CPU.

You can see an example of how zkCloudWorker does this. If you calculate and merge 128 proofs in zkCloudWorker, it will utilize a total of 768 CPU cores and 1310 GB of RAM, making it much faster than would be possible on a local machine.
https://github.com/zkcloudworker/worker-example/blob/main/tests/contract.test.ts#L273