(de)serialization of preprocessing

[moodlezoup]

Currently, the jolt::provable proc_macro generates a build_* function for each decorated function (see e.g. build_fib). Calling this function from the host builds the guest program and computes preprocessing. It then returns two closures, prove_* and verify_* which prove and verify executions of the guest, respectively.

Of course, a more realistic usage pattern would be to compile the guest and compute the preprocessing only once per guest program (more precisely, preprocessing can also depend on parameters specifying e.g. the max input size of the guest), and then serialize it or post it to some public bulletin.
Then a prover could read the preprocessing from disk once and use it for multiple invocations of Jolt::prove.

Similarly, the verifier needs a subset of the JoltPreprocessing data (currently the same preprocessing struct is used for prover and verifier, but this should be changed). This verifier should be able to read the required data from disk once and use it for multiple invocations of Jolt::verify.

[protoben]

@jprider63 and I have discussed what the API for this should look like, and we were thinking something like this:

let target_dir = "/tmp/target";
guest::compile_fib(&target_dir);

let prover_preprocessing = guest::prover_preprocess_fib(&program);
/* Internally, this is
let config = ...; // Specify max input size, etc.
preprocess_prover(&program, &config)
*/
let prover_preprocessing_file = "/tmp/prover_preprocessing";
prover_preprocessing.save_to_file(&prover_preprocessing_file);

let prove_fib = guest::build_prover(&program, prover_preprocessing_file);
/* Internally, this is
let program = load_program(&target_dir);
let preprocessing = load_prover_preprocessing(&prover_preprocessing_file);
|inputs, ...| {
    let input = serialize(inputs);
    run_prover(program, preprocessing, input)
}
*/
let (output, proof) = prove_fib(50);

let verifier_preprocessing = guest::verifier_preprocess_fib(&program);
/* Internally, this is
let config = ...; // Specify max input size, etc.
preprocess_verifier(&program, &config)
*/
let verifier_preprocessing_file = "/tmp/verifier_preprocessing";
verifier_preprocessing.save_to_file(&verifier_preprocessing_file);

let verify_fib = guest::build_verifier(&target_dir, verifier_preprocessing_file);
/* Internally, this is
let program = load_program(&target_dir);
let preprocessing = load_verifier_preprocessing(&prover_verifier_file);
|proof, claimed_output, inputs, ...| {
    let input = serialize(inputs);
    let output = serialize(claimed_output);
    run_verifier(program, preprocessing, proof, input, output)
}
*/
let is_valid = verify_fib(proof, output, 50);

The stuff in comments would be a lower-level interface, whereas the stuff outside of the comments would be the higher-level interface, using proc macros, as is done currently. How does this look to you?

There were a couple of things we weren't clear on:

1. In the current Jolt code, the verify_{guest} function doesn't take the guest input or output. It seems like it should (since otherwise, how does the verifier know that the proof corresponds to this specific input/output pair?); does that seem right? If so, how should that be accomplished? I don't see how the input/output would get supplied to the Jolt::verify function. Is that in scope for this issue?
2. The above design generates the prover preprocessing and verifier preprocessing completely separately. Is that the right thing to do, or is there some subset of both that should be generated first and shared between the prover and verifier. I.e., is there any part of the preprocessing that can/should be done just once, rather than by the prover and verifier individually?

[Roee-87]

Regarding your first point, the Jolt proof does indeed contain a copy of the guest program inputs and outputs, so there's no need to pass them as arguments to the verify function. The Jolt proof struct contains a field for program IO).

The JoltDevice struct contains fileds for the guest program inputs and outputs (see here).

[moodlezoup]

The interface looks good to me @protoben
@Roee-87 is right about the first point, the verifier has the inputs/outputs in the JoltDevice struct
Regarding the second point, I think it's fine to generate the prover and verifier preprocessing separately for now. If we find that the verifier preprocessing is taking an unreasonably long time to generate, we can revisit.

[jprider63]

Would you be open to pulling out the inputs and outputs from JoltProof (and JoltDevice)? Our reasoning is:

1. It more closely matches at least my intuition that a VC proof is just the cryptographic proofs (and machine settings) and the verifier needs to provide the input and claimed output for the scenario they care about.
2. It is potentially error prone since it is easy for a verifier to forget and blindly accept the input and output provided by the prover in the proof (especially since all the examples currently do this).
3. Inputs and outputs could be very large so if a prover is sending a verifier a succinct cryptographic JoltProof, it might waste space to send inputs and outputs that the verifier doesn't need.

[protoben]

To echo what @jprider63 is saying, it seems like even if the inputs/outputs are contained in the proof, the high-level API should have some way of checking that the proof corresponds to the ones the verifier expects.

One possibility would be to add a helper function to the high-level API to do this check that inherits the signature of the guest function (like the functions returned by build_{guest}), e.g.,

let (output1, ..., proof) = ... // get these from the prover

let instance_check = check_instance_myfunc(proof, input1, ..., output1, ...); // This checks that the inputs/outputs are the ones in the proof
let proof_check = verify_myfunc(proof);

[moodlezoup]

Makes sense; let's pull the inputs and outputs out of JoltProof

[protoben]

Made a PR here: #616