Conflation Backtesting Endpoint

[fluentcrafter]

Description

The coordinator should implement an endpoint to reconflate and regenerate prover and tracer requests for a given block range.

This would allow the prover and arithmatization team to perform backtesting of new tracer/prover versions.

Motivation

• allow back testing of conflation
• generate prover tracer requests quickly and fast

Tasks

• implement JSON-RPC endpoint that reconflates a given block range, by regenerating prover requests files

Acceptance criteria

• List tests that need to be implemented or checks to consider this task completed.

Risks

• If called in a production environment, using the same prod Shomei and Besu Tracer, it can overwhelm the nodes. It's a known risk, and the endpoint should be used carefully depending on the test scope.

Specification

Request conflation_createProverRequests

{
    "jsonrpc": "2.0",
    "id": "1",
    "method": "conflation_createProverRequests",
    "params": [{
        "startBlockNumber": 234,
        "endBlockNumber": 999,
        "parentShnarf": "0x00...",
        "l1MessageNumber": 12312,
        "l1MessageRollingHash": "0x0...",
        "compressorVersion": "V3",
        "batchFixedSize (optional)": "when defined, batches have fixed size - allows skipping trace counting, which is a bottleneck", 
        "tracesLimits (optional) - overrides coordinator's current traces limits": {""},
        "aggregationProofLimit": 123,
        "aggregationBlobsLimit (optional)": 6,
        "proverRequestsDestinationDir": "/path/to/prover/ - cannot be same as current prover, otherwise will with prod prover", 
        "conflatedTracesDestinationDir": "(trace: override the destination path)",
        "ethApi(optional)": {
            "endpoint": "http://besu-full:8545/",
            "polling-interval": "PT1S",
        },
        "tracesApi": {
            "endpoint": "http://traces-api:8545/",
            "version": "V1.2.3",
            "requests-limit-per-endpoint": 10
        },
        "shomeiApi": {
            "endpoint": "http://traces-api:8545/",
            "version": "V1.2.3",
            "requests-limit-per-endpoint": 10
        }
    }]
}

It should generate all pover requests (execution, compression, aggregation) into the specified proverRequestsDestinationDir/{execution, compression,aggregation}.
conflatedTracesDestinationDir shall be sent as parameter to linea_generateConflatedTracesToFile NOTE: tracer needs development to support this feature.

Since the conflation flow can take several minutes, and connection dropped, the request returns imedeatly upon validation of the arguments and statung the conflation flow. It should return a JobID which can be latter on used to get it's status.

Responses

• success result: conflationJobId

{"result": "2025-11-17...."}

• errors

  • re conflation job already inprogress
    {"error": {"code": -32602, "message": "There already a reconflation process inprogress"}}

  • invalid proverRequestsDestinationDir
    {"error": {"code": -32602, "message": "Invalid proverRequestsDestinationDir - conflicts with production dir"}}

  • invalid conflatedTracesDestinationDir
    {"error": {"code": -32602, "message": "Invalid conflatedTracesDestinationDir - conflicts with production dir"}}

Request conflation_getReconflationJobsStatus

{
    "jsonrpc": "2.0",
    "id": "1",
    "method": "conflation_getReconflationJobsStatus",
    "params": ["JOB_ID"]
}

JOB_ID is optional, when not provided return all jobs atm.

{
    "jsonrpc": "2.0",
    "id": 4,
    "result": {
        "jobId" {
            "status": "INPROGRESS/FINISHED/ERROR",
            "startBlockNumber": 123,
            "endBlockNumber": 999,
            "highestConflatedBlockNuber": 555,
            "error" "error message"
        }
    }
}