bug: Event logs are incrementing from last returned event instead of full batch range

[escottalexander]

Is there an existing issue for this?

• I have looked through the existing issues

Which method was used to setup Scaffold-ETH 2 ?

npx create-eth@latest

Current Behavior

First here is the code that is being used:

const {
    data: sentMessages,
    isLoading: isLoadingSentMessages,
    error: errorReadingSentMessages,
  } = useScaffoldEventHistory({
    contractName: "ArbAddressTableExample",
    eventName: "MessageSent",
    blocksBatchSize: 100000,
    filters: { sender: address },
  });

Here is what I am seeing when I copy all the RPC logs for eth_getLogs.

// First call 
{"jsonrpc":"2.0","id":15,"method":"eth_getLogs","params":[{"fromBlock":"0xa909f82","toBlock":"0xa90f864"}]}
// ^^^ returns vvv
{
    "jsonrpc": "2.0",
    "id": 15,
    "result": [
        {
            "address": "0xc731bd6b975526e089ca1329d15c0c03b6faaf63",
            "topics": [
                "0x7e49d574e0305424aa957cb3f69b25f92aa58d53fdee73f05e40019210a70bce",
                "0x0000000000000000000000002f1ad369d2c81ad8620e47d80a28bf35fabcf030",
                "0x0000000000000000000000002f1ad369d2c81ad8620e47d80a28bf35fabcf030"
            ],
            "data": "0x00000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001148656c6c6f20616761696e20616761696e000000000000000000000000000000",
            "blockNumber": "0xa90eb00", // Block number of the last event returned
            "transactionHash": "0x1a1d2221c4987ea480737cc15966492e4e6794fbd7fb9f6216e43da6c7275d13",
            "transactionIndex": "0x1",
            "blockHash": "0x0423a4a94421cb2a4581978462b8ba6dc5ce0735f6d3ce2cf4402b1f72ac1a7b",
            "logIndex": "0x0",
            "removed": false
        }
    ]
}
// Notice that the fromBlock is incremented by 1 for each call but from the last event returned, not the last toBlock searched
{"jsonrpc":"2.0","id":18,"method":"eth_getLogs","params":[{"fromBlock":"0xa90eb01","toBlock":"0xa90f864"}]}
// And they continue to increment from there instead of using the toBlock of the last call
{"jsonrpc":"2.0","id":22,"method":"eth_getLogs","params":[{"fromBlock":"0xa90eb02","toBlock":"0xa90f864"}]}
{"jsonrpc":"2.0","id":26,"method":"eth_getLogs","params":[{"fromBlock":"0xa90eb03","toBlock":"0xa90f864"}]}
{"jsonrpc":"2.0","id":31,"method":"eth_getLogs","params":[{"fromBlock":"0xa90eb04","toBlock":"0xa90f864"}]}
{"jsonrpc":"2.0","id":33,"method":"eth_getLogs","params":[{"fromBlock":"0xa90eb05","toBlock":"0xa90f864"}]}
{"jsonrpc":"2.0","id":37,"method":"eth_getLogs","params":[{"fromBlock":"0xa90eb06","toBlock":"0xa90f864"}]}
// Forever... 

As a result, setting the batch size doesn't help create fewer queries since it will send a call for each block between the last event found's block and the toBlock. Then it is only incrementing the fromBlock by one for each call. Effectively not proceeding to search a new span of blocks until a single request has been made for every block in current range!! Yikes!

Expected Behavior

I would expect that each request would only search each fromBlock <> toBlock span once and then move on to the next span.
Assuming a batch size of 10 It should go through them like this:

0 - 10
10 - 20
20 - 30
...

Steps To Reproduce

If you need to recreate it exactly you can use npx create-eth -e BuidlGuidl/arbitrum-extension. Then deploy the contract to arbitrumSepolia. Make a few transactions (if you even can, the default RPC is usually too overloaded to estimateGas properly).

Might would be useful to create a specific test case on a local hardhat network and see if it could be recreated there.

Anything else?

I know that we added the batchSize limit of 500 because of some changes to Alchemy. I read somewhere that some RPCs may have an event limit (not block range limit) though it seemed to say it was 10,000 for this specific RPC. We should just keep that in mind that potentially we would need to grab the latest event block instead of the last toBlock but that would only apply if we reached that maximum for events returned (if the query returned 10,000 events).