Unable to be used in conjuction with forwarding time helpers

[area]

It is currently (as far as I can tell) impossible to trace a transaction that has happened in the future, as might happen in a development environment when using helpers.

Running npx hardhat node in one console, in another inside a npx hardhat console connected to the node:

<Command to do Transaction 1> 
{ time } = await import("@nomicfoundation/hardhat-network-helpers")
time.increase(1000000000)
<Command to do Transaction 2> 

If you now try and trace those transactions:

npx hardhat trace --hash <tx1 hash>  --rpc http://127.0.0.1:8545

This command works

npx hardhat trace --hash <tx2 hash>  --rpc http://127.0.0.1:8545

This does not work, and fails with:

Nothing to compile
Activating mainnet fork at block 4
thread 'tokio-runtime-worker' panicked at /build/crates/edr_provider/src/data.rs:2464:18:
current time must be after fork block: SystemTimeError(999999463.793941211s)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
An unexpected error occurred:

[Error: task 37 panicked] { code: 'GenericFailure' }

This error is being thrown here

I was unsure whether to make this here or in the EDR repository directly, but I see @fvictorio is active in this repository as well, so I am hopeful it will end up in the right place regardless!

I haven't tried with a pre-EDR version of hardhat, so do not know if this a regression.

[fvictorio]

Hey @area, there's definitely something weird going on here, but something that caught my attention is your use of --rpc http://127.0.0.1:8545. I think you meant to use --network localhost instead, if the intention was to run the trace in the node?

I don't fully understand what --rpc does in hardhat-tracer, but it seems to me that it expects a "real node" URL (e.g., a synced local node, or a service provider like Alchemy).

[area]

I can't claim to know exactly what's going on either! However, when trying to use --network localhost in place of --rpc or equivalent I get

TypeError: this.edrProvider._setVerboseTracing is not a function
    at Switch.enable (/path/node_modules/.pnpm/hardhat-tracer@3.0.3_chai@4.4.1_hardhat@2.22.6/node_modules/hardhat-tracer/src/switch.ts:15:28)
    at SimpleTaskDefinition.action (/path/node_modules/.pnpm/hardhat-tracer@3.0.3_chai@4.4.1_hardhat@2.22.6/node_modules/hardhat-tracer/src/tasks/trace.ts:144:30)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Environment._runTaskDefinition (/path/node_modules/.pnpm/hardhat@2.22.6_typescript@4.9.5/node_modules/hardhat/src/internal/core/runtime-environment.ts:359:14)
    at Environment.run (/path/node_modules/.pnpm/hardhat@2.22.6_typescript@4.9.5/node_modules/hardhat/src/internal/core/runtime-environment.ts:192:14)
    at main (/path/node_modules/.pnpm/hardhat@2.22.6_typescript@4.9.5/node_modules/hardhat/src/internal/cli/cli.ts:323:7)

I assumed that this was something to do with not tracing against an 'in-process' node and didn't think much further about it, but if this could be a symptom of something related that's not working, happy to explore further.

[fvictorio]

That's weird, can you double-check that you are using the latest versions of Hardhat and hardhat-tracer?

[area]

I appear to be. Full reproduction steps for that side of things, for you, using openzeppelin's repository:

git clone git@github.com:OpenZeppelin/openzeppelin-contracts.git --depth 1
cd openzeppelin-contracts
nvm use 20
npm i
npm i hardhat-tracer@latest
npx hardhat --version // Returns 2.22.6, which is latest
grep hardhat-tracer package.json // Returns ^3.0.3, with 3.0.3 being latest

Download this patch for the hardhat config and apply with git apply hh.patch.txt from the repository root: hh.patch.txt

Then in one console: npx hardhat node --trace (I've tried with and without --trace, with the same result either way)

In a second console:

npx hardhat console --network localhost
s = (await ethers.getSigners())[0]
await s.sendTransaction({to:s.address, data:"0xdeadbeef"})

And then:

npx hardhat trace --network localhost --hash 0x2cd4d8943c4a2ac44a8feba891c4483b1003e8fab7656c4ea8318860abe4d36e

Nothing to compile
An unexpected error occurred:

TypeError: this.edrProvider._setVerboseTracing is not a function

npx hardhat trace --rpc http://127.0.0.1:8545 --hash 0x2cd4d8943c4a2ac44a8feba891c4483b1003e8fab7656c4ea8318860abe4d36e
Nothing to compile
Activating mainnet fork at block 1
0x2cd4d8943c4a2ac44a8feba891c4483b1003e8fab7656c4ea8318860abe4d36e
[CALL] UnknownContract(0xf39fd6e51aad88f6f4ce6ab8827279cfffb92266).CodeIsLawZ95677371()

[fvictorio]

In that case this seems like a problem on the hardhat-tracer side. I'll let @zemse take a look first, but happy to help if needed.

[zemse]

Sorry I was afk last week. I will look into this problem tomorrow and try to fix it.

[zemse]

Sorry for the long delay.

I have tried a fix. npx hardhat node --trace had a problem, it wasn't collecting traces, now should collect the traces. I've also re-implemented the trace task to handle the case when rpc is a hardhat node, it would attempt to pull trace data directly through a custom rpc tracer_getTrace(txHash) otherwise fallback to mainnetfork debug tracing.

Can you please try the release hardhat-tracer@3.1.0?

[area]

This appears to now be possible when using npx hardhat node --trace, though my reproduction steps in the opening post result in the same error as before when using just npx hardhat node.

--network localhost and --rpc http://127.0.0.1:8545 now seem to behave the same in each case.

[zemse]

I was able to reproduce this problem when using npx hardhat node (without --trace). Can you confirm if you used --trace while starting node or not?

If --trace is used while starting hardhat node then it will record and store traces for the locally executed transactions and npx hardhat trace --rpc <url> will simply make an RPC call tracer_getTrace with the tx hash and display the trace, this is preferred. This flow should avoid the forking and the original error.

Without the --trace on the node, the trace task would create a fork and that's where error is coming in this case.

Here are steps to reproduce outside of hardhat tracer.

Terminal 1:

$ npx hardhat node

Terminal 2:

$ npx hardhat console --network localhost

> s = await hre.ethers.provider.getSigner(0)

> await s.sendTransaction({to:s.address, data:"0xdeadbeef"})

> await hre.ethers.provider.send("evm_increaseTime", [10000])

> await s.sendTransaction({to:s.address, data:"0xdeadbeef"})

> block_num = await hre.ethers.provider.getBlockNumber()
// 2

Terminal 3:

$ npx hardhat console

> hre.network.name
// "hardhat"

> await hre.network.provider.send("hardhat_reset", [{forking: {jsonRpcUrl: "http://127.0.0.1:8545/", blockNumber: 2 }}]);
// thread 'tokio-runtime-worker' panicked at /Users/runner/work/edr/edr/crates/edr_provider/src/data.rs:2503:18:
// current time must be after fork block: SystemTimeError(9978.213897s)
// note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
// Uncaught [Error: task 15 panicked] { code: 'GenericFailure' }

> await hre.network.provider.send("hardhat_reset", [{forking: {jsonRpcUrl: "http://127.0.0.1:8545/", blockNumber: 1 }}]);
// true

cc @fvictorio The reason I am doing hardhat_reset is to be able to replay remote transactions locally on EDR. Is it possible to allow remote blocks to use future timestamp? Given that many startups these days offer customisable private testnets RPCs which might have future timestamp. Or maybe a flag that allows for it?

[area]

I was able to reproduce this problem when using npx hardhat node (without --trace). Can you confirm if you used --trace while starting node or not?

Maybe my original message was unclear, but I think we're in agreement - when using --trace, the issue does not appear. But when just using npx hardhat node, it is still present.