[Test] Use spawn in older pytorch

[vmoens]

Stack from ghstack (oldest at bottom):

• [CI] Fix tensordict install in lib tests #3286
• -> [Test] Use spawn in older pytorch #3285
• [Feature] auto_wrap_envs in PEnv #3284
• [Test] Fix test_num_threads that instantiate the env in the main process #3283
• [BugFix] Fix unique ref to lambda func #3282

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3285

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• B200 runners are down due to network issues

❌ 5 New Failures

As of commit 633b83c with merge base 7866d11 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 139.
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t eee135966cc9e12568cd99f04cad727c3cdf967976c422886e0856d544212e91 /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 6c30b00e2f3c33fe5df1341b4513933213582a8a917ade047397b47774d7177a /exec failed with exit code 1
• Unit-tests on Linux / tests-olddeps (3.9, 11.6) / linux-job (gh)
  test/test_transforms.py::TestVecNorm::test_parallelenv_vecnorm
• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh)
  RuntimeError: Command failed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 164. Improved: $\large\color{#35bf28}22$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	86.7453μs	85.3693μs	11.7138 KOps/s	12.3663 KOps/s	$\textbf{\color{#d91a1a}-5.28\%}$
test_tensor_to_bytestream_speed[torch.save]	0.1421ms	0.1412ms	7.0811 KOps/s	7.1557 KOps/s	$\color{#d91a1a}-1.04\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1128s	0.1121s	8.9224 Ops/s	8.8507 Ops/s	$\color{#35bf28}+0.81\%$
test_tensor_to_bytestream_speed[numpy]	2.7711μs	2.7661μs	361.5140 KOps/s	359.9879 KOps/s	$\color{#35bf28}+0.42\%$
test_tensor_to_bytestream_speed[safetensors]	40.1591μs	39.0086μs	25.6354 KOps/s	26.6461 KOps/s	$\color{#d91a1a}-3.79\%$
test_simple	0.5635s	0.5550s	1.8018 Ops/s	1.7364 Ops/s	$\color{#35bf28}+3.76\%$
test_transformed	1.1259s	1.1244s	0.8894 Ops/s	0.8680 Ops/s	$\color{#35bf28}+2.46\%$
test_serial	1.6687s	1.6659s	0.6003 Ops/s	0.5909 Ops/s	$\color{#35bf28}+1.58\%$
test_parallel	1.2779s	1.1804s	0.8471 Ops/s	0.8082 Ops/s	$\color{#35bf28}+4.81\%$
test_step_mdp_speed[True-True-True-True-True]	0.3226ms	46.0835μs	21.6997 KOps/s	22.6824 KOps/s	$\color{#d91a1a}-4.33\%$
test_step_mdp_speed[True-True-True-True-False]	55.9720μs	25.6279μs	39.0200 KOps/s	39.7179 KOps/s	$\color{#d91a1a}-1.76\%$
test_step_mdp_speed[True-True-True-False-True]	54.5430μs	25.7628μs	38.8156 KOps/s	39.6836 KOps/s	$\color{#d91a1a}-2.19\%$
test_step_mdp_speed[True-True-True-False-False]	43.4420μs	13.9692μs	71.5860 KOps/s	72.5060 KOps/s	$\color{#d91a1a}-1.27\%$
test_step_mdp_speed[True-True-False-True-True]	98.7750μs	48.9692μs	20.4210 KOps/s	20.8795 KOps/s	$\color{#d91a1a}-2.20\%$
test_step_mdp_speed[True-True-False-True-False]	61.3530μs	28.7646μs	34.7650 KOps/s	35.3267 KOps/s	$\color{#d91a1a}-1.59\%$
test_step_mdp_speed[True-True-False-False-True]	65.1330μs	28.7536μs	34.7782 KOps/s	35.8124 KOps/s	$\color{#d91a1a}-2.89\%$
test_step_mdp_speed[True-True-False-False-False]	62.3030μs	16.9262μs	59.0799 KOps/s	59.4033 KOps/s	$\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-False-True-True-True]	91.8150μs	52.2411μs	19.1420 KOps/s	19.5924 KOps/s	$\color{#d91a1a}-2.30\%$
test_step_mdp_speed[True-False-True-True-False]	77.4040μs	31.6699μs	31.5757 KOps/s	32.8099 KOps/s	$\color{#d91a1a}-3.76\%$
test_step_mdp_speed[True-False-True-False-True]	61.5830μs	28.7875μs	34.7373 KOps/s	35.5331 KOps/s	$\color{#d91a1a}-2.24\%$
test_step_mdp_speed[True-False-True-False-False]	41.9420μs	16.9660μs	58.9413 KOps/s	59.3062 KOps/s	$\color{#d91a1a}-0.62\%$
test_step_mdp_speed[True-False-False-True-True]	85.6040μs	55.1562μs	18.1303 KOps/s	18.8266 KOps/s	$\color{#d91a1a}-3.70\%$
test_step_mdp_speed[True-False-False-True-False]	62.1430μs	34.7148μs	28.8062 KOps/s	29.8530 KOps/s	$\color{#d91a1a}-3.51\%$
test_step_mdp_speed[True-False-False-False-True]	58.2030μs	31.3559μs	31.8919 KOps/s	32.7133 KOps/s	$\color{#d91a1a}-2.51\%$
test_step_mdp_speed[True-False-False-False-False]	41.9220μs	19.7367μs	50.6671 KOps/s	50.8913 KOps/s	$\color{#d91a1a}-0.44\%$
test_step_mdp_speed[False-True-True-True-True]	83.1550μs	51.2801μs	19.5008 KOps/s	19.5889 KOps/s	$\color{#d91a1a}-0.45\%$
test_step_mdp_speed[False-True-True-True-False]	59.5330μs	31.6347μs	31.6109 KOps/s	32.4643 KOps/s	$\color{#d91a1a}-2.63\%$
test_step_mdp_speed[False-True-True-False-True]	2.2748ms	32.7273μs	30.5556 KOps/s	31.5107 KOps/s	$\color{#d91a1a}-3.03\%$
test_step_mdp_speed[False-True-True-False-False]	44.8520μs	18.8499μs	53.0507 KOps/s	54.3807 KOps/s	$\color{#d91a1a}-2.45\%$
test_step_mdp_speed[False-True-False-True-True]	0.1018ms	54.2438μs	18.4353 KOps/s	18.8569 KOps/s	$\color{#d91a1a}-2.24\%$
test_step_mdp_speed[False-True-False-True-False]	79.7340μs	34.3206μs	29.1370 KOps/s	29.8700 KOps/s	$\color{#d91a1a}-2.45\%$
test_step_mdp_speed[False-True-False-False-True]	73.6140μs	35.1841μs	28.4219 KOps/s	28.9153 KOps/s	$\color{#d91a1a}-1.71\%$
test_step_mdp_speed[False-True-False-False-False]	62.7140μs	21.8246μs	45.8199 KOps/s	47.0101 KOps/s	$\color{#d91a1a}-2.53\%$
test_step_mdp_speed[False-False-True-True-True]	0.1059ms	58.2045μs	17.1808 KOps/s	17.7467 KOps/s	$\color{#d91a1a}-3.19\%$
test_step_mdp_speed[False-False-True-True-False]	82.6940μs	36.5452μs	27.3634 KOps/s	27.5793 KOps/s	$\color{#d91a1a}-0.78\%$
test_step_mdp_speed[False-False-True-False-True]	83.4340μs	34.7333μs	28.7908 KOps/s	29.0159 KOps/s	$\color{#d91a1a}-0.78\%$
test_step_mdp_speed[False-False-True-False-False]	58.2130μs	21.2083μs	47.1514 KOps/s	47.0414 KOps/s	$\color{#35bf28}+0.23\%$
test_step_mdp_speed[False-False-False-True-True]	0.1118ms	59.3116μs	16.8601 KOps/s	17.3116 KOps/s	$\color{#d91a1a}-2.61\%$
test_step_mdp_speed[False-False-False-True-False]	62.0430μs	39.2679μs	25.4661 KOps/s	26.4377 KOps/s	$\color{#d91a1a}-3.68\%$
test_step_mdp_speed[False-False-False-False-True]	70.1030μs	36.8952μs	27.1038 KOps/s	27.6151 KOps/s	$\color{#d91a1a}-1.85\%$
test_step_mdp_speed[False-False-False-False-False]	57.3930μs	23.9778μs	41.7053 KOps/s	43.0554 KOps/s	$\color{#d91a1a}-3.14\%$
test_non_tensor_env_rollout_speed[1000-single-True]	0.8815s	0.7811s	1.2802 Ops/s	1.2791 Ops/s	$\color{#35bf28}+0.09\%$
test_non_tensor_env_rollout_speed[1000-single-False]	0.7394s	0.6451s	1.5502 Ops/s	1.5423 Ops/s	$\color{#35bf28}+0.51\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True]	1.7898s	1.7056s	0.5863 Ops/s	0.5875 Ops/s	$\color{#d91a1a}-0.19\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False]	1.5585s	1.4774s	0.6769 Ops/s	0.6757 Ops/s	$\color{#35bf28}+0.17\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True]	2.0226s	1.9465s	0.5138 Ops/s	0.5139 Ops/s	$\color{#d91a1a}-0.04\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False]	1.8073s	1.7224s	0.5806 Ops/s	0.5825 Ops/s	$\color{#d91a1a}-0.33\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True]	4.8264s	4.7116s	0.2122 Ops/s	0.2118 Ops/s	$\color{#35bf28}+0.23\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False]	4.5789s	4.4915s	0.2226 Ops/s	0.2224 Ops/s	$\color{#35bf28}+0.09\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True]	2.1067s	2.0072s	0.4982 Ops/s	0.4932 Ops/s	$\color{#35bf28}+1.02\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False]	1.8956s	1.7536s	0.5703 Ops/s	0.5839 Ops/s	$\color{#d91a1a}-2.34\%$
test_values[generalized_advantage_estimate-True-True]	10.7529ms	10.3858ms	96.2857 Ops/s	96.5973 Ops/s	$\color{#d91a1a}-0.32\%$
test_values[vec_generalized_advantage_estimate-True-True]	19.8219ms	17.4893ms	57.1777 Ops/s	56.8734 Ops/s	$\color{#35bf28}+0.54\%$
test_values[td0_return_estimate-False-False]	0.2218ms	0.1303ms	7.6751 KOps/s	7.6874 KOps/s	$\color{#d91a1a}-0.16\%$
test_values[td1_return_estimate-False-False]	28.1077ms	27.4528ms	36.4262 Ops/s	36.7105 Ops/s	$\color{#d91a1a}-0.77\%$
test_values[vec_td1_return_estimate-False-False]	17.9875ms	17.6301ms	56.7211 Ops/s	56.6703 Ops/s	$\color{#35bf28}+0.09\%$
test_values[td_lambda_return_estimate-True-False]	41.2183ms	40.4769ms	24.7055 Ops/s	24.8280 Ops/s	$\color{#d91a1a}-0.49\%$
test_values[vec_td_lambda_return_estimate-True-False]	17.8936ms	17.6236ms	56.7420 Ops/s	56.7224 Ops/s	$\color{#35bf28}+0.03\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	9.4055ms	9.3012ms	107.5128 Ops/s	109.1291 Ops/s	$\color{#d91a1a}-1.48\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.7268ms	1.5149ms	660.0976 Ops/s	671.3049 Ops/s	$\color{#d91a1a}-1.67\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4742ms	0.4171ms	2.3978 KOps/s	2.3935 KOps/s	$\color{#35bf28}+0.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	34.7977ms	34.1993ms	29.2404 Ops/s	28.7908 Ops/s	$\color{#35bf28}+1.56\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.8509ms	1.7196ms	581.5270 Ops/s	577.8634 Ops/s	$\color{#35bf28}+0.63\%$
test_dqn_speed[False-None]	1.5110ms	1.4133ms	707.5595 Ops/s	717.9486 Ops/s	$\color{#d91a1a}-1.45\%$
test_dqn_speed[False-backward]	2.0265ms	1.9180ms	521.3630 Ops/s	510.1648 Ops/s	$\color{#35bf28}+2.20\%$
test_dqn_speed[True-None]	0.6380ms	0.5251ms	1.9045 KOps/s	1.8885 KOps/s	$\color{#35bf28}+0.85\%$
test_dqn_speed[True-backward]	1.0548ms	0.9870ms	1.0132 KOps/s	842.5194 Ops/s	$\textbf{\color{#35bf28}+20.26\%}$
test_dqn_speed[reduce-overhead-None]	0.9576ms	0.5215ms	1.9177 KOps/s	1.8049 KOps/s	$\textbf{\color{#35bf28}+6.25\%}$
test_dqn_speed[reduce-overhead-backward]	1.0146ms	0.9691ms	1.0319 KOps/s	1.0108 KOps/s	$\color{#35bf28}+2.09\%$
test_ddpg_speed[False-None]	3.2746ms	2.8667ms	348.8304 Ops/s	347.2198 Ops/s	$\color{#35bf28}+0.46\%$
test_ddpg_speed[False-backward]	4.1669ms	4.0819ms	244.9845 Ops/s	241.2546 Ops/s	$\color{#35bf28}+1.55\%$
test_ddpg_speed[True-None]	1.8096ms	1.3838ms	722.6448 Ops/s	722.8956 Ops/s	$\color{#d91a1a}-0.03\%$
test_ddpg_speed[True-backward]	2.3913ms	2.3530ms	424.9879 Ops/s	378.3509 Ops/s	$\textbf{\color{#35bf28}+12.33\%}$
test_ddpg_speed[reduce-overhead-None]	1.8054ms	1.3767ms	726.3939 Ops/s	694.2969 Ops/s	$\color{#35bf28}+4.62\%$
test_ddpg_speed[reduce-overhead-backward]	2.3962ms	2.3409ms	427.1794 Ops/s	356.6113 Ops/s	$\textbf{\color{#35bf28}+19.79\%}$
test_sac_speed[False-None]	8.3897ms	7.9542ms	125.7196 Ops/s	121.9409 Ops/s	$\color{#35bf28}+3.10\%$
test_sac_speed[False-backward]	11.5570ms	11.2148ms	89.1678 Ops/s	87.2360 Ops/s	$\color{#35bf28}+2.21\%$
test_sac_speed[True-None]	2.4714ms	2.1136ms	473.1266 Ops/s	474.7843 Ops/s	$\color{#d91a1a}-0.35\%$
test_sac_speed[True-backward]	4.1041ms	3.9844ms	250.9806 Ops/s	247.8110 Ops/s	$\color{#35bf28}+1.28\%$
test_sac_speed[reduce-overhead-None]	2.5574ms	2.1053ms	474.9854 Ops/s	469.3241 Ops/s	$\color{#35bf28}+1.21\%$
test_sac_speed[reduce-overhead-backward]	4.0949ms	3.9835ms	251.0346 Ops/s	245.7311 Ops/s	$\color{#35bf28}+2.16\%$
test_redq_speed[False-None]	14.9984ms	10.4181ms	95.9867 Ops/s	94.3655 Ops/s	$\color{#35bf28}+1.72\%$
test_redq_speed[False-backward]	18.4825ms	17.7828ms	56.2342 Ops/s	56.6980 Ops/s	$\color{#d91a1a}-0.82\%$
test_redq_speed[True-None]	4.8066ms	4.3687ms	228.8996 Ops/s	227.1341 Ops/s	$\color{#35bf28}+0.78\%$
test_redq_speed[True-backward]	10.0068ms	9.7104ms	102.9823 Ops/s	97.3223 Ops/s	$\textbf{\color{#35bf28}+5.82\%}$
test_redq_speed[reduce-overhead-None]	5.3737ms	4.3082ms	232.1148 Ops/s	225.6391 Ops/s	$\color{#35bf28}+2.87\%$
test_redq_speed[reduce-overhead-backward]	9.9121ms	9.7630ms	102.4280 Ops/s	103.6632 Ops/s	$\color{#d91a1a}-1.19\%$
test_redq_deprec_speed[False-None]	11.4745ms	11.1254ms	89.8845 Ops/s	88.9013 Ops/s	$\color{#35bf28}+1.11\%$
test_redq_deprec_speed[False-backward]	16.4723ms	16.0590ms	62.2704 Ops/s	61.5660 Ops/s	$\color{#35bf28}+1.14\%$
test_redq_deprec_speed[True-None]	3.9228ms	3.5658ms	280.4458 Ops/s	271.7174 Ops/s	$\color{#35bf28}+3.21\%$
test_redq_deprec_speed[True-backward]	7.8235ms	7.5554ms	132.3557 Ops/s	129.8927 Ops/s	$\color{#35bf28}+1.90\%$
test_redq_deprec_speed[reduce-overhead-None]	4.0817ms	3.5676ms	280.3016 Ops/s	274.4424 Ops/s	$\color{#35bf28}+2.13\%$
test_redq_deprec_speed[reduce-overhead-backward]	7.9528ms	7.5719ms	132.0671 Ops/s	122.5521 Ops/s	$\textbf{\color{#35bf28}+7.76\%}$
test_td3_speed[False-None]	9.2174ms	8.0855ms	123.6785 Ops/s	123.8945 Ops/s	$\color{#d91a1a}-0.17\%$
test_td3_speed[False-backward]	11.4442ms	10.9169ms	91.6010 Ops/s	90.6400 Ops/s	$\color{#35bf28}+1.06\%$
test_td3_speed[True-None]	1.8222ms	1.7830ms	560.8545 Ops/s	550.2706 Ops/s	$\color{#35bf28}+1.92\%$
test_td3_speed[True-backward]	3.7134ms	3.5670ms	280.3509 Ops/s	278.6717 Ops/s	$\color{#35bf28}+0.60\%$
test_td3_speed[reduce-overhead-None]	1.8032ms	1.7743ms	563.6175 Ops/s	564.1370 Ops/s	$\color{#d91a1a}-0.09\%$
test_td3_speed[reduce-overhead-backward]	3.7971ms	3.6457ms	274.2970 Ops/s	275.4438 Ops/s	$\color{#d91a1a}-0.42\%$
test_cql_speed[False-None]	29.0023ms	25.9730ms	38.5015 Ops/s	38.8710 Ops/s	$\color{#d91a1a}-0.95\%$
test_cql_speed[False-backward]	35.6096ms	34.9595ms	28.6045 Ops/s	28.5921 Ops/s	$\color{#35bf28}+0.04\%$
test_cql_speed[True-None]	12.6155ms	12.2955ms	81.3309 Ops/s	82.6887 Ops/s	$\color{#d91a1a}-1.64\%$
test_cql_speed[True-backward]	18.6515ms	18.2417ms	54.8194 Ops/s	57.0414 Ops/s	$\color{#d91a1a}-3.90\%$
test_cql_speed[reduce-overhead-None]	14.5238ms	12.5517ms	79.6706 Ops/s	78.9658 Ops/s	$\color{#35bf28}+0.89\%$
test_cql_speed[reduce-overhead-backward]	18.7973ms	18.4264ms	54.2699 Ops/s	55.3037 Ops/s	$\color{#d91a1a}-1.87\%$
test_a2c_speed[False-None]	5.9472ms	5.4018ms	185.1220 Ops/s	186.7960 Ops/s	$\color{#d91a1a}-0.90\%$
test_a2c_speed[False-backward]	12.0916ms	11.8224ms	84.5855 Ops/s	85.0281 Ops/s	$\color{#d91a1a}-0.52\%$
test_a2c_speed[True-None]	3.8107ms	3.6497ms	273.9921 Ops/s	271.4418 Ops/s	$\color{#35bf28}+0.94\%$
test_a2c_speed[True-backward]	8.8295ms	8.5555ms	116.8834 Ops/s	109.9362 Ops/s	$\textbf{\color{#35bf28}+6.32\%}$
test_a2c_speed[reduce-overhead-None]	4.0443ms	3.7049ms	269.9136 Ops/s	270.9987 Ops/s	$\color{#d91a1a}-0.40\%$
test_a2c_speed[reduce-overhead-backward]	8.8579ms	8.6173ms	116.0459 Ops/s	110.2473 Ops/s	$\textbf{\color{#35bf28}+5.26\%}$
test_ppo_speed[False-None]	6.3346ms	5.9084ms	169.2506 Ops/s	168.7244 Ops/s	$\color{#35bf28}+0.31\%$
test_ppo_speed[False-backward]	12.8598ms	12.4674ms	80.2089 Ops/s	79.9907 Ops/s	$\color{#35bf28}+0.27\%$
test_ppo_speed[True-None]	4.0075ms	3.6077ms	277.1863 Ops/s	255.4867 Ops/s	$\textbf{\color{#35bf28}+8.49\%}$
test_ppo_speed[True-backward]	8.9237ms	8.4216ms	118.7425 Ops/s	114.9179 Ops/s	$\color{#35bf28}+3.33\%$
test_ppo_speed[reduce-overhead-None]	3.7320ms	3.5716ms	279.9893 Ops/s	278.8981 Ops/s	$\color{#35bf28}+0.39\%$
test_ppo_speed[reduce-overhead-backward]	9.1608ms	8.6806ms	115.1999 Ops/s	104.9824 Ops/s	$\textbf{\color{#35bf28}+9.73\%}$
test_reinforce_speed[False-None]	6.2203ms	4.6055ms	217.1336 Ops/s	219.6830 Ops/s	$\color{#d91a1a}-1.16\%$
test_reinforce_speed[False-backward]	7.7963ms	7.3777ms	135.5439 Ops/s	134.4103 Ops/s	$\color{#35bf28}+0.84\%$
test_reinforce_speed[True-None]	3.3291ms	2.8615ms	349.4726 Ops/s	343.7090 Ops/s	$\color{#35bf28}+1.68\%$
test_reinforce_speed[True-backward]	8.1866ms	7.6426ms	130.8461 Ops/s	127.4780 Ops/s	$\color{#35bf28}+2.64\%$
test_reinforce_speed[reduce-overhead-None]	3.2850ms	2.8446ms	351.5437 Ops/s	346.8349 Ops/s	$\color{#35bf28}+1.36\%$
test_reinforce_speed[reduce-overhead-backward]	8.0539ms	7.8560ms	127.2918 Ops/s	112.9424 Ops/s	$\textbf{\color{#35bf28}+12.71\%}$
test_iql_speed[False-None]	20.5626ms	19.8848ms	50.2896 Ops/s	48.4235 Ops/s	$\color{#35bf28}+3.85\%$
test_iql_speed[False-backward]	35.8182ms	30.5086ms	32.7776 Ops/s	32.5096 Ops/s	$\color{#35bf28}+0.82\%$
test_iql_speed[True-None]	9.0589ms	8.4532ms	118.2978 Ops/s	115.7474 Ops/s	$\color{#35bf28}+2.20\%$
test_iql_speed[True-backward]	17.0153ms	16.6072ms	60.2149 Ops/s	57.0371 Ops/s	$\textbf{\color{#35bf28}+5.57\%}$
test_iql_speed[reduce-overhead-None]	8.7720ms	8.4978ms	117.6771 Ops/s	116.4390 Ops/s	$\color{#35bf28}+1.06\%$
test_iql_speed[reduce-overhead-backward]	19.3372ms	17.2742ms	57.8898 Ops/s	58.7027 Ops/s	$\color{#d91a1a}-1.38\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	8.3804ms	6.1760ms	161.9161 Ops/s	163.5318 Ops/s	$\color{#d91a1a}-0.99\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7143ms	0.2843ms	3.5177 KOps/s	2.9506 KOps/s	$\textbf{\color{#35bf28}+19.22\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5267ms	0.2649ms	3.7749 KOps/s	3.0854 KOps/s	$\textbf{\color{#35bf28}+22.35\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.1347ms	5.8970ms	169.5779 Ops/s	170.5219 Ops/s	$\color{#d91a1a}-0.55\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.2269ms	0.2793ms	3.5802 KOps/s	3.0008 KOps/s	$\textbf{\color{#35bf28}+19.31\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5871ms	0.2838ms	3.5237 KOps/s	3.2986 KOps/s	$\textbf{\color{#35bf28}+6.82\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.6634ms	1.2970ms	770.9853 Ops/s	746.4426 Ops/s	$\color{#35bf28}+3.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.5873ms	1.1801ms	847.3871 Ops/s	795.3842 Ops/s	$\textbf{\color{#35bf28}+6.54\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	9.7461ms	6.1786ms	161.8502 Ops/s	166.3514 Ops/s	$\color{#d91a1a}-2.71\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0931ms	0.4868ms	2.0541 KOps/s	2.3084 KOps/s	$\textbf{\color{#d91a1a}-11.02\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7151ms	0.4904ms	2.0394 KOps/s	2.4447 KOps/s	$\textbf{\color{#d91a1a}-16.58\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.9703ms	5.8065ms	172.2221 Ops/s	170.0213 Ops/s	$\color{#35bf28}+1.29\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.1605ms	0.3352ms	2.9835 KOps/s	3.5751 KOps/s	$\textbf{\color{#d91a1a}-16.55\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5080ms	0.3122ms	3.2033 KOps/s	3.7630 KOps/s	$\textbf{\color{#d91a1a}-14.88\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.0097ms	5.7571ms	173.6995 Ops/s	170.8159 Ops/s	$\color{#35bf28}+1.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.0160ms	0.2900ms	3.4481 KOps/s	3.5413 KOps/s	$\color{#d91a1a}-2.63\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7138ms	0.2611ms	3.8303 KOps/s	3.2741 KOps/s	$\textbf{\color{#35bf28}+16.99\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.1043ms	6.0103ms	166.3800 Ops/s	167.1138 Ops/s	$\color{#d91a1a}-0.44\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	2.4642ms	0.4602ms	2.1730 KOps/s	1.9443 KOps/s	$\textbf{\color{#35bf28}+11.76\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6480ms	0.4634ms	2.1581 KOps/s	2.0876 KOps/s	$\color{#35bf28}+3.38\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	6.4352ms	4.9688ms	201.2540 Ops/s	197.3825 Ops/s	$\color{#35bf28}+1.96\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	8.0194ms	2.3460ms	426.2497 Ops/s	406.4474 Ops/s	$\color{#35bf28}+4.87\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	8.0383ms	1.2081ms	827.7176 Ops/s	925.4371 Ops/s	$\textbf{\color{#d91a1a}-10.56\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	7.1553ms	5.0592ms	197.6595 Ops/s	52.5994 Ops/s	$\textbf{\color{#35bf28}+275.78\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.6133s	14.4176ms	69.3595 Ops/s	691.2709 Ops/s	$\textbf{\color{#d91a1a}-89.97\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	7.2252ms	1.2157ms	822.5546 Ops/s	783.8800 Ops/s	$\color{#35bf28}+4.93\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	6.8735ms	5.2322ms	191.1229 Ops/s	188.7132 Ops/s	$\color{#35bf28}+1.28\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	3.9903ms	1.7545ms	569.9618 Ops/s	431.7836 Ops/s	$\textbf{\color{#35bf28}+32.00\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.5725ms	1.3898ms	719.5096 Ops/s	682.2018 Ops/s	$\textbf{\color{#35bf28}+5.47\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	35.8883ms	33.8314ms	29.5583 Ops/s	28.9017 Ops/s	$\color{#35bf28}+2.27\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	19.5070ms	17.7165ms	56.4447 Ops/s	56.3047 Ops/s	$\color{#35bf28}+0.25\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	37.0749ms	34.8269ms	28.7134 Ops/s	27.7751 Ops/s	$\color{#35bf28}+3.38\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.3766ms	17.8995ms	55.8674 Ops/s	54.8856 Ops/s	$\color{#35bf28}+1.79\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	42.4461ms	37.0274ms	27.0070 Ops/s	26.5975 Ops/s	$\color{#35bf28}+1.54\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.6789ms	19.4197ms	51.4940 Ops/s	51.1053 Ops/s	$\color{#35bf28}+0.76\%$