Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] transformers policy #2825

Merged
merged 7 commits into from
Mar 11, 2025
Merged

[Feature] transformers policy #2825

merged 7 commits into from
Mar 11, 2025

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Mar 4, 2025 •
edited
Loading

@vmoens
Update
d5cdc48 
[ghstack-poisoned]
This was referenced Mar 4, 2025
Merged
Merged
Merged
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 4, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2825

Note: Links to docs will display an error until the docs builds have been completed.

❌ 19 New Failures

As of commit 8d13206 with merge base 6e40548 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Mar 4, 2025
@vmoens
[Feature] transformers policy
b3224ee 
ghstack-source-id: 5a6b6e8eccafb7c89ece8557223ffc678711b449
Pull Request resolved: #2825
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 4, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 4, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}9$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.6157s 0.5281s 1.8936 Ops/s 1.9198 Ops/s $\color{#d91a1a}-1.36\%$
test_transformed 1.1454s 1.0501s 0.9523 Ops/s 0.9832 Ops/s $\color{#d91a1a}-3.14\%$
test_serial 1.6320s 1.5337s 0.6520 Ops/s 0.6546 Ops/s $\color{#d91a1a}-0.40\%$
test_parallel 1.4295s 1.3416s 0.7454 Ops/s 0.7502 Ops/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-True-True-True-True] 0.2311ms 29.9600μs 33.3778 KOps/s 32.8379 KOps/s $\color{#35bf28}+1.64\%$
test_step_mdp_speed[True-True-True-True-False] 46.0760μs 17.8251μs 56.1005 KOps/s 54.0565 KOps/s $\color{#35bf28}+3.78\%$
test_step_mdp_speed[True-True-True-False-True] 49.7030μs 17.3328μs 57.6942 KOps/s 57.7734 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[True-True-True-False-False] 37.1490μs 10.1444μs 98.5770 KOps/s 96.3006 KOps/s $\color{#35bf28}+2.36\%$
test_step_mdp_speed[True-True-False-True-True] 87.8540μs 32.5191μs 30.7511 KOps/s 30.7026 KOps/s $\color{#35bf28}+0.16\%$
test_step_mdp_speed[True-True-False-True-False] 56.7360μs 19.8441μs 50.3928 KOps/s 49.2566 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[True-True-False-False-True] 74.7370μs 18.8514μs 53.0465 KOps/s 52.1682 KOps/s $\color{#35bf28}+1.68\%$
test_step_mdp_speed[True-True-False-False-False] 61.9760μs 11.9972μs 83.3530 KOps/s 81.4074 KOps/s $\color{#35bf28}+2.39\%$
test_step_mdp_speed[True-False-True-True-True] 0.1351ms 34.3497μs 29.1124 KOps/s 28.9069 KOps/s $\color{#35bf28}+0.71\%$
test_step_mdp_speed[True-False-True-True-False] 0.1254ms 21.8595μs 45.7468 KOps/s 44.7656 KOps/s $\color{#35bf28}+2.19\%$
test_step_mdp_speed[True-False-True-False-True] 46.6170μs 19.0569μs 52.4744 KOps/s 52.8485 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[True-False-True-False-False] 70.2440μs 11.9097μs 83.9653 KOps/s 81.5127 KOps/s $\color{#35bf28}+3.01\%$
test_step_mdp_speed[True-False-False-True-True] 76.3430μs 36.1156μs 27.6888 KOps/s 27.7399 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[True-False-False-True-False] 51.1250μs 23.2649μs 42.9832 KOps/s 41.6931 KOps/s $\color{#35bf28}+3.09\%$
test_step_mdp_speed[True-False-False-False-True] 88.0610μs 20.9016μs 47.8432 KOps/s 47.7179 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-False-False-False-False] 34.3440μs 13.6881μs 73.0564 KOps/s 71.4573 KOps/s $\color{#35bf28}+2.24\%$
test_step_mdp_speed[False-True-True-True-True] 90.6690μs 34.3281μs 29.1307 KOps/s 28.8291 KOps/s $\color{#35bf28}+1.05\%$
test_step_mdp_speed[False-True-True-True-False] 73.4570μs 21.6981μs 46.0869 KOps/s 45.1390 KOps/s $\color{#35bf28}+2.10\%$
test_step_mdp_speed[False-True-True-False-True] 44.6940μs 21.7523μs 45.9722 KOps/s 45.4613 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[False-True-True-False-False] 0.1037ms 13.5188μs 73.9713 KOps/s 72.6924 KOps/s $\color{#35bf28}+1.76\%$
test_step_mdp_speed[False-True-False-True-True] 82.2730μs 36.8360μs 27.1473 KOps/s 27.3951 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-True-False-True-False] 76.4400μs 23.2330μs 43.0421 KOps/s 41.6106 KOps/s $\color{#35bf28}+3.44\%$
test_step_mdp_speed[False-True-False-False-True] 2.7730ms 23.7385μs 42.1257 KOps/s 42.1874 KOps/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[False-True-False-False-False] 42.3690μs 15.1671μs 65.9323 KOps/s 64.5917 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[False-False-True-True-True] 0.1102ms 37.4847μs 26.6776 KOps/s 26.3382 KOps/s $\color{#35bf28}+1.29\%$
test_step_mdp_speed[False-False-True-True-False] 0.1174ms 25.1833μs 39.7089 KOps/s 38.5942 KOps/s $\color{#35bf28}+2.89\%$
test_step_mdp_speed[False-False-True-False-True] 56.9560μs 23.3228μs 42.8765 KOps/s 42.0664 KOps/s $\color{#35bf28}+1.93\%$
test_step_mdp_speed[False-False-True-False-False] 60.7500μs 15.1452μs 66.0274 KOps/s 64.3701 KOps/s $\color{#35bf28}+2.57\%$
test_step_mdp_speed[False-False-False-True-True] 89.5370μs 39.0430μs 25.6128 KOps/s 25.2359 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[False-False-False-True-False] 70.3110μs 26.6828μs 37.4773 KOps/s 36.2873 KOps/s $\color{#35bf28}+3.28\%$
test_step_mdp_speed[False-False-False-False-True] 78.6760μs 24.8700μs 40.2090 KOps/s 39.1716 KOps/s $\color{#35bf28}+2.65\%$
test_step_mdp_speed[False-False-False-False-False] 85.9340μs 16.7768μs 59.6060 KOps/s 56.2328 KOps/s $\textbf{\color{#35bf28}+6.00\%}$
test_values[generalized_advantage_estimate-True-True] 12.6949ms 9.7433ms 102.6344 Ops/s 102.9597 Ops/s $\color{#d91a1a}-0.32\%$
test_values[vec_generalized_advantage_estimate-True-True] 31.5098ms 26.6277ms 37.5548 Ops/s 41.1026 Ops/s $\textbf{\color{#d91a1a}-8.63\%}$
test_values[td0_return_estimate-False-False] 0.2406ms 0.1842ms 5.4295 KOps/s 5.5062 KOps/s $\color{#d91a1a}-1.39\%$
test_values[td1_return_estimate-False-False] 24.0506ms 23.7809ms 42.0506 Ops/s 41.8782 Ops/s $\color{#35bf28}+0.41\%$
test_values[vec_td1_return_estimate-False-False] 28.5582ms 26.3104ms 38.0078 Ops/s 40.9398 Ops/s $\textbf{\color{#d91a1a}-7.16\%}$
test_values[td_lambda_return_estimate-True-False] 34.9150ms 34.2373ms 29.2079 Ops/s 29.0005 Ops/s $\color{#35bf28}+0.72\%$
test_values[vec_td_lambda_return_estimate-True-False] 29.2403ms 26.3257ms 37.9857 Ops/s 40.8930 Ops/s $\textbf{\color{#d91a1a}-7.11\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.7491ms 8.3310ms 120.0338 Ops/s 118.6611 Ops/s $\color{#35bf28}+1.16\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.1835ms 1.8122ms 551.8172 Ops/s 537.4924 Ops/s $\color{#35bf28}+2.67\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5568ms 0.3626ms 2.7581 KOps/s 2.7015 KOps/s $\color{#35bf28}+2.10\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 45.8200ms 43.1336ms 23.1838 Ops/s 26.8025 Ops/s $\textbf{\color{#d91a1a}-13.50\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.2874ms 3.4555ms 289.3914 Ops/s 289.2367 Ops/s $\color{#35bf28}+0.05\%$
test_dqn_speed[False-None] 6.1613ms 1.4119ms 708.2643 Ops/s 703.0920 Ops/s $\color{#35bf28}+0.74\%$
test_dqn_speed[False-backward] 2.0214ms 1.8916ms 528.6578 Ops/s 527.6366 Ops/s $\color{#35bf28}+0.19\%$
test_dqn_speed[True-None] 0.9140ms 0.5728ms 1.7457 KOps/s 1.7539 KOps/s $\color{#d91a1a}-0.47\%$
test_dqn_speed[True-backward] 1.1405ms 1.0167ms 983.6142 Ops/s 790.5356 Ops/s $\textbf{\color{#35bf28}+24.42\%}$
test_dqn_speed[reduce-overhead-None] 0.7138ms 0.5657ms 1.7677 KOps/s 1.7701 KOps/s $\color{#d91a1a}-0.14\%$
test_dqn_speed[reduce-overhead-backward] 1.0625ms 0.9892ms 1.0109 KOps/s 1.0179 KOps/s $\color{#d91a1a}-0.69\%$
test_ddpg_speed[False-None] 3.3988ms 2.8942ms 345.5195 Ops/s 342.6749 Ops/s $\color{#35bf28}+0.83\%$
test_ddpg_speed[False-backward] 4.1531ms 4.0138ms 249.1400 Ops/s 248.5312 Ops/s $\color{#35bf28}+0.24\%$
test_ddpg_speed[True-None] 1.8042ms 1.4441ms 692.4726 Ops/s 693.3351 Ops/s $\color{#d91a1a}-0.12\%$
test_ddpg_speed[True-backward] 2.5259ms 2.3572ms 424.2324 Ops/s 424.2131 Ops/s $+0.00\%$
test_ddpg_speed[reduce-overhead-None] 1.7943ms 1.4526ms 688.4055 Ops/s 513.7332 Ops/s $\textbf{\color{#35bf28}+34.00\%}$
test_ddpg_speed[reduce-overhead-backward] 3.1490ms 2.3611ms 423.5362 Ops/s 428.3528 Ops/s $\color{#d91a1a}-1.12\%$
test_sac_speed[False-None] 9.7636ms 8.2376ms 121.3943 Ops/s 123.3808 Ops/s $\color{#d91a1a}-1.61\%$
test_sac_speed[False-backward] 11.3831ms 10.8807ms 91.9059 Ops/s 91.3883 Ops/s $\color{#35bf28}+0.57\%$
test_sac_speed[True-None] 3.1387ms 2.6034ms 384.1131 Ops/s 368.2723 Ops/s $\color{#35bf28}+4.30\%$
test_sac_speed[True-backward] 4.4729ms 4.3022ms 232.4413 Ops/s 204.1028 Ops/s $\textbf{\color{#35bf28}+13.88\%}$
test_sac_speed[reduce-overhead-None] 2.8441ms 2.6115ms 382.9192 Ops/s 373.1191 Ops/s $\color{#35bf28}+2.63\%$
test_sac_speed[reduce-overhead-backward] 5.0943ms 4.4732ms 223.5547 Ops/s 225.9867 Ops/s $\color{#d91a1a}-1.08\%$
test_redq_speed[False-None] 20.0186ms 13.9716ms 71.5738 Ops/s 74.7985 Ops/s $\color{#d91a1a}-4.31\%$
test_redq_speed[False-backward] 25.0686ms 22.9462ms 43.5801 Ops/s 43.5035 Ops/s $\color{#35bf28}+0.18\%$
test_redq_speed[True-None] 8.7759ms 7.3569ms 135.9275 Ops/s 131.7131 Ops/s $\color{#35bf28}+3.20\%$
test_redq_speed[True-backward] 16.4050ms 15.2741ms 65.4701 Ops/s 67.8032 Ops/s $\color{#d91a1a}-3.44\%$
test_redq_speed[reduce-overhead-None] 8.8252ms 7.4339ms 134.5184 Ops/s 141.2908 Ops/s $\color{#d91a1a}-4.79\%$
test_redq_speed[reduce-overhead-backward] 15.6274ms 14.7970ms 67.5811 Ops/s 68.0656 Ops/s $\color{#d91a1a}-0.71\%$
test_redq_deprec_speed[False-None] 16.1012ms 13.7972ms 72.4783 Ops/s 74.2712 Ops/s $\color{#d91a1a}-2.41\%$
test_redq_deprec_speed[False-backward] 21.4343ms 19.0418ms 52.5161 Ops/s 51.2972 Ops/s $\color{#35bf28}+2.38\%$
test_redq_deprec_speed[True-None] 6.2432ms 5.3791ms 185.9035 Ops/s 185.6094 Ops/s $\color{#35bf28}+0.16\%$
test_redq_deprec_speed[True-backward] 11.7049ms 10.6477ms 93.9172 Ops/s 91.9254 Ops/s $\color{#35bf28}+2.17\%$
test_redq_deprec_speed[reduce-overhead-None] 6.3280ms 5.4358ms 183.9664 Ops/s 183.5528 Ops/s $\color{#35bf28}+0.23\%$
test_redq_deprec_speed[reduce-overhead-backward] 11.6863ms 10.8275ms 92.3572 Ops/s 92.9287 Ops/s $\color{#d91a1a}-0.62\%$
test_td3_speed[False-None] 9.1549ms 8.4489ms 118.3589 Ops/s 118.5596 Ops/s $\color{#d91a1a}-0.17\%$
test_td3_speed[False-backward] 11.2855ms 10.8105ms 92.5025 Ops/s 87.2373 Ops/s $\textbf{\color{#35bf28}+6.04\%}$
test_td3_speed[True-None] 2.4624ms 2.3138ms 432.1811 Ops/s 434.5991 Ops/s $\color{#d91a1a}-0.56\%$
test_td3_speed[True-backward] 4.5556ms 4.0759ms 245.3470 Ops/s 244.7895 Ops/s $\color{#35bf28}+0.23\%$
test_td3_speed[reduce-overhead-None] 2.6196ms 2.3133ms 432.2918 Ops/s 437.1378 Ops/s $\color{#d91a1a}-1.11\%$
test_td3_speed[reduce-overhead-backward] 4.4220ms 4.0592ms 246.3514 Ops/s 244.9318 Ops/s $\color{#35bf28}+0.58\%$
test_cql_speed[False-None] 37.5611ms 36.5857ms 27.3331 Ops/s 26.1790 Ops/s $\color{#35bf28}+4.41\%$
test_cql_speed[False-backward] 51.2157ms 48.1581ms 20.7649 Ops/s 20.5471 Ops/s $\color{#35bf28}+1.06\%$
test_cql_speed[True-None] 25.7985ms 23.7989ms 42.0188 Ops/s 44.0664 Ops/s $\color{#d91a1a}-4.65\%$
test_cql_speed[True-backward] 31.6994ms 29.8269ms 33.5268 Ops/s 33.1582 Ops/s $\color{#35bf28}+1.11\%$
test_cql_speed[reduce-overhead-None] 24.0744ms 22.6556ms 44.1392 Ops/s 44.1092 Ops/s $\color{#35bf28}+0.07\%$
test_cql_speed[reduce-overhead-backward] 30.8017ms 29.7045ms 33.6650 Ops/s 33.7102 Ops/s $\color{#d91a1a}-0.13\%$
test_a2c_speed[False-None] 8.6579ms 7.3358ms 136.3187 Ops/s 137.0312 Ops/s $\color{#d91a1a}-0.52\%$
test_a2c_speed[False-backward] 15.7764ms 14.6295ms 68.3550 Ops/s 67.7740 Ops/s $\color{#35bf28}+0.86\%$
test_a2c_speed[True-None] 6.4055ms 4.8249ms 207.2588 Ops/s 210.2991 Ops/s $\color{#d91a1a}-1.45\%$
test_a2c_speed[True-backward] 13.3312ms 11.5460ms 86.6104 Ops/s 86.7532 Ops/s $\color{#d91a1a}-0.16\%$
test_a2c_speed[reduce-overhead-None] 5.5826ms 4.7968ms 208.4717 Ops/s 203.4949 Ops/s $\color{#35bf28}+2.45\%$
test_a2c_speed[reduce-overhead-backward] 12.5096ms 11.6146ms 86.0985 Ops/s 86.6033 Ops/s $\color{#d91a1a}-0.58\%$
test_ppo_speed[False-None] 9.2466ms 7.5910ms 131.7357 Ops/s 129.3510 Ops/s $\color{#35bf28}+1.84\%$
test_ppo_speed[False-backward] 18.7498ms 15.1268ms 66.1077 Ops/s 65.2465 Ops/s $\color{#35bf28}+1.32\%$
test_ppo_speed[True-None] 5.7444ms 5.1297ms 194.9421 Ops/s 193.6584 Ops/s $\color{#35bf28}+0.66\%$
test_ppo_speed[True-backward] 11.7121ms 11.1303ms 89.8452 Ops/s 87.1202 Ops/s $\color{#35bf28}+3.13\%$
test_ppo_speed[reduce-overhead-None] 5.6037ms 5.0719ms 197.1638 Ops/s 195.3967 Ops/s $\color{#35bf28}+0.90\%$
test_ppo_speed[reduce-overhead-backward] 12.1213ms 11.2384ms 88.9807 Ops/s 87.3820 Ops/s $\color{#35bf28}+1.83\%$
test_reinforce_speed[False-None] 8.0072ms 6.6525ms 150.3199 Ops/s 148.7967 Ops/s $\color{#35bf28}+1.02\%$
test_reinforce_speed[False-backward] 11.5705ms 10.0182ms 99.8182 Ops/s 98.7591 Ops/s $\color{#35bf28}+1.07\%$
test_reinforce_speed[True-None] 5.0593ms 4.1550ms 240.6716 Ops/s 237.9188 Ops/s $\color{#35bf28}+1.16\%$
test_reinforce_speed[True-backward] 11.6108ms 10.6111ms 94.2413 Ops/s 94.1244 Ops/s $\color{#35bf28}+0.12\%$
test_reinforce_speed[reduce-overhead-None] 4.9578ms 4.1254ms 242.3996 Ops/s 238.3999 Ops/s $\color{#35bf28}+1.68\%$
test_reinforce_speed[reduce-overhead-backward] 11.9062ms 10.4128ms 96.0353 Ops/s 91.3016 Ops/s $\textbf{\color{#35bf28}+5.18\%}$
test_iql_speed[False-None] 40.3731ms 33.5921ms 29.7689 Ops/s 30.5806 Ops/s $\color{#d91a1a}-2.65\%$
test_iql_speed[False-backward] 48.7945ms 46.4927ms 21.5088 Ops/s 21.9014 Ops/s $\color{#d91a1a}-1.79\%$
test_iql_speed[True-None] 17.6513ms 16.1607ms 61.8784 Ops/s 61.3985 Ops/s $\color{#35bf28}+0.78\%$
test_iql_speed[True-backward] 29.8445ms 27.6245ms 36.1998 Ops/s 36.2700 Ops/s $\color{#d91a1a}-0.19\%$
test_iql_speed[reduce-overhead-None] 17.3965ms 16.1492ms 61.9225 Ops/s 62.0714 Ops/s $\color{#d91a1a}-0.24\%$
test_iql_speed[reduce-overhead-backward] 28.6784ms 27.9146ms 35.8235 Ops/s 36.5333 Ops/s $\color{#d91a1a}-1.94\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.7878ms 5.0522ms 197.9324 Ops/s 199.9567 Ops/s $\color{#d91a1a}-1.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8061ms 0.5306ms 1.8846 KOps/s 1.9358 KOps/s $\color{#d91a1a}-2.64\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7291ms 0.5059ms 1.9767 KOps/s 2.0040 KOps/s $\color{#d91a1a}-1.36\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.6487ms 4.8766ms 205.0619 Ops/s 213.2480 Ops/s $\color{#d91a1a}-3.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.8426ms 0.5279ms 1.8944 KOps/s 1.9430 KOps/s $\color{#d91a1a}-2.50\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8194ms 0.5046ms 1.9817 KOps/s 2.0353 KOps/s $\color{#d91a1a}-2.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.9817ms 1.6640ms 600.9471 Ops/s 605.1636 Ops/s $\color{#d91a1a}-0.70\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.8844ms 1.6017ms 624.3283 Ops/s 640.2821 Ops/s $\color{#d91a1a}-2.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.6436ms 4.9379ms 202.5150 Ops/s 207.5141 Ops/s $\color{#d91a1a}-2.41\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2314ms 0.6721ms 1.4878 KOps/s 1.5103 KOps/s $\color{#d91a1a}-1.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9547ms 0.6408ms 1.5605 KOps/s 1.5732 KOps/s $\color{#d91a1a}-0.81\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5351ms 4.7432ms 210.8282 Ops/s 214.0798 Ops/s $\color{#d91a1a}-1.52\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.3720ms 0.5275ms 1.8957 KOps/s 1.9223 KOps/s $\color{#d91a1a}-1.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7852ms 0.5116ms 1.9548 KOps/s 2.0268 KOps/s $\color{#d91a1a}-3.55\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1450ms 4.7046ms 212.5584 Ops/s 210.8272 Ops/s $\color{#35bf28}+0.82\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.4689ms 0.5287ms 1.8913 KOps/s 1.9536 KOps/s $\color{#d91a1a}-3.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7762ms 0.5046ms 1.9816 KOps/s 2.0212 KOps/s $\color{#d91a1a}-1.96\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.7074ms 5.0876ms 196.5548 Ops/s 210.4517 Ops/s $\textbf{\color{#d91a1a}-6.60\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.9446ms 0.6899ms 1.4495 KOps/s 1.5185 KOps/s $\color{#d91a1a}-4.55\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9379ms 0.6473ms 1.5449 KOps/s 1.5772 KOps/s $\color{#d91a1a}-2.05\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.3741ms 4.3845ms 228.0775 Ops/s 246.9433 Ops/s $\textbf{\color{#d91a1a}-7.64\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 7.1286ms 2.3769ms 420.7148 Ops/s 438.8950 Ops/s $\color{#d91a1a}-4.14\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.9867ms 1.4449ms 692.0835 Ops/s 729.6241 Ops/s $\textbf{\color{#d91a1a}-5.15\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 12.5730ms 4.6743ms 213.9343 Ops/s 23.1374 Ops/s $\textbf{\color{#35bf28}+824.62\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 7.6269ms 2.3721ms 421.5604 Ops/s 411.1481 Ops/s $\color{#35bf28}+2.53\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.2149ms 1.3966ms 716.0412 Ops/s 712.3750 Ops/s $\color{#35bf28}+0.51\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.8177s 20.7158ms 48.2723 Ops/s 221.2637 Ops/s $\textbf{\color{#d91a1a}-78.18\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 6.0044ms 2.5103ms 398.3630 Ops/s 400.7880 Ops/s $\color{#d91a1a}-0.61\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.7682ms 1.6287ms 614.0016 Ops/s 599.9564 Ops/s $\color{#35bf28}+2.34\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 14.5567ms 12.8372ms 77.8986 Ops/s 82.7978 Ops/s $\textbf{\color{#d91a1a}-5.92\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 15.5218ms 14.4254ms 69.3221 Ops/s 69.4787 Ops/s $\color{#d91a1a}-0.23\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 23.8058ms 21.6971ms 46.0891 Ops/s 47.2702 Ops/s $\color{#d91a1a}-2.50\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 15.9634ms 14.7761ms 67.6770 Ops/s 67.8012 Ops/s $\color{#d91a1a}-0.18\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 22.5669ms 21.4488ms 46.6228 Ops/s 48.2338 Ops/s $\color{#d91a1a}-3.34\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.9104ms 15.8736ms 62.9977 Ops/s 63.3654 Ops/s $\color{#d91a1a}-0.58\%$

@vmoens
Update
d370867 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Mar 4, 2025
@vmoens
[Feature] transformers policy
0b3f7a4 
ghstack-source-id: 6f509ae0803783176c0b2acecbac20e848d96273
Pull Request resolved: #2825
@vmoens
Update
fc163e4 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Mar 5, 2025
@vmoens
[Feature] transformers policy
215ed76 
ghstack-source-id: b333d2e94ee8387e60dfb303fc9365dd0e6e43a8
Pull Request resolved: #2825
@vmoens
Update
8302b2e 
[ghstack-poisoned]
This was referenced Mar 5, 2025
Merged
Open
Open
@vmoens
Update
5a174ec 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Mar 6, 2025
@vmoens
[Feature] transformers policy
98fdba9 
ghstack-source-id: c2349a8ddff03e4fcd55c07d3dd1a64592dad186
Pull Request resolved: #2825
vmoens added a commit that referenced this pull request Mar 6, 2025
@vmoens
[Feature] transformers policy
f3fa95b 
ghstack-source-id: c2349a8ddff03e4fcd55c07d3dd1a64592dad186
Pull Request resolved: #2825
vmoens added 2 commits March 6, 2025 14:29
@vmoens
Update
ef90882 
[ghstack-poisoned]
@vmoens
Update
8d13206 
[ghstack-poisoned]
@vmoens vmoens added the enhancement New feature or request label Mar 11, 2025
@vmoens vmoens merged commit 8d13206 into gh/vmoens/102/base Mar 11, 2025
49 of 72 checks passed
vmoens added a commit that referenced this pull request Mar 11, 2025
@vmoens
[Feature] transformers policy
eea932c 
ghstack-source-id: 870c221b4ebae132a44944f0be0ee78da540d115
Pull Request resolved: #2825
@vmoens vmoens deleted the gh/vmoens/102/head branch March 11, 2025 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot