Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Right log-prob size in transformer wrapper #2854

Merged
merged 6 commits into from
Mar 20, 2025
Merged

[BugFix] Right log-prob size in transformer wrapper #2854

merged 6 commits into from
Mar 20, 2025

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Mar 14, 2025 •
edited
Loading

@vmoens
Update
3ed348a 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2854

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Mar 14, 2025
Merged
Open
Closed
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 14, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 14, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.5006s 0.4978s 2.0086 Ops/s 1.9099 Ops/s $\textbf{\color{#35bf28}+5.17\%}$
test_transformed 1.1117s 1.0280s 0.9728 Ops/s 0.9758 Ops/s $\color{#d91a1a}-0.31\%$
test_serial 1.5117s 1.5105s 0.6620 Ops/s 0.6437 Ops/s $\color{#35bf28}+2.85\%$
test_parallel 1.3841s 1.3019s 0.7681 Ops/s 0.7574 Ops/s $\color{#35bf28}+1.42\%$
test_step_mdp_speed[True-True-True-True-True] 0.1519ms 29.8646μs 33.4845 KOps/s 32.9484 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-True-True-True-False] 46.1460μs 17.6302μs 56.7210 KOps/s 54.7494 KOps/s $\color{#35bf28}+3.60\%$
test_step_mdp_speed[True-True-True-False-True] 46.0760μs 17.0048μs 58.8068 KOps/s 58.0381 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-True-True-False-False] 64.4880μs 9.8653μs 101.3659 KOps/s 99.9253 KOps/s $\color{#35bf28}+1.44\%$
test_step_mdp_speed[True-True-False-True-True] 70.2110μs 31.9123μs 31.3358 KOps/s 30.5636 KOps/s $\color{#35bf28}+2.53\%$
test_step_mdp_speed[True-True-False-True-False] 44.7630μs 19.4133μs 51.5111 KOps/s 50.0065 KOps/s $\color{#35bf28}+3.01\%$
test_step_mdp_speed[True-True-False-False-True] 63.2870μs 18.9752μs 52.7003 KOps/s 52.0470 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[True-True-False-False-False] 0.6101ms 11.8151μs 84.6374 KOps/s 84.5426 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-False-True-True-True] 0.1275ms 33.8527μs 29.5397 KOps/s 28.0951 KOps/s $\textbf{\color{#35bf28}+5.14\%}$
test_step_mdp_speed[True-False-True-True-False] 51.4060μs 21.3616μs 46.8131 KOps/s 45.5565 KOps/s $\color{#35bf28}+2.76\%$
test_step_mdp_speed[True-False-True-False-True] 45.9760μs 18.6912μs 53.5012 KOps/s 52.8799 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-False-True-False-False] 33.7430μs 11.7330μs 85.2297 KOps/s 83.4603 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[True-False-False-True-True] 69.7900μs 35.8390μs 27.9025 KOps/s 27.9077 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-False-False-True-False] 57.4170μs 22.9888μs 43.4994 KOps/s 42.9025 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-False-False-False-True] 64.1090μs 20.6365μs 48.4578 KOps/s 48.1479 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[True-False-False-False-False] 39.1630μs 13.5983μs 73.5385 KOps/s 73.1985 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-True-True-True-True] 76.3220μs 33.9852μs 29.4246 KOps/s 28.6849 KOps/s $\color{#35bf28}+2.58\%$
test_step_mdp_speed[False-True-True-True-False] 46.1260μs 21.4861μs 46.5418 KOps/s 45.9879 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[False-True-True-False-True] 49.6820μs 21.6239μs 46.2451 KOps/s 45.3349 KOps/s $\color{#35bf28}+2.01\%$
test_step_mdp_speed[False-True-True-False-False] 30.8580μs 13.2559μs 75.4380 KOps/s 74.9653 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[False-True-False-True-True] 2.3882ms 35.8329μs 27.9073 KOps/s 27.8290 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[False-True-False-True-False] 59.0810μs 23.1411μs 43.2132 KOps/s 42.7535 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[False-True-False-False-True] 55.4440μs 23.3547μs 42.8179 KOps/s 42.0365 KOps/s $\color{#35bf28}+1.86\%$
test_step_mdp_speed[False-True-False-False-False] 45.4340μs 14.9579μs 66.8545 KOps/s 65.9280 KOps/s $\color{#35bf28}+1.41\%$
test_step_mdp_speed[False-False-True-True-True] 0.6148ms 37.5423μs 26.6366 KOps/s 26.2484 KOps/s $\color{#35bf28}+1.48\%$
test_step_mdp_speed[False-False-True-True-False] 57.2570μs 24.8595μs 40.2260 KOps/s 39.1844 KOps/s $\color{#35bf28}+2.66\%$
test_step_mdp_speed[False-False-True-False-True] 70.8920μs 23.2967μs 42.9245 KOps/s 42.1211 KOps/s $\color{#35bf28}+1.91\%$
test_step_mdp_speed[False-False-True-False-False] 40.0850μs 14.8585μs 67.3014 KOps/s 66.7137 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-False-False-True-True] 75.0700μs 38.8324μs 25.7517 KOps/s 25.4188 KOps/s $\color{#35bf28}+1.31\%$
test_step_mdp_speed[False-False-False-True-False] 61.0030μs 26.4560μs 37.7986 KOps/s 36.9776 KOps/s $\color{#35bf28}+2.22\%$
test_step_mdp_speed[False-False-False-False-True] 65.1520μs 24.7658μs 40.3783 KOps/s 39.5092 KOps/s $\color{#35bf28}+2.20\%$
test_step_mdp_speed[False-False-False-False-False] 45.0540μs 16.5365μs 60.4724 KOps/s 59.5353 KOps/s $\color{#35bf28}+1.57\%$
test_values[generalized_advantage_estimate-True-True] 9.8655ms 9.5974ms 104.1951 Ops/s 99.3985 Ops/s $\color{#35bf28}+4.83\%$
test_values[vec_generalized_advantage_estimate-True-True] 31.6923ms 26.0727ms 38.3542 Ops/s 40.9738 Ops/s $\textbf{\color{#d91a1a}-6.39\%}$
test_values[td0_return_estimate-False-False] 0.2528ms 0.1796ms 5.5672 KOps/s 5.2305 KOps/s $\textbf{\color{#35bf28}+6.44\%}$
test_values[td1_return_estimate-False-False] 24.5654ms 24.0199ms 41.6321 Ops/s 40.5916 Ops/s $\color{#35bf28}+2.56\%$
test_values[vec_td1_return_estimate-False-False] 28.7707ms 26.2070ms 38.1578 Ops/s 40.7252 Ops/s $\textbf{\color{#d91a1a}-6.30\%}$
test_values[td_lambda_return_estimate-True-False] 37.6946ms 34.6613ms 28.8506 Ops/s 28.0395 Ops/s $\color{#35bf28}+2.89\%$
test_values[vec_td_lambda_return_estimate-True-False] 28.5878ms 26.0906ms 38.3279 Ops/s 40.9890 Ops/s $\textbf{\color{#d91a1a}-6.49\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 10.7803ms 8.3223ms 120.1584 Ops/s 114.0464 Ops/s $\textbf{\color{#35bf28}+5.36\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.5242ms 1.9717ms 507.1703 Ops/s 512.0222 Ops/s $\color{#d91a1a}-0.95\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5708ms 0.3763ms 2.6576 KOps/s 2.6967 KOps/s $\color{#d91a1a}-1.45\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 46.8097ms 44.4174ms 22.5137 Ops/s 23.2079 Ops/s $\color{#d91a1a}-2.99\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8903ms 3.4444ms 290.3224 Ops/s 288.6801 Ops/s $\color{#35bf28}+0.57\%$
test_dqn_speed[False-None] 1.6609ms 1.4252ms 701.6538 Ops/s 700.6457 Ops/s $\color{#35bf28}+0.14\%$
test_dqn_speed[False-backward] 2.4648ms 1.9588ms 510.5125 Ops/s 515.1958 Ops/s $\color{#d91a1a}-0.91\%$
test_dqn_speed[True-None] 0.7898ms 0.5647ms 1.7707 KOps/s 1.7376 KOps/s $\color{#35bf28}+1.91\%$
test_dqn_speed[True-backward] 1.1006ms 0.9966ms 1.0034 KOps/s 999.2761 Ops/s $\color{#35bf28}+0.42\%$
test_dqn_speed[reduce-overhead-None] 0.7329ms 0.5635ms 1.7746 KOps/s 1.7663 KOps/s $\color{#35bf28}+0.47\%$
test_dqn_speed[reduce-overhead-backward] 1.0256ms 0.9761ms 1.0244 KOps/s 966.7622 Ops/s $\textbf{\color{#35bf28}+5.97\%}$
test_ddpg_speed[False-None] 3.7150ms 2.9405ms 340.0824 Ops/s 342.1490 Ops/s $\color{#d91a1a}-0.60\%$
test_ddpg_speed[False-backward] 4.2433ms 4.0835ms 244.8858 Ops/s 245.6843 Ops/s $\color{#d91a1a}-0.33\%$
test_ddpg_speed[True-None] 1.8651ms 1.4306ms 699.0030 Ops/s 691.9977 Ops/s $\color{#35bf28}+1.01\%$
test_ddpg_speed[True-backward] 2.4200ms 2.3287ms 429.4226 Ops/s 419.0769 Ops/s $\color{#35bf28}+2.47\%$
test_ddpg_speed[reduce-overhead-None] 2.4026ms 1.4572ms 686.2386 Ops/s 691.3885 Ops/s $\color{#d91a1a}-0.74\%$
test_ddpg_speed[reduce-overhead-backward] 2.4641ms 2.3489ms 425.7383 Ops/s 426.6655 Ops/s $\color{#d91a1a}-0.22\%$
test_sac_speed[False-None] 8.7161ms 8.2065ms 121.8540 Ops/s 121.0642 Ops/s $\color{#35bf28}+0.65\%$
test_sac_speed[False-backward] 11.5086ms 10.9269ms 91.5173 Ops/s 89.7256 Ops/s $\color{#35bf28}+2.00\%$
test_sac_speed[True-None] 3.6484ms 2.6530ms 376.9381 Ops/s 382.7979 Ops/s $\color{#d91a1a}-1.53\%$
test_sac_speed[True-backward] 4.3489ms 4.2511ms 235.2307 Ops/s 233.1022 Ops/s $\color{#35bf28}+0.91\%$
test_sac_speed[reduce-overhead-None] 3.2555ms 2.5813ms 387.4008 Ops/s 381.9487 Ops/s $\color{#35bf28}+1.43\%$
test_sac_speed[reduce-overhead-backward] 4.3949ms 4.2647ms 234.4815 Ops/s 233.7027 Ops/s $\color{#35bf28}+0.33\%$
test_redq_speed[False-None] 14.5432ms 12.9104ms 77.4570 Ops/s 74.9441 Ops/s $\color{#35bf28}+3.35\%$
test_redq_speed[False-backward] 29.2731ms 22.9942ms 43.4893 Ops/s 44.3262 Ops/s $\color{#d91a1a}-1.89\%$
test_redq_speed[True-None] 7.6277ms 6.8760ms 145.4336 Ops/s 143.3061 Ops/s $\color{#35bf28}+1.48\%$
test_redq_speed[True-backward] 15.4328ms 14.5434ms 68.7596 Ops/s 68.9932 Ops/s $\color{#d91a1a}-0.34\%$
test_redq_speed[reduce-overhead-None] 7.5818ms 6.9336ms 144.2242 Ops/s 144.8787 Ops/s $\color{#d91a1a}-0.45\%$
test_redq_speed[reduce-overhead-backward] 19.5755ms 15.2724ms 65.4778 Ops/s 66.9188 Ops/s $\color{#d91a1a}-2.15\%$
test_redq_deprec_speed[False-None] 15.5301ms 13.4927ms 74.1140 Ops/s 75.6982 Ops/s $\color{#d91a1a}-2.09\%$
test_redq_deprec_speed[False-backward] 20.3352ms 19.4771ms 51.3423 Ops/s 52.9059 Ops/s $\color{#d91a1a}-2.96\%$
test_redq_deprec_speed[True-None] 5.6918ms 5.2312ms 191.1625 Ops/s 192.1559 Ops/s $\color{#d91a1a}-0.52\%$
test_redq_deprec_speed[True-backward] 11.2995ms 10.1440ms 98.5806 Ops/s 95.7856 Ops/s $\color{#35bf28}+2.92\%$
test_redq_deprec_speed[reduce-overhead-None] 6.0488ms 5.2411ms 190.7998 Ops/s 191.6292 Ops/s $\color{#d91a1a}-0.43\%$
test_redq_deprec_speed[reduce-overhead-backward] 11.2067ms 10.3260ms 96.8431 Ops/s 96.8999 Ops/s $\color{#d91a1a}-0.06\%$
test_td3_speed[False-None] 8.3960ms 8.1494ms 122.7081 Ops/s 123.5920 Ops/s $\color{#d91a1a}-0.72\%$
test_td3_speed[False-backward] 13.6711ms 10.9639ms 91.2082 Ops/s 92.4292 Ops/s $\color{#d91a1a}-1.32\%$
test_td3_speed[True-None] 2.6168ms 2.2907ms 436.5472 Ops/s 435.8301 Ops/s $\color{#35bf28}+0.16\%$
test_td3_speed[True-backward] 4.4655ms 3.9836ms 251.0273 Ops/s 253.5210 Ops/s $\color{#d91a1a}-0.98\%$
test_td3_speed[reduce-overhead-None] 2.4303ms 2.2821ms 438.1983 Ops/s 434.8487 Ops/s $\color{#35bf28}+0.77\%$
test_td3_speed[reduce-overhead-backward] 4.2130ms 3.9496ms 253.1903 Ops/s 251.6124 Ops/s $\color{#35bf28}+0.63\%$
test_cql_speed[False-None] 38.8221ms 37.0041ms 27.0240 Ops/s 27.2713 Ops/s $\color{#d91a1a}-0.91\%$
test_cql_speed[False-backward] 49.8158ms 47.0266ms 21.2646 Ops/s 21.3160 Ops/s $\color{#d91a1a}-0.24\%$
test_cql_speed[True-None] 23.1128ms 22.4332ms 44.5767 Ops/s 44.4068 Ops/s $\color{#35bf28}+0.38\%$
test_cql_speed[True-backward] 30.8738ms 29.7395ms 33.6253 Ops/s 34.1871 Ops/s $\color{#d91a1a}-1.64\%$
test_cql_speed[reduce-overhead-None] 23.8092ms 22.7394ms 43.9765 Ops/s 44.6754 Ops/s $\color{#d91a1a}-1.56\%$
test_cql_speed[reduce-overhead-backward] 31.2135ms 29.6894ms 33.6821 Ops/s 34.4266 Ops/s $\color{#d91a1a}-2.16\%$
test_a2c_speed[False-None] 7.8785ms 7.2794ms 137.3737 Ops/s 137.7889 Ops/s $\color{#d91a1a}-0.30\%$
test_a2c_speed[False-backward] 16.5447ms 14.4851ms 69.0366 Ops/s 69.8781 Ops/s $\color{#d91a1a}-1.20\%$
test_a2c_speed[True-None] 5.2815ms 4.7364ms 211.1330 Ops/s 214.9219 Ops/s $\color{#d91a1a}-1.76\%$
test_a2c_speed[True-backward] 11.7111ms 11.2567ms 88.8361 Ops/s 89.4871 Ops/s $\color{#d91a1a}-0.73\%$
test_a2c_speed[reduce-overhead-None] 5.5142ms 4.7148ms 212.0975 Ops/s 214.8557 Ops/s $\color{#d91a1a}-1.28\%$
test_a2c_speed[reduce-overhead-backward] 11.5343ms 11.1765ms 89.4736 Ops/s 89.0629 Ops/s $\color{#35bf28}+0.46\%$
test_ppo_speed[False-None] 9.2062ms 7.6189ms 131.2517 Ops/s 130.4877 Ops/s $\color{#35bf28}+0.59\%$
test_ppo_speed[False-backward] 17.6849ms 14.9100ms 67.0691 Ops/s 67.1122 Ops/s $\color{#d91a1a}-0.06\%$
test_ppo_speed[True-None] 5.6505ms 5.0547ms 197.8339 Ops/s 198.2911 Ops/s $\color{#d91a1a}-0.23\%$
test_ppo_speed[True-backward] 11.4849ms 11.0580ms 90.4323 Ops/s 90.8394 Ops/s $\color{#d91a1a}-0.45\%$
test_ppo_speed[reduce-overhead-None] 6.3812ms 5.0990ms 196.1171 Ops/s 198.5624 Ops/s $\color{#d91a1a}-1.23\%$
test_ppo_speed[reduce-overhead-backward] 13.0844ms 11.4022ms 87.7026 Ops/s 90.7455 Ops/s $\color{#d91a1a}-3.35\%$
test_reinforce_speed[False-None] 9.1394ms 6.6998ms 149.2578 Ops/s 151.9087 Ops/s $\color{#d91a1a}-1.75\%$
test_reinforce_speed[False-backward] 11.1777ms 9.9135ms 100.8721 Ops/s 101.4897 Ops/s $\color{#d91a1a}-0.61\%$
test_reinforce_speed[True-None] 4.7911ms 4.0536ms 246.6932 Ops/s 244.0416 Ops/s $\color{#35bf28}+1.09\%$
test_reinforce_speed[True-backward] 10.6850ms 10.0810ms 99.1961 Ops/s 98.9257 Ops/s $\color{#35bf28}+0.27\%$
test_reinforce_speed[reduce-overhead-None] 4.5353ms 4.0640ms 246.0647 Ops/s 246.6072 Ops/s $\color{#d91a1a}-0.22\%$
test_reinforce_speed[reduce-overhead-backward] 10.6985ms 10.1213ms 98.8014 Ops/s 98.7158 Ops/s $\color{#35bf28}+0.09\%$
test_iql_speed[False-None] 40.2583ms 32.9465ms 30.3522 Ops/s 30.6943 Ops/s $\color{#d91a1a}-1.11\%$
test_iql_speed[False-backward] 54.1638ms 46.3185ms 21.5896 Ops/s 21.9954 Ops/s $\color{#d91a1a}-1.84\%$
test_iql_speed[True-None] 17.3990ms 15.8677ms 63.0209 Ops/s 62.1694 Ops/s $\color{#35bf28}+1.37\%$
test_iql_speed[True-backward] 28.4234ms 27.4474ms 36.4334 Ops/s 36.4303 Ops/s $+0.01\%$
test_iql_speed[reduce-overhead-None] 16.4892ms 15.9055ms 62.8712 Ops/s 62.1394 Ops/s $\color{#35bf28}+1.18\%$
test_iql_speed[reduce-overhead-backward] 28.2978ms 27.2320ms 36.7215 Ops/s 36.7411 Ops/s $\color{#d91a1a}-0.05\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5176ms 4.9263ms 202.9914 Ops/s 205.5919 Ops/s $\color{#d91a1a}-1.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0894ms 0.5406ms 1.8498 KOps/s 1.8318 KOps/s $\color{#35bf28}+0.98\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7963ms 0.5120ms 1.9533 KOps/s 1.8998 KOps/s $\color{#35bf28}+2.81\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.4258ms 4.6857ms 213.4152 Ops/s 212.7015 Ops/s $\color{#35bf28}+0.34\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.1088ms 0.5277ms 1.8949 KOps/s 505.1715 Ops/s $\textbf{\color{#35bf28}+275.09\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9334ms 0.5007ms 1.9972 KOps/s 1.9396 KOps/s $\color{#35bf28}+2.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.5568ms 1.7147ms 583.2055 Ops/s 580.5042 Ops/s $\color{#35bf28}+0.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3001ms 1.6212ms 616.8399 Ops/s 612.0394 Ops/s $\color{#35bf28}+0.78\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.4370ms 4.8638ms 205.6000 Ops/s 206.2208 Ops/s $\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.0352ms 0.6751ms 1.4812 KOps/s 1.4778 KOps/s $\color{#35bf28}+0.23\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9970ms 0.6488ms 1.5413 KOps/s 1.5397 KOps/s $\color{#35bf28}+0.10\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.4213ms 4.6833ms 213.5242 Ops/s 210.4184 Ops/s $\color{#35bf28}+1.48\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.6978ms 0.5433ms 1.8406 KOps/s 1.8295 KOps/s $\color{#35bf28}+0.61\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7326ms 0.5110ms 1.9569 KOps/s 1.9310 KOps/s $\color{#35bf28}+1.34\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.4233ms 4.7464ms 210.6863 Ops/s 210.9053 Ops/s $\color{#d91a1a}-0.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9178ms 0.5301ms 1.8866 KOps/s 1.8672 KOps/s $\color{#35bf28}+1.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7720ms 0.4992ms 2.0030 KOps/s 1.9564 KOps/s $\color{#35bf28}+2.38\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.3164ms 4.7295ms 211.4406 Ops/s 207.9860 Ops/s $\color{#35bf28}+1.66\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0669ms 0.6733ms 1.4852 KOps/s 1.4830 KOps/s $\color{#35bf28}+0.15\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0408ms 0.6545ms 1.5280 KOps/s 1.5096 KOps/s $\color{#35bf28}+1.22\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 5.5221ms 4.2792ms 233.6903 Ops/s 230.3330 Ops/s $\color{#35bf28}+1.46\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 8.2320ms 2.3823ms 419.7620 Ops/s 416.3933 Ops/s $\color{#35bf28}+0.81\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.7461ms 1.4022ms 713.1654 Ops/s 742.0771 Ops/s $\color{#d91a1a}-3.90\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 9.7116ms 4.4297ms 225.7489 Ops/s 233.6566 Ops/s $\color{#d91a1a}-3.38\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 4.8153ms 2.3419ms 427.0125 Ops/s 386.4740 Ops/s $\textbf{\color{#35bf28}+10.49\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 8.3601ms 1.4724ms 679.1545 Ops/s 764.9250 Ops/s $\textbf{\color{#d91a1a}-11.21\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.7631s 19.5922ms 51.0408 Ops/s 234.0781 Ops/s $\textbf{\color{#d91a1a}-78.19\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 8.2832ms 2.5040ms 399.3687 Ops/s 403.7429 Ops/s $\color{#d91a1a}-1.08\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.9557ms 1.3774ms 725.9849 Ops/s 648.6671 Ops/s $\textbf{\color{#35bf28}+11.92\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 59.8308ms 48.7055ms 20.5316 Ops/s 19.6180 Ops/s $\color{#35bf28}+4.66\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 15.5803ms 14.4535ms 69.1873 Ops/s 68.5157 Ops/s $\color{#35bf28}+0.98\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 59.5166ms 48.9661ms 20.4223 Ops/s 19.5578 Ops/s $\color{#35bf28}+4.42\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 16.6324ms 14.6421ms 68.2963 Ops/s 67.0970 Ops/s $\color{#35bf28}+1.79\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 60.2171ms 50.1384ms 19.9448 Ops/s 19.9604 Ops/s $\color{#d91a1a}-0.08\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 17.5182ms 15.8885ms 62.9384 Ops/s 62.5574 Ops/s $\color{#35bf28}+0.61\%$

vmoens added 3 commits March 17, 2025 10:37
@vmoens
Update
185d953 
[ghstack-poisoned]
@vmoens
Update
0c9a0c0 
[ghstack-poisoned]
@vmoens
Update
f8872db 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Mar 17, 2025
@vmoens
[BugFix] Right log-prob size in transformer wrapper
867d870 
ghstack-source-id: e7c6f04f19cb5b78191478fe8fcbacf2130efb62
Pull Request resolved: #2854
@vmoens vmoens added the bug Something isn't working label Mar 17, 2025
@vmoens vmoens mentioned this pull request Mar 17, 2025
Merged
@vmoens
Update
7a4051e 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Mar 17, 2025
Merged
tianyu1997 pushed a commit to tianyu1997/RL that referenced this pull request Mar 18, 2025
@vmoens
[BugFix] Right log-prob size in transformer wrapper
95d4b04 
ghstack-source-id: 5226bb4d25bbaaf139b24cf96d096f1d732013d3
Pull Request resolved: pytorch/rl#2854
This was referenced Mar 18, 2025
Merged
Merged
vmoens added a commit that referenced this pull request Mar 20, 2025
@vmoens
[BugFix] Right log-prob size in transformer wrapper
4c4e4f9 
ghstack-source-id: fd11bc55e61c1e3b40ce6702c075da885f6dca27
Pull Request resolved: #2854
@vmoens
Update
4f99627 
[ghstack-poisoned]
@vmoens vmoens merged commit 4f99627 into gh/vmoens/117/base Mar 20, 2025
24 of 44 checks passed
vmoens added a commit that referenced this pull request Mar 20, 2025
@vmoens
[BugFix] Right log-prob size in transformer wrapper
f81deac 
ghstack-source-id: 98baa635ca07d5bf7e69a9e3bc43012ae2d91bf0
Pull Request resolved: #2854
@vmoens vmoens deleted the gh/vmoens/117/head branch March 20, 2025 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot