rl
rl copied to clipboard
Published
20 hours ago •
pytorch
pytorch
[BugFix] Make sure keys are exclusive in envs
:link: Helpful Links
:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1912
- :page_facing_up: Preview Python docs built from this PR
Note: Links to docs will display an error until the docs builds have been completed.
:white_check_mark: You can merge normally! (3 Unrelated Failures)
As of commit 6848746e296ad80e779fd6ea95be8e900bd25062 with merge base aebc6a2981c18c4f1f1825564ebe423b47bd2c09 (
FLAKY - The following job failed but was likely due to flakiness present on trunk:
-
Unit-tests on Windows / unittests-cpu / windows-job (gh)
##[error]The operation was canceled.
BROKEN TRUNK - The following jobs failed but was present on the merge base:
👉 Rebase onto the `viable/strict` branch to avoid these failures
-
Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh)
##[error]The operation was canceled. -
Unit-tests on Windows / unittests-gpu / windows-job (gh)
##[error]The operation was canceled.
This comment was automatically generated by Dr. CI and updates every 15 minutes.
![pytorch-bot[bot] avatar](https://avatars.githubusercontent.com/in/40112?v=4 "Published by a pub.dev verified publisher"){kind=small}
$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests
Total Benchmarks: 89. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}8$.
Expand to view detailed results
| Name | Max | Mean | Ops | Ops on Repo HEAD |
Change |
|---|---|---|---|---|---|
| test_single | 0.1165s | 65.6997ms | 15.2208 Ops/s | 16.2664 Ops/s | $\textbf{\color{#d91a1a}-6.43\%}$ |
| test_sync | 46.9739ms | 38.4868ms | 25.9829 Ops/s | 29.8788 Ops/s | $\textbf{\color{#d91a1a}-13.04\%}$ |
| test_async | 68.4339ms | 31.6644ms | 31.5812 Ops/s | 30.9924 Ops/s | $\color{#35bf28}+1.90\%$ |
| test_simple | 0.4305s | 0.4270s | 2.3420 Ops/s | 2.2608 Ops/s | $\color{#35bf28}+3.59\%$ |
| test_transformed | 0.6400s | 0.5948s | 1.6812 Ops/s | 1.7128 Ops/s | $\color{#d91a1a}-1.85\%$ |
| test_serial | 1.4950s | 1.4442s | 0.6924 Ops/s | 0.6858 Ops/s | $\color{#35bf28}+0.97\%$ |
| test_parallel | 1.4509s | 1.3977s | 0.7155 Ops/s | 0.7018 Ops/s | $\color{#35bf28}+1.95\%$ |
| test_step_mdp_speed[True-True-True-True-True] | 0.1441ms | 21.0126μs | 47.5905 KOps/s | 47.0664 KOps/s | $\color{#35bf28}+1.11\%$ |
| test_step_mdp_speed[True-True-True-True-False] | 65.3220μs | 13.0348μs | 76.7178 KOps/s | 78.6228 KOps/s | $\color{#d91a1a}-2.42\%$ |
| test_step_mdp_speed[True-True-True-False-True] | 33.2920μs | 12.3282μs | 81.1148 KOps/s | 80.5780 KOps/s | $\color{#35bf28}+0.67\%$ |
| test_step_mdp_speed[True-True-True-False-False] | 27.1320μs | 7.5768μs | 131.9811 KOps/s | 134.4723 KOps/s | $\color{#d91a1a}-1.85\%$ |
| test_step_mdp_speed[True-True-False-True-True] | 56.4590μs | 22.3380μs | 44.7668 KOps/s | 44.3837 KOps/s | $\color{#35bf28}+0.86\%$ |
| test_step_mdp_speed[True-True-False-True-False] | 64.9470μs | 14.2326μs | 70.2614 KOps/s | 70.8490 KOps/s | $\color{#d91a1a}-0.83\%$ |
| test_step_mdp_speed[True-True-False-False-True] | 36.7610μs | 13.4179μs | 74.5273 KOps/s | 72.9250 KOps/s | $\color{#35bf28}+2.20\%$ |
| test_step_mdp_speed[True-True-False-False-False] | 36.2600μs | 8.8195μs | 113.3856 KOps/s | 114.4306 KOps/s | $\color{#d91a1a}-0.91\%$ |
| test_step_mdp_speed[True-False-True-True-True] | 57.7320μs | 23.7742μs | 42.0624 KOps/s | 41.8060 KOps/s | $\color{#35bf28}+0.61\%$ |
| test_step_mdp_speed[True-False-True-True-False] | 38.1330μs | 15.6852μs | 63.7544 KOps/s | 64.4015 KOps/s | $\color{#d91a1a}-1.00\%$ |
| test_step_mdp_speed[True-False-True-False-True] | 47.6030μs | 13.4446μs | 74.3790 KOps/s | 72.8829 KOps/s | $\color{#35bf28}+2.05\%$ |
| test_step_mdp_speed[True-False-True-False-False] | 52.8110μs | 8.6942μs | 115.0186 KOps/s | 114.5687 KOps/s | $\color{#35bf28}+0.39\%$ |
| test_step_mdp_speed[True-False-False-True-True] | 57.7480μs | 24.7151μs | 40.4611 KOps/s | 39.8302 KOps/s | $\color{#35bf28}+1.58\%$ |
| test_step_mdp_speed[True-False-False-True-False] | 46.3160μs | 16.7576μs | 59.6745 KOps/s | 60.3161 KOps/s | $\color{#d91a1a}-1.06\%$ |
| test_step_mdp_speed[True-False-False-False-True] | 38.7830μs | 14.6571μs | 68.2263 KOps/s | 67.5936 KOps/s | $\color{#35bf28}+0.94\%$ |
| test_step_mdp_speed[True-False-False-False-False] | 36.9580μs | 9.9959μs | 100.0406 KOps/s | 101.4655 KOps/s | $\color{#d91a1a}-1.40\%$ |
| test_step_mdp_speed[False-True-True-True-True] | 59.6420μs | 23.5651μs | 42.4357 KOps/s | 41.4637 KOps/s | $\color{#35bf28}+2.34\%$ |
| test_step_mdp_speed[False-True-True-True-False] | 40.9860μs | 15.6216μs | 64.0140 KOps/s | 65.0245 KOps/s | $\color{#d91a1a}-1.55\%$ |
| test_step_mdp_speed[False-True-True-False-True] | 42.6700μs | 15.6338μs | 63.9641 KOps/s | 62.2534 KOps/s | $\color{#35bf28}+2.75\%$ |
| test_step_mdp_speed[False-True-True-False-False] | 31.4920μs | 9.9078μs | 100.9302 KOps/s | 101.1841 KOps/s | $\color{#d91a1a}-0.25\%$ |
| test_step_mdp_speed[False-True-False-True-True] | 48.9430μs | 25.2616μs | 39.5858 KOps/s | 38.8883 KOps/s | $\color{#35bf28}+1.79\%$ |
| test_step_mdp_speed[False-True-False-True-False] | 40.4760μs | 16.8280μs | 59.4248 KOps/s | 60.0518 KOps/s | $\color{#d91a1a}-1.04\%$ |
| test_step_mdp_speed[False-True-False-False-True] | 39.3140μs | 16.9587μs | 58.9667 KOps/s | 57.6789 KOps/s | $\color{#35bf28}+2.23\%$ |
| test_step_mdp_speed[False-True-False-False-False] | 35.1960μs | 11.1203μs | 89.9257 KOps/s | 90.6459 KOps/s | $\color{#d91a1a}-0.79\%$ |
| test_step_mdp_speed[False-False-True-True-True] | 64.9020μs | 26.1048μs | 38.3071 KOps/s | 37.5347 KOps/s | $\color{#35bf28}+2.06\%$ |
| test_step_mdp_speed[False-False-True-True-False] | 45.2260μs | 18.0228μs | 55.4853 KOps/s | 56.1505 KOps/s | $\color{#d91a1a}-1.18\%$ |
| test_step_mdp_speed[False-False-True-False-True] | 39.8350μs | 17.0289μs | 58.7237 KOps/s | 58.3334 KOps/s | $\color{#35bf28}+0.67\%$ |
| test_step_mdp_speed[False-False-True-False-False] | 33.5930μs | 11.2287μs | 89.0575 KOps/s | 89.9389 KOps/s | $\color{#d91a1a}-0.98\%$ |
| test_step_mdp_speed[False-False-False-True-True] | 55.0540μs | 27.1334μs | 36.8550 KOps/s | 36.4013 KOps/s | $\color{#35bf28}+1.25\%$ |
| test_step_mdp_speed[False-False-False-True-False] | 43.8230μs | 19.1531μs | 52.2108 KOps/s | 53.3120 KOps/s | $\color{#d91a1a}-2.07\%$ |
| test_step_mdp_speed[False-False-False-False-True] | 45.8470μs | 18.0293μs | 55.4653 KOps/s | 55.3723 KOps/s | $\color{#35bf28}+0.17\%$ |
| test_step_mdp_speed[False-False-False-False-False] | 32.7810μs | 12.2824μs | 81.4171 KOps/s | 82.7467 KOps/s | $\color{#d91a1a}-1.61\%$ |
| test_values[generalized_advantage_estimate-True-True] | 10.0375ms | 9.3159ms | 107.3433 Ops/s | 109.5767 Ops/s | $\color{#d91a1a}-2.04\%$ |
| test_values[vec_generalized_advantage_estimate-True-True] | 40.0113ms | 35.5001ms | 28.1689 Ops/s | 29.8896 Ops/s | $\textbf{\color{#d91a1a}-5.76\%}$ |
| test_values[td0_return_estimate-False-False] | 0.2616ms | 0.1688ms | 5.9238 KOps/s | 5.9900 KOps/s | $\color{#d91a1a}-1.11\%$ |
| test_values[td1_return_estimate-False-False] | 25.8573ms | 23.3084ms | 42.9030 Ops/s | 43.8917 Ops/s | $\color{#d91a1a}-2.25\%$ |
| test_values[vec_td1_return_estimate-False-False] | 39.2824ms | 35.5141ms | 28.1578 Ops/s | 29.7511 Ops/s | $\textbf{\color{#d91a1a}-5.36\%}$ |
| test_values[td_lambda_return_estimate-True-False] | 36.5446ms | 33.7961ms | 29.5892 Ops/s | 30.1021 Ops/s | $\color{#d91a1a}-1.70\%$ |
| test_values[vec_td_lambda_return_estimate-True-False] | 42.1465ms | 35.8911ms | 27.8620 Ops/s | 29.7248 Ops/s | $\textbf{\color{#d91a1a}-6.27\%}$ |
| test_gae_speed[generalized_advantage_estimate-False-1-512] | 10.2684ms | 8.1558ms | 122.6128 Ops/s | 125.9801 Ops/s | $\color{#d91a1a}-2.67\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.2576ms | 1.7885ms | 559.1159 Ops/s | 492.4398 Ops/s | $\textbf{\color{#35bf28}+13.54\%}$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4293ms | 0.3504ms | 2.8539 KOps/s | 2.8961 KOps/s | $\color{#d91a1a}-1.46\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 47.4929ms | 46.3622ms | 21.5693 Ops/s | 23.2424 Ops/s | $\textbf{\color{#d91a1a}-7.20\%}$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 4.5472ms | 3.0366ms | 329.3104 Ops/s | 330.9004 Ops/s | $\color{#d91a1a}-0.48\%$ |
| test_dqn_speed | 70.0745ms | 1.4863ms | 672.8061 Ops/s | 721.2645 Ops/s | $\textbf{\color{#d91a1a}-6.72\%}$ |
| test_ddpg_speed | 3.4095ms | 2.7267ms | 366.7381 Ops/s | 370.0429 Ops/s | $\color{#d91a1a}-0.89\%$ |
| test_sac_speed | 9.9664ms | 8.3284ms | 120.0708 Ops/s | 121.4954 Ops/s | $\color{#d91a1a}-1.17\%$ |
| test_redq_speed | 14.2568ms | 13.1444ms | 76.0778 Ops/s | 76.0481 Ops/s | $\color{#35bf28}+0.04\%$ |
| test_redq_deprec_speed | 14.3908ms | 13.3914ms | 74.6746 Ops/s | 75.7453 Ops/s | $\color{#d91a1a}-1.41\%$ |
| test_td3_speed | 8.9125ms | 8.2597ms | 121.0703 Ops/s | 122.8072 Ops/s | $\color{#d91a1a}-1.41\%$ |
| test_cql_speed | 38.1084ms | 36.4245ms | 27.4541 Ops/s | 27.5882 Ops/s | $\color{#d91a1a}-0.49\%$ |
| test_a2c_speed | 8.6813ms | 7.4245ms | 134.6889 Ops/s | 136.0687 Ops/s | $\color{#d91a1a}-1.01\%$ |
| test_ppo_speed | 8.3361ms | 7.6446ms | 130.8113 Ops/s | 131.4882 Ops/s | $\color{#d91a1a}-0.51\%$ |
| test_reinforce_speed | 7.6084ms | 6.5660ms | 152.3007 Ops/s | 154.0223 Ops/s | $\color{#d91a1a}-1.12\%$ |
| test_iql_speed | 33.2768ms | 32.4750ms | 30.7929 Ops/s | 30.7012 Ops/s | $\color{#35bf28}+0.30\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.3415ms | 2.2816ms | 438.2799 Ops/s | 434.3861 Ops/s | $\color{#35bf28}+0.90\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8310ms | 0.4955ms | 2.0181 KOps/s | 2.0156 KOps/s | $\color{#35bf28}+0.12\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7924ms | 0.4695ms | 2.1298 KOps/s | 2.1308 KOps/s | $\color{#d91a1a}-0.05\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.5095ms | 2.3066ms | 433.5383 Ops/s | 436.3972 Ops/s | $\color{#d91a1a}-0.66\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.9788ms | 0.4906ms | 2.0383 KOps/s | 2.0355 KOps/s | $\color{#35bf28}+0.14\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7802ms | 0.4637ms | 2.1566 KOps/s | 2.1610 KOps/s | $\color{#d91a1a}-0.21\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 2.6942ms | 2.3662ms | 422.6232 Ops/s | 401.3040 Ops/s | $\textbf{\color{#35bf28}+5.31\%}$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.0992ms | 0.6124ms | 1.6328 KOps/s | 1.6475 KOps/s | $\color{#d91a1a}-0.89\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8397ms | 0.5857ms | 1.7074 KOps/s | 1.7087 KOps/s | $\color{#d91a1a}-0.07\%$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.3422ms | 2.1995ms | 454.6488 Ops/s | 448.6572 Ops/s | $\color{#35bf28}+1.34\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 95.6312ms | 0.5841ms | 1.7120 KOps/s | 2.0267 KOps/s | $\textbf{\color{#d91a1a}-15.53\%}$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.5757ms | 0.4724ms | 2.1170 KOps/s | 2.1152 KOps/s | $\color{#35bf28}+0.08\%$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.5304ms | 2.3925ms | 417.9778 Ops/s | 436.4131 Ops/s | $\color{#d91a1a}-4.22\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.9926ms | 0.4922ms | 2.0319 KOps/s | 2.0748 KOps/s | $\color{#d91a1a}-2.07\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7356ms | 0.4629ms | 2.1601 KOps/s | 2.1198 KOps/s | $\color{#35bf28}+1.90\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.4519ms | 2.3461ms | 426.2390 Ops/s | 415.1748 Ops/s | $\color{#35bf28}+2.66\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8793ms | 0.6163ms | 1.6227 KOps/s | 1.6344 KOps/s | $\color{#d91a1a}-0.72\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 3.6310ms | 0.5912ms | 1.6915 KOps/s | 1.7079 KOps/s | $\color{#d91a1a}-0.96\%$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1088s | 5.7170ms | 174.9169 Ops/s | 139.5698 Ops/s | $\textbf{\color{#35bf28}+25.33\%}$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 14.0620ms | 11.9949ms | 83.3690 Ops/s | 83.5697 Ops/s | $\color{#d91a1a}-0.24\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.5668ms | 1.0423ms | 959.4460 Ops/s | 985.0049 Ops/s | $\color{#d91a1a}-2.59\%$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 93.8191ms | 7.1894ms | 139.0935 Ops/s | 136.4282 Ops/s | $\color{#35bf28}+1.95\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 14.3516ms | 11.9022ms | 84.0184 Ops/s | 83.7037 Ops/s | $\color{#35bf28}+0.38\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.8636ms | 1.0566ms | 946.4525 Ops/s | 928.7216 Ops/s | $\color{#35bf28}+1.91\%$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 97.7424ms | 7.5772ms | 131.9752 Ops/s | 136.4097 Ops/s | $\color{#d91a1a}-3.25\%$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 14.7276ms | 12.2160ms | 81.8597 Ops/s | 81.6325 Ops/s | $\color{#35bf28}+0.28\%$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.0822ms | 1.3640ms | 733.1155 Ops/s | 696.6578 Ops/s | $\textbf{\color{#35bf28}+5.23\%}$ |
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests
Total Benchmarks: 92. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}2$.
Expand to view detailed results
| Name | Max | Mean | Ops | Ops on Repo HEAD |
Change |
|---|---|---|---|---|---|
| test_single | 0.1177s | 0.1161s | 8.6129 Ops/s | 8.8597 Ops/s | $\color{#d91a1a}-2.79\%$ |
| test_sync | 95.5061ms | 95.2619ms | 10.4974 Ops/s | 10.4594 Ops/s | $\color{#35bf28}+0.36\%$ |
| test_async | 0.1802s | 91.7175ms | 10.9030 Ops/s | 10.9353 Ops/s | $\color{#d91a1a}-0.30\%$ |
| test_single_pixels | 0.1959s | 0.1342s | 7.4522 Ops/s | 7.7539 Ops/s | $\color{#d91a1a}-3.89\%$ |
| test_sync_pixels | 82.3166ms | 78.8286ms | 12.6858 Ops/s | 12.4006 Ops/s | $\color{#35bf28}+2.30\%$ |
| test_async_pixels | 0.1499s | 72.5688ms | 13.7800 Ops/s | 13.4040 Ops/s | $\color{#35bf28}+2.81\%$ |
| test_simple | 0.9015s | 0.8427s | 1.1866 Ops/s | 1.2188 Ops/s | $\color{#d91a1a}-2.64\%$ |
| test_transformed | 1.1194s | 1.0696s | 0.9349 Ops/s | 0.9613 Ops/s | $\color{#d91a1a}-2.75\%$ |
| test_serial | 2.5632s | 2.4940s | 0.4010 Ops/s | 0.4181 Ops/s | $\color{#d91a1a}-4.09\%$ |
| test_parallel | 2.1736s | 2.1003s | 0.4761 Ops/s | 0.4871 Ops/s | $\color{#d91a1a}-2.26\%$ |
| test_step_mdp_speed[True-True-True-True-True] | 95.1310μs | 32.2331μs | 31.0240 KOps/s | 30.8101 KOps/s | $\color{#35bf28}+0.69\%$ |
| test_step_mdp_speed[True-True-True-True-False] | 44.0200μs | 19.3888μs | 51.5761 KOps/s | 51.5202 KOps/s | $\color{#35bf28}+0.11\%$ |
| test_step_mdp_speed[True-True-True-False-True] | 49.8700μs | 18.0835μs | 55.2991 KOps/s | 54.6669 KOps/s | $\color{#35bf28}+1.16\%$ |
| test_step_mdp_speed[True-True-True-False-False] | 80.3810μs | 10.8568μs | 92.1079 KOps/s | 91.5624 KOps/s | $\color{#35bf28}+0.60\%$ |
| test_step_mdp_speed[True-True-False-True-True] | 58.2000μs | 33.5323μs | 29.8220 KOps/s | 29.4318 KOps/s | $\color{#35bf28}+1.33\%$ |
| test_step_mdp_speed[True-True-False-True-False] | 41.5500μs | 21.0091μs | 47.5983 KOps/s | 47.7763 KOps/s | $\color{#d91a1a}-0.37\%$ |
| test_step_mdp_speed[True-True-False-False-True] | 40.0210μs | 19.9664μs | 50.0841 KOps/s | 49.2292 KOps/s | $\color{#35bf28}+1.74\%$ |
| test_step_mdp_speed[True-True-False-False-False] | 35.8900μs | 12.6721μs | 78.9137 KOps/s | 77.7978 KOps/s | $\color{#35bf28}+1.43\%$ |
| test_step_mdp_speed[True-False-True-True-True] | 67.4810μs | 35.6439μs | 28.0553 KOps/s | 27.9172 KOps/s | $\color{#35bf28}+0.49\%$ |
| test_step_mdp_speed[True-False-True-True-False] | 45.5510μs | 22.8718μs | 43.7219 KOps/s | 43.2880 KOps/s | $\color{#35bf28}+1.00\%$ |
| test_step_mdp_speed[True-False-True-False-True] | 52.9800μs | 19.9807μs | 50.0483 KOps/s | 50.0253 KOps/s | $\color{#35bf28}+0.05\%$ |
| test_step_mdp_speed[True-False-True-False-False] | 44.2400μs | 12.7674μs | 78.3242 KOps/s | 77.6664 KOps/s | $\color{#35bf28}+0.85\%$ |
| test_step_mdp_speed[True-False-False-True-True] | 64.7710μs | 37.2077μs | 26.8761 KOps/s | 26.4923 KOps/s | $\color{#35bf28}+1.45\%$ |
| test_step_mdp_speed[True-False-False-True-False] | 44.0310μs | 24.5076μs | 40.8036 KOps/s | 40.5280 KOps/s | $\color{#35bf28}+0.68\%$ |
| test_step_mdp_speed[True-False-False-False-True] | 44.9300μs | 21.6050μs | 46.2855 KOps/s | 46.0009 KOps/s | $\color{#35bf28}+0.62\%$ |
| test_step_mdp_speed[True-False-False-False-False] | 34.4000μs | 14.4783μs | 69.0689 KOps/s | 68.0562 KOps/s | $\color{#35bf28}+1.49\%$ |
| test_step_mdp_speed[False-True-True-True-True] | 71.2200μs | 35.8648μs | 27.8825 KOps/s | 27.9714 KOps/s | $\color{#d91a1a}-0.32\%$ |
| test_step_mdp_speed[False-True-True-True-False] | 47.1710μs | 22.7484μs | 43.9592 KOps/s | 43.3692 KOps/s | $\color{#35bf28}+1.36\%$ |
| test_step_mdp_speed[False-True-True-False-True] | 57.4510μs | 23.6732μs | 42.2418 KOps/s | 42.8236 KOps/s | $\color{#d91a1a}-1.36\%$ |
| test_step_mdp_speed[False-True-True-False-False] | 49.9110μs | 14.4863μs | 69.0307 KOps/s | 69.1330 KOps/s | $\color{#d91a1a}-0.15\%$ |
| test_step_mdp_speed[False-True-False-True-True] | 63.8610μs | 37.4865μs | 26.6763 KOps/s | 26.3790 KOps/s | $\color{#35bf28}+1.13\%$ |
| test_step_mdp_speed[False-True-False-True-False] | 46.1400μs | 24.8325μs | 40.2698 KOps/s | 39.7758 KOps/s | $\color{#35bf28}+1.24\%$ |
| test_step_mdp_speed[False-True-False-False-True] | 51.6710μs | 25.6144μs | 39.0405 KOps/s | 39.7080 KOps/s | $\color{#d91a1a}-1.68\%$ |
| test_step_mdp_speed[False-True-False-False-False] | 37.2910μs | 16.3978μs | 60.9840 KOps/s | 60.7359 KOps/s | $\color{#35bf28}+0.41\%$ |
| test_step_mdp_speed[False-False-True-True-True] | 89.3510μs | 39.5235μs | 25.3014 KOps/s | 24.9417 KOps/s | $\color{#35bf28}+1.44\%$ |
| test_step_mdp_speed[False-False-True-True-False] | 65.5700μs | 26.9024μs | 37.1714 KOps/s | 37.2033 KOps/s | $\color{#d91a1a}-0.09\%$ |
| test_step_mdp_speed[False-False-True-False-True] | 50.2200μs | 25.6005μs | 39.0617 KOps/s | 39.5106 KOps/s | $\color{#d91a1a}-1.14\%$ |
| test_step_mdp_speed[False-False-True-False-False] | 43.3110μs | 16.4076μs | 60.9474 KOps/s | 61.4249 KOps/s | $\color{#d91a1a}-0.78\%$ |
| test_step_mdp_speed[False-False-False-True-True] | 71.8710μs | 40.7047μs | 24.5672 KOps/s | 24.4329 KOps/s | $\color{#35bf28}+0.55\%$ |
| test_step_mdp_speed[False-False-False-True-False] | 51.5600μs | 28.5913μs | 34.9757 KOps/s | 35.1802 KOps/s | $\color{#d91a1a}-0.58\%$ |
| test_step_mdp_speed[False-False-False-False-True] | 50.4110μs | 26.9222μs | 37.1441 KOps/s | 37.3737 KOps/s | $\color{#d91a1a}-0.61\%$ |
| test_step_mdp_speed[False-False-False-False-False] | 53.1000μs | 18.1440μs | 55.1145 KOps/s | 54.7918 KOps/s | $\color{#35bf28}+0.59\%$ |
| test_values[generalized_advantage_estimate-True-True] | 25.8830ms | 25.2793ms | 39.5581 Ops/s | 41.5641 Ops/s | $\color{#d91a1a}-4.83\%$ |
| test_values[vec_generalized_advantage_estimate-True-True] | 89.3838ms | 3.3447ms | 298.9804 Ops/s | 309.8012 Ops/s | $\color{#d91a1a}-3.49\%$ |
| test_values[td0_return_estimate-False-False] | 98.1210μs | 58.2632μs | 17.1635 KOps/s | 16.1742 KOps/s | $\textbf{\color{#35bf28}+6.12\%}$ |
| test_values[td1_return_estimate-False-False] | 54.5305ms | 53.6129ms | 18.6522 Ops/s | 19.2062 Ops/s | $\color{#d91a1a}-2.88\%$ |
| test_values[vec_td1_return_estimate-False-False] | 2.0412ms | 1.7689ms | 565.3330 Ops/s | 570.1051 Ops/s | $\color{#d91a1a}-0.84\%$ |
| test_values[td_lambda_return_estimate-True-False] | 86.4221ms | 85.7150ms | 11.6666 Ops/s | 12.0327 Ops/s | $\color{#d91a1a}-3.04\%$ |
| test_values[vec_td_lambda_return_estimate-True-False] | 3.9704ms | 1.7791ms | 562.0939 Ops/s | 558.6561 Ops/s | $\color{#35bf28}+0.62\%$ |
| test_gae_speed[generalized_advantage_estimate-False-1-512] | 22.9231ms | 22.5326ms | 44.3801 Ops/s | 43.5071 Ops/s | $\color{#35bf28}+2.01\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.8657ms | 0.6727ms | 1.4865 KOps/s | 1.4346 KOps/s | $\color{#35bf28}+3.62\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.6916ms | 0.6317ms | 1.5832 KOps/s | 1.5580 KOps/s | $\color{#35bf28}+1.61\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.4991ms | 1.4342ms | 697.2509 Ops/s | 694.7089 Ops/s | $\color{#35bf28}+0.37\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.9174ms | 0.6517ms | 1.5344 KOps/s | 1.5056 KOps/s | $\color{#35bf28}+1.92\%$ |
| test_dqn_speed | 8.0984ms | 1.4157ms | 706.3845 Ops/s | 637.7852 Ops/s | $\textbf{\color{#35bf28}+10.76\%}$ |
| test_ddpg_speed | 3.0126ms | 2.6762ms | 373.6593 Ops/s | 378.2620 Ops/s | $\color{#d91a1a}-1.22\%$ |
| test_sac_speed | 8.3619ms | 7.8777ms | 126.9402 Ops/s | 128.4829 Ops/s | $\color{#d91a1a}-1.20\%$ |
| test_redq_speed | 11.3692ms | 10.1000ms | 99.0103 Ops/s | 100.7166 Ops/s | $\color{#d91a1a}-1.69\%$ |
| test_redq_deprec_speed | 11.1822ms | 10.7369ms | 93.1369 Ops/s | 93.5584 Ops/s | $\color{#d91a1a}-0.45\%$ |
| test_td3_speed | 15.4857ms | 7.8685ms | 127.0897 Ops/s | 128.2191 Ops/s | $\color{#d91a1a}-0.88\%$ |
| test_cql_speed | 26.3021ms | 24.7957ms | 40.3296 Ops/s | 40.1345 Ops/s | $\color{#35bf28}+0.49\%$ |
| test_a2c_speed | 5.6515ms | 5.4851ms | 182.3110 Ops/s | 186.9011 Ops/s | $\color{#d91a1a}-2.46\%$ |
| test_ppo_speed | 5.9751ms | 5.7607ms | 173.5910 Ops/s | 176.8494 Ops/s | $\color{#d91a1a}-1.84\%$ |
| test_reinforce_speed | 4.6749ms | 4.4694ms | 223.7421 Ops/s | 226.0058 Ops/s | $\color{#d91a1a}-1.00\%$ |
| test_iql_speed | 21.0787ms | 19.5298ms | 51.2037 Ops/s | 52.1729 Ops/s | $\color{#d91a1a}-1.86\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 2.9651ms | 2.8348ms | 352.7626 Ops/s | 353.8444 Ops/s | $\color{#d91a1a}-0.31\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 87.7743ms | 0.5982ms | 1.6716 KOps/s | 1.8729 KOps/s | $\textbf{\color{#d91a1a}-10.75\%}$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7138ms | 0.5097ms | 1.9620 KOps/s | 1.9674 KOps/s | $\color{#d91a1a}-0.27\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.2356ms | 2.8724ms | 348.1461 Ops/s | 348.3321 Ops/s | $\color{#d91a1a}-0.05\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6587ms | 0.5269ms | 1.8981 KOps/s | 1.8967 KOps/s | $\color{#35bf28}+0.07\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.6235ms | 0.5086ms | 1.9662 KOps/s | 1.9989 KOps/s | $\color{#d91a1a}-1.64\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.0669ms | 2.9750ms | 336.1329 Ops/s | 336.1550 Ops/s | $-0.01\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8309ms | 0.6586ms | 1.5184 KOps/s | 1.5442 KOps/s | $\color{#d91a1a}-1.67\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8678ms | 0.6331ms | 1.5794 KOps/s | 1.5974 KOps/s | $\color{#d91a1a}-1.13\%$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.0022ms | 2.8550ms | 350.2611 Ops/s | 349.8474 Ops/s | $\color{#35bf28}+0.12\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.6563ms | 0.5355ms | 1.8675 KOps/s | 1.8769 KOps/s | $\color{#d91a1a}-0.50\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.6667ms | 0.5170ms | 1.9343 KOps/s | 1.9420 KOps/s | $\color{#d91a1a}-0.40\%$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.0403ms | 2.8699ms | 348.4440 Ops/s | 348.9179 Ops/s | $\color{#d91a1a}-0.14\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6490ms | 0.5297ms | 1.8879 KOps/s | 1.9071 KOps/s | $\color{#d91a1a}-1.01\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.5948ms | 0.5099ms | 1.9612 KOps/s | 1.9836 KOps/s | $\color{#d91a1a}-1.13\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.1018ms | 2.9795ms | 335.6218 Ops/s | 334.5113 Ops/s | $\color{#35bf28}+0.33\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8244ms | 0.6528ms | 1.5319 KOps/s | 1.5411 KOps/s | $\color{#d91a1a}-0.59\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8372ms | 0.6316ms | 1.5832 KOps/s | 1.3903 KOps/s | $\textbf{\color{#35bf28}+13.87\%}$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1125s | 8.6599ms | 115.4746 Ops/s | 151.0408 Ops/s | $\textbf{\color{#d91a1a}-23.55\%}$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 24.0859ms | 14.6245ms | 68.3784 Ops/s | 70.6284 Ops/s | $\color{#d91a1a}-3.19\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.2468ms | 1.0771ms | 928.4096 Ops/s | 872.8319 Ops/s | $\textbf{\color{#35bf28}+6.37\%}$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1013s | 8.5173ms | 117.4077 Ops/s | 117.9165 Ops/s | $\color{#d91a1a}-0.43\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.4367ms | 14.6807ms | 68.1168 Ops/s | 70.9857 Ops/s | $\color{#d91a1a}-4.04\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.1266ms | 1.0418ms | 959.8707 Ops/s | 799.9040 Ops/s | $\textbf{\color{#35bf28}+20.00\%}$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 99.5601ms | 6.8764ms | 145.4257 Ops/s | 145.0052 Ops/s | $\color{#35bf28}+0.29\%$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.4921ms | 14.8862ms | 67.1761 Ops/s | 69.1346 Ops/s | $\color{#d91a1a}-2.83\%$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 1.4863ms | 1.3727ms | 728.4998 Ops/s | 633.1578 Ops/s | $\textbf{\color{#35bf28}+15.06\%}$ |