rl
rl copied to clipboard
pytorch
[Feature] (Un)Flatten data during RB (de)serialization
This PR introduces the concept of storage save hooks, which can modify a storage before serializing it
TODO:
- Account for more complex data structures (e.g. nested done, nested rewards)
- Identify where the last step from the last trajectory is when the buffer is at capacity
- nd trajectories
:link: Helpful Links
:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2137
- :page_facing_up: Preview Python docs built from this PR
Note: Links to docs will display an error until the docs builds have been completed.
:x: 6 New Failures, 5 Unrelated Failures
As of commit 2a184b3b9412fa3402d5332833b6150772fb1fc4 with merge base e77f0dd8a8c129ddd213c29a23637655081fc7d7 (
NEW FAILURES - The following jobs have failed:
- Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
RuntimeError: Command docker exec -t 3da9bfbd7822dcedcdc52cd5bc7332df8832b3829e4f893f1192085ddbdb03d0 /exec failed with exit code 1 - Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
RuntimeError: Command docker exec -t ee18f16d110ed8947925a985791b64a12bbb4b788890a0d7167c5e135a0481ff /exec failed with exit code 139 - Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job (gh)
test/test_transforms.py::TestVC1::test_vc1_spec_against_real[False-device0] - Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh)
RuntimeError: Command docker exec -t a6567131aeea3005a8a85c25e336e1d2ed0410225231b7adc4e88b235fbc52a5 /exec failed with exit code 1 - Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
test/test_transforms.py::TestVC1::test_vc1_spec_against_real[False-device0] - Unit-tests on Windows / unittests-cpu / windows-job (gh)
The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128
BROKEN TRUNK - The following jobs failed but were present on the merge base:
👉 Rebase onto the `viable/strict` branch to avoid these failures
- Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh) (trunk failure)
test/test_env.py::TestInfoDict::test_auto_register[device0] - Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh) (trunk failure)
test/test_env.py::TestInfoDict::test_auto_register[device0] - Unit-tests on Linux / tests-cpu (3.8) / linux-job (gh) (trunk failure)
test/test_env.py::TestInfoDict::test_auto_register[device0] - Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh) (trunk failure)
test/test_env.py::TestInfoDict::test_auto_register[device0] - Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh) (trunk failure)
test/test_transforms.py::TestVecNorm::test_state_dict_vecnorm
This comment was automatically generated by Dr. CI and updates every 15 minutes.
![pytorch-bot[bot] avatar](https://avatars.githubusercontent.com/in/40112?v=4 "Published by a pub.dev verified publisher"){kind=small}
$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests
Total Benchmarks: 91. Improved: $\large\color{#35bf28}10$. Worsened: $\large\color{#d91a1a}2$.
Expand to view detailed results
| Name | Max | Mean | Ops | Ops on Repo HEAD |
Change |
|---|---|---|---|---|---|
| test_single | 54.2698ms | 53.8804ms | 18.5596 Ops/s | 18.1807 Ops/s | $\color{#35bf28}+2.08\%$ |
| test_sync | 41.4596ms | 31.0590ms | 32.1967 Ops/s | 31.4228 Ops/s | $\color{#35bf28}+2.46\%$ |
| test_async | 55.0691ms | 27.3634ms | 36.5451 Ops/s | 34.7455 Ops/s | $\textbf{\color{#35bf28}+5.18\%}$ |
| test_simple | 0.4066s | 0.3474s | 2.8787 Ops/s | 2.9040 Ops/s | $\color{#d91a1a}-0.87\%$ |
| test_transformed | 0.5407s | 0.4884s | 2.0475 Ops/s | 1.9999 Ops/s | $\color{#35bf28}+2.38\%$ |
| test_serial | 1.2684s | 1.2023s | 0.8317 Ops/s | 0.7913 Ops/s | $\textbf{\color{#35bf28}+5.11\%}$ |
| test_parallel | 1.0590s | 1.0010s | 0.9990 Ops/s | 0.9717 Ops/s | $\color{#35bf28}+2.80\%$ |
| test_step_mdp_speed[True-True-True-True-True] | 0.1748ms | 21.5470μs | 46.4101 KOps/s | 43.5978 KOps/s | $\textbf{\color{#35bf28}+6.45\%}$ |
| test_step_mdp_speed[True-True-True-True-False] | 58.4810μs | 13.2770μs | 75.3183 KOps/s | 75.3939 KOps/s | $\color{#d91a1a}-0.10\%$ |
| test_step_mdp_speed[True-True-True-False-True] | 37.7300μs | 12.6323μs | 79.1622 KOps/s | 78.6764 KOps/s | $\color{#35bf28}+0.62\%$ |
| test_step_mdp_speed[True-True-True-False-False] | 60.4330μs | 7.6160μs | 131.3033 KOps/s | 127.3069 KOps/s | $\color{#35bf28}+3.14\%$ |
| test_step_mdp_speed[True-True-False-True-True] | 47.0570μs | 22.8899μs | 43.6875 KOps/s | 43.0264 KOps/s | $\color{#35bf28}+1.54\%$ |
| test_step_mdp_speed[True-True-False-True-False] | 61.7950μs | 14.4272μs | 69.3133 KOps/s | 69.1941 KOps/s | $\color{#35bf28}+0.17\%$ |
| test_step_mdp_speed[True-True-False-False-True] | 66.9950μs | 13.9568μs | 71.6499 KOps/s | 71.3765 KOps/s | $\color{#35bf28}+0.38\%$ |
| test_step_mdp_speed[True-True-False-False-False] | 66.3240μs | 8.9193μs | 112.1170 KOps/s | 111.2464 KOps/s | $\color{#35bf28}+0.78\%$ |
| test_step_mdp_speed[True-False-True-True-True] | 75.0200μs | 24.7935μs | 40.3331 KOps/s | 40.4512 KOps/s | $\color{#d91a1a}-0.29\%$ |
| test_step_mdp_speed[True-False-True-True-False] | 45.7850μs | 16.1646μs | 61.8635 KOps/s | 62.9713 KOps/s | $\color{#d91a1a}-1.76\%$ |
| test_step_mdp_speed[True-False-True-False-True] | 46.5560μs | 13.9880μs | 71.4899 KOps/s | 71.5144 KOps/s | $\color{#d91a1a}-0.03\%$ |
| test_step_mdp_speed[True-False-True-False-False] | 23.2540μs | 9.0827μs | 110.0993 KOps/s | 112.2896 KOps/s | $\color{#d91a1a}-1.95\%$ |
| test_step_mdp_speed[True-False-False-True-True] | 86.6020μs | 25.8567μs | 38.6748 KOps/s | 38.4073 KOps/s | $\color{#35bf28}+0.70\%$ |
| test_step_mdp_speed[True-False-False-True-False] | 39.8240μs | 17.3111μs | 57.7666 KOps/s | 58.4621 KOps/s | $\color{#d91a1a}-1.19\%$ |
| test_step_mdp_speed[True-False-False-False-True] | 61.4550μs | 15.2610μs | 65.5264 KOps/s | 66.0260 KOps/s | $\color{#d91a1a}-0.76\%$ |
| test_step_mdp_speed[True-False-False-False-False] | 43.5210μs | 10.2544μs | 97.5191 KOps/s | 98.7992 KOps/s | $\color{#d91a1a}-1.30\%$ |
| test_step_mdp_speed[False-True-True-True-True] | 74.4590μs | 24.6354μs | 40.5920 KOps/s | 39.9266 KOps/s | $\color{#35bf28}+1.67\%$ |
| test_step_mdp_speed[False-True-True-True-False] | 59.1700μs | 16.2131μs | 61.6784 KOps/s | 62.0847 KOps/s | $\color{#d91a1a}-0.65\%$ |
| test_step_mdp_speed[False-True-True-False-True] | 39.2630μs | 16.2779μs | 61.4330 KOps/s | 61.3047 KOps/s | $\color{#35bf28}+0.21\%$ |
| test_step_mdp_speed[False-True-True-False-False] | 56.4450μs | 10.2412μs | 97.6446 KOps/s | 96.7279 KOps/s | $\color{#35bf28}+0.95\%$ |
| test_step_mdp_speed[False-True-False-True-True] | 59.7020μs | 25.4127μs | 39.3504 KOps/s | 38.7533 KOps/s | $\color{#35bf28}+1.54\%$ |
| test_step_mdp_speed[False-True-False-True-False] | 55.2730μs | 17.1035μs | 58.4677 KOps/s | 57.7954 KOps/s | $\color{#35bf28}+1.16\%$ |
| test_step_mdp_speed[False-True-False-False-True] | 44.1720μs | 17.4633μs | 57.2631 KOps/s | 56.5512 KOps/s | $\color{#35bf28}+1.26\%$ |
| test_step_mdp_speed[False-True-False-False-False] | 72.2950μs | 11.5203μs | 86.8037 KOps/s | 85.7605 KOps/s | $\color{#35bf28}+1.22\%$ |
| test_step_mdp_speed[False-False-True-True-True] | 86.6810μs | 27.1981μs | 36.7673 KOps/s | 36.6139 KOps/s | $\color{#35bf28}+0.42\%$ |
| test_step_mdp_speed[False-False-True-True-False] | 47.6390μs | 18.5789μs | 53.8245 KOps/s | 53.4197 KOps/s | $\color{#35bf28}+0.76\%$ |
| test_step_mdp_speed[False-False-True-False-True] | 65.9630μs | 17.2546μs | 57.9555 KOps/s | 56.9784 KOps/s | $\color{#35bf28}+1.71\%$ |
| test_step_mdp_speed[False-False-True-False-False] | 34.3240μs | 11.4385μs | 87.4241 KOps/s | 87.9153 KOps/s | $\color{#d91a1a}-0.56\%$ |
| test_step_mdp_speed[False-False-False-True-True] | 43.7820μs | 28.7157μs | 34.8241 KOps/s | 34.6462 KOps/s | $\color{#35bf28}+0.51\%$ |
| test_step_mdp_speed[False-False-False-True-False] | 73.1560μs | 19.5834μs | 51.0636 KOps/s | 50.9585 KOps/s | $\color{#35bf28}+0.21\%$ |
| test_step_mdp_speed[False-False-False-False-True] | 35.3860μs | 18.6051μs | 53.7486 KOps/s | 52.9428 KOps/s | $\color{#35bf28}+1.52\%$ |
| test_step_mdp_speed[False-False-False-False-False] | 62.2670μs | 12.6253μs | 79.2060 KOps/s | 78.8259 KOps/s | $\color{#35bf28}+0.48\%$ |
| test_values[generalized_advantage_estimate-True-True] | 9.8038ms | 9.3477ms | 106.9783 Ops/s | 102.5110 Ops/s | $\color{#35bf28}+4.36\%$ |
| test_values[vec_generalized_advantage_estimate-True-True] | 37.1094ms | 33.5259ms | 29.8277 Ops/s | 29.8811 Ops/s | $\color{#d91a1a}-0.18\%$ |
| test_values[td0_return_estimate-False-False] | 0.2330ms | 0.1804ms | 5.5445 KOps/s | 5.4880 KOps/s | $\color{#35bf28}+1.03\%$ |
| test_values[td1_return_estimate-False-False] | 26.1356ms | 22.8502ms | 43.7632 Ops/s | 41.4164 Ops/s | $\textbf{\color{#35bf28}+5.67\%}$ |
| test_values[vec_td1_return_estimate-False-False] | 35.5232ms | 33.6493ms | 29.7183 Ops/s | 29.7692 Ops/s | $\color{#d91a1a}-0.17\%$ |
| test_values[td_lambda_return_estimate-True-False] | 35.9929ms | 33.1154ms | 30.1974 Ops/s | 28.5451 Ops/s | $\textbf{\color{#35bf28}+5.79\%}$ |
| test_values[vec_td_lambda_return_estimate-True-False] | 36.1030ms | 33.6655ms | 29.7040 Ops/s | 29.7782 Ops/s | $\color{#d91a1a}-0.25\%$ |
| test_gae_speed[generalized_advantage_estimate-False-1-512] | 11.3004ms | 8.2385ms | 121.3815 Ops/s | 116.9440 Ops/s | $\color{#35bf28}+3.79\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.1403ms | 1.7758ms | 563.1186 Ops/s | 529.5992 Ops/s | $\textbf{\color{#35bf28}+6.33\%}$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4445ms | 0.3477ms | 2.8760 KOps/s | 2.8095 KOps/s | $\color{#35bf28}+2.37\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 48.0511ms | 45.5206ms | 21.9681 Ops/s | 23.6932 Ops/s | $\textbf{\color{#d91a1a}-7.28\%}$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 4.8934ms | 3.1446ms | 318.0062 Ops/s | 328.0703 Ops/s | $\color{#d91a1a}-3.07\%$ |
| test_dqn_speed | 6.3870ms | 1.3552ms | 737.8909 Ops/s | 719.0404 Ops/s | $\color{#35bf28}+2.62\%$ |
| test_ddpg_speed | 3.7247ms | 2.9099ms | 343.6555 Ops/s | 342.9027 Ops/s | $\color{#35bf28}+0.22\%$ |
| test_sac_speed | 11.4324ms | 8.6781ms | 115.2325 Ops/s | 105.1648 Ops/s | $\textbf{\color{#35bf28}+9.57\%}$ |
| test_redq_speed | 0.1004s | 14.6201ms | 68.3992 Ops/s | 74.5903 Ops/s | $\textbf{\color{#d91a1a}-8.30\%}$ |
| test_redq_deprec_speed | 15.1831ms | 14.2596ms | 70.1284 Ops/s | 70.9278 Ops/s | $\color{#d91a1a}-1.13\%$ |
| test_td3_speed | 8.8918ms | 8.5586ms | 116.8409 Ops/s | 114.0368 Ops/s | $\color{#35bf28}+2.46\%$ |
| test_cql_speed | 38.1979ms | 36.8355ms | 27.1477 Ops/s | 26.8119 Ops/s | $\color{#35bf28}+1.25\%$ |
| test_a2c_speed | 8.7842ms | 7.5392ms | 132.6401 Ops/s | 130.1530 Ops/s | $\color{#35bf28}+1.91\%$ |
| test_ppo_speed | 13.5535ms | 8.1119ms | 123.2751 Ops/s | 126.3649 Ops/s | $\color{#d91a1a}-2.45\%$ |
| test_reinforce_speed | 7.7784ms | 6.7982ms | 147.0976 Ops/s | 145.5173 Ops/s | $\color{#35bf28}+1.09\%$ |
| test_iql_speed | 38.2174ms | 33.6702ms | 29.6999 Ops/s | 29.8851 Ops/s | $\color{#d91a1a}-0.62\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.3231ms | 2.3715ms | 421.6810 Ops/s | 434.6546 Ops/s | $\color{#d91a1a}-2.98\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8463ms | 0.5235ms | 1.9104 KOps/s | 1.9305 KOps/s | $\color{#d91a1a}-1.04\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.4924ms | 0.4847ms | 2.0632 KOps/s | 2.0032 KOps/s | $\color{#35bf28}+3.00\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.9475ms | 2.3069ms | 433.4815 Ops/s | 414.1645 Ops/s | $\color{#35bf28}+4.66\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.9364ms | 0.5071ms | 1.9718 KOps/s | 1.9664 KOps/s | $\color{#35bf28}+0.28\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 1.3122ms | 0.4961ms | 2.0156 KOps/s | 2.0252 KOps/s | $\color{#d91a1a}-0.48\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.0155ms | 1.2375ms | 808.0866 Ops/s | 786.7068 Ops/s | $\color{#35bf28}+2.72\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 1.6949ms | 1.1732ms | 852.3914 Ops/s | 836.7683 Ops/s | $\color{#35bf28}+1.87\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 2.5292ms | 2.3644ms | 422.9357 Ops/s | 413.5734 Ops/s | $\color{#35bf28}+2.26\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.7556ms | 0.6188ms | 1.6160 KOps/s | 1.5879 KOps/s | $\color{#35bf28}+1.77\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 3.6146ms | 0.6002ms | 1.6661 KOps/s | 1.6452 KOps/s | $\color{#35bf28}+1.27\%$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 2.4535ms | 2.2600ms | 442.4699 Ops/s | 435.1385 Ops/s | $\color{#35bf28}+1.68\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.8308ms | 0.5067ms | 1.9734 KOps/s | 1.9374 KOps/s | $\color{#35bf28}+1.86\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7758ms | 0.4853ms | 2.0604 KOps/s | 1.8778 KOps/s | $\textbf{\color{#35bf28}+9.73\%}$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.8922ms | 2.3108ms | 432.7457 Ops/s | 431.5418 Ops/s | $\color{#35bf28}+0.28\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.1090ms | 0.5011ms | 1.9957 KOps/s | 1.9547 KOps/s | $\color{#35bf28}+2.10\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 1.0926ms | 0.4913ms | 2.0356 KOps/s | 1.9800 KOps/s | $\color{#35bf28}+2.81\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 2.5359ms | 2.3471ms | 426.0551 Ops/s | 411.8816 Ops/s | $\color{#35bf28}+3.44\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8995ms | 0.6263ms | 1.5966 KOps/s | 1.5762 KOps/s | $\color{#35bf28}+1.29\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.9175ms | 0.5987ms | 1.6703 KOps/s | 1.6450 KOps/s | $\color{#35bf28}+1.54\%$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1199s | 8.2055ms | 121.8701 Ops/s | 121.2272 Ops/s | $\color{#35bf28}+0.53\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 15.3006ms | 12.7320ms | 78.5421 Ops/s | 76.1609 Ops/s | $\color{#35bf28}+3.13\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 1.7119ms | 1.0764ms | 929.0355 Ops/s | 823.5155 Ops/s | $\textbf{\color{#35bf28}+12.81\%}$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1072s | 5.7720ms | 173.2514 Ops/s | 170.2512 Ops/s | $\color{#35bf28}+1.76\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 14.8745ms | 12.6689ms | 78.9333 Ops/s | 76.7655 Ops/s | $\color{#35bf28}+2.82\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 1.8506ms | 1.0907ms | 916.8477 Ops/s | 814.2885 Ops/s | $\textbf{\color{#35bf28}+12.59\%}$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1119s | 8.2437ms | 121.3042 Ops/s | 119.0641 Ops/s | $\color{#35bf28}+1.88\%$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 16.9090ms | 13.1295ms | 76.1646 Ops/s | 74.8992 Ops/s | $\color{#35bf28}+1.69\%$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.1344ms | 1.3791ms | 725.0892 Ops/s | 698.9755 Ops/s | $\color{#35bf28}+3.74\%$ |
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests
Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}5$.
Expand to view detailed results
| Name | Max | Mean | Ops | Ops on Repo HEAD |
Change |
|---|---|---|---|---|---|
| test_single | 0.1160s | 0.1158s | 8.6367 Ops/s | 8.5706 Ops/s | $\color{#35bf28}+0.77\%$ |
| test_sync | 0.1023s | 0.1015s | 9.8479 Ops/s | 9.8118 Ops/s | $\color{#35bf28}+0.37\%$ |
| test_async | 0.1914s | 95.0424ms | 10.5216 Ops/s | 10.3051 Ops/s | $\color{#35bf28}+2.10\%$ |
| test_single_pixels | 0.1279s | 0.1267s | 7.8897 Ops/s | 7.8849 Ops/s | $\color{#35bf28}+0.06\%$ |
| test_sync_pixels | 85.0584ms | 81.8755ms | 12.2137 Ops/s | 12.2706 Ops/s | $\color{#d91a1a}-0.46\%$ |
| test_async_pixels | 0.1542s | 65.6919ms | 15.2226 Ops/s | 12.3883 Ops/s | $\textbf{\color{#35bf28}+22.88\%}$ |
| test_simple | 0.8325s | 0.7778s | 1.2857 Ops/s | 1.2906 Ops/s | $\color{#d91a1a}-0.38\%$ |
| test_transformed | 1.0831s | 1.0241s | 0.9764 Ops/s | 0.9747 Ops/s | $\color{#35bf28}+0.18\%$ |
| test_serial | 2.5257s | 2.4750s | 0.4040 Ops/s | 0.4047 Ops/s | $\color{#d91a1a}-0.17\%$ |
| test_parallel | 2.3280s | 2.3032s | 0.4342 Ops/s | 0.4332 Ops/s | $\color{#35bf28}+0.24\%$ |
| test_step_mdp_speed[True-True-True-True-True] | 96.2820μs | 33.3931μs | 29.9463 KOps/s | 30.5849 KOps/s | $\color{#d91a1a}-2.09\%$ |
| test_step_mdp_speed[True-True-True-True-False] | 41.9510μs | 19.8931μs | 50.2687 KOps/s | 50.6029 KOps/s | $\color{#d91a1a}-0.66\%$ |
| test_step_mdp_speed[True-True-True-False-True] | 39.4700μs | 18.7494μs | 53.3350 KOps/s | 53.5147 KOps/s | $\color{#d91a1a}-0.34\%$ |
| test_step_mdp_speed[True-True-True-False-False] | 36.9410μs | 11.3721μs | 87.9344 KOps/s | 89.2989 KOps/s | $\color{#d91a1a}-1.53\%$ |
| test_step_mdp_speed[True-True-False-True-True] | 50.5920μs | 35.2637μs | 28.3577 KOps/s | 28.8091 KOps/s | $\color{#d91a1a}-1.57\%$ |
| test_step_mdp_speed[True-True-False-True-False] | 40.8800μs | 21.8100μs | 45.8504 KOps/s | 46.0118 KOps/s | $\color{#d91a1a}-0.35\%$ |
| test_step_mdp_speed[True-True-False-False-True] | 41.6120μs | 20.6315μs | 48.4695 KOps/s | 48.4935 KOps/s | $\color{#d91a1a}-0.05\%$ |
| test_step_mdp_speed[True-True-False-False-False] | 28.0610μs | 13.1862μs | 75.8370 KOps/s | 76.3053 KOps/s | $\color{#d91a1a}-0.61\%$ |
| test_step_mdp_speed[True-False-True-True-True] | 64.1520μs | 37.3944μs | 26.7420 KOps/s | 27.0942 KOps/s | $\color{#d91a1a}-1.30\%$ |
| test_step_mdp_speed[True-False-True-True-False] | 52.3310μs | 23.6431μs | 42.2956 KOps/s | 42.5157 KOps/s | $\color{#d91a1a}-0.52\%$ |
| test_step_mdp_speed[True-False-True-False-True] | 38.4120μs | 20.6728μs | 48.3727 KOps/s | 47.7711 KOps/s | $\color{#35bf28}+1.26\%$ |
| test_step_mdp_speed[True-False-True-False-False] | 31.4910μs | 13.2720μs | 75.3463 KOps/s | 75.7364 KOps/s | $\color{#d91a1a}-0.52\%$ |
| test_step_mdp_speed[True-False-False-True-True] | 65.8610μs | 38.8919μs | 25.7123 KOps/s | 25.8796 KOps/s | $\color{#d91a1a}-0.65\%$ |
| test_step_mdp_speed[True-False-False-True-False] | 96.0820μs | 25.5997μs | 39.0630 KOps/s | 39.3394 KOps/s | $\color{#d91a1a}-0.70\%$ |
| test_step_mdp_speed[True-False-False-False-True] | 49.5510μs | 22.4488μs | 44.5458 KOps/s | 45.0086 KOps/s | $\color{#d91a1a}-1.03\%$ |
| test_step_mdp_speed[True-False-False-False-False] | 31.7210μs | 15.1653μs | 65.9400 KOps/s | 67.2557 KOps/s | $\color{#d91a1a}-1.96\%$ |
| test_step_mdp_speed[False-True-True-True-True] | 55.1710μs | 37.2965μs | 26.8122 KOps/s | 27.0783 KOps/s | $\color{#d91a1a}-0.98\%$ |
| test_step_mdp_speed[False-True-True-True-False] | 42.8610μs | 23.7071μs | 42.1815 KOps/s | 42.1439 KOps/s | $\color{#35bf28}+0.09\%$ |
| test_step_mdp_speed[False-True-True-False-True] | 40.7400μs | 24.8752μs | 40.2006 KOps/s | 40.8364 KOps/s | $\color{#d91a1a}-1.56\%$ |
| test_step_mdp_speed[False-True-True-False-False] | 34.9100μs | 15.0851μs | 66.2906 KOps/s | 67.2354 KOps/s | $\color{#d91a1a}-1.41\%$ |
| test_step_mdp_speed[False-True-False-True-True] | 65.4910μs | 38.7929μs | 25.7779 KOps/s | 25.6723 KOps/s | $\color{#35bf28}+0.41\%$ |
| test_step_mdp_speed[False-True-False-True-False] | 41.5510μs | 25.5755μs | 39.0999 KOps/s | 39.5501 KOps/s | $\color{#d91a1a}-1.14\%$ |
| test_step_mdp_speed[False-True-False-False-True] | 43.8210μs | 26.4008μs | 37.8777 KOps/s | 38.0910 KOps/s | $\color{#d91a1a}-0.56\%$ |
| test_step_mdp_speed[False-True-False-False-False] | 38.7310μs | 17.0030μs | 58.8131 KOps/s | 59.1732 KOps/s | $\color{#d91a1a}-0.61\%$ |
| test_step_mdp_speed[False-False-True-True-True] | 60.1410μs | 41.1495μs | 24.3016 KOps/s | 24.6983 KOps/s | $\color{#d91a1a}-1.61\%$ |
| test_step_mdp_speed[False-False-True-True-False] | 85.3310μs | 27.7281μs | 36.0645 KOps/s | 36.7163 KOps/s | $\color{#d91a1a}-1.78\%$ |
| test_step_mdp_speed[False-False-True-False-True] | 52.9210μs | 26.8747μs | 37.2098 KOps/s | 38.1930 KOps/s | $\color{#d91a1a}-2.57\%$ |
| test_step_mdp_speed[False-False-True-False-False] | 39.4710μs | 17.0234μs | 58.7427 KOps/s | 60.3084 KOps/s | $\color{#d91a1a}-2.60\%$ |
| test_step_mdp_speed[False-False-False-True-True] | 65.6410μs | 42.6143μs | 23.4663 KOps/s | 23.5304 KOps/s | $\color{#d91a1a}-0.27\%$ |
| test_step_mdp_speed[False-False-False-True-False] | 52.5610μs | 29.4425μs | 33.9645 KOps/s | 34.2799 KOps/s | $\color{#d91a1a}-0.92\%$ |
| test_step_mdp_speed[False-False-False-False-True] | 43.7010μs | 28.1828μs | 35.4826 KOps/s | 36.0431 KOps/s | $\color{#d91a1a}-1.56\%$ |
| test_step_mdp_speed[False-False-False-False-False] | 43.6700μs | 18.5376μs | 53.9445 KOps/s | 54.4524 KOps/s | $\color{#d91a1a}-0.93\%$ |
| test_values[generalized_advantage_estimate-True-True] | 25.3544ms | 24.9137ms | 40.1386 Ops/s | 40.0983 Ops/s | $\color{#35bf28}+0.10\%$ |
| test_values[vec_generalized_advantage_estimate-True-True] | 87.5329ms | 3.3209ms | 301.1218 Ops/s | 314.2743 Ops/s | $\color{#d91a1a}-4.19\%$ |
| test_values[td0_return_estimate-False-False] | 92.3420μs | 64.0622μs | 15.6098 KOps/s | 15.6764 KOps/s | $\color{#d91a1a}-0.42\%$ |
| test_values[td1_return_estimate-False-False] | 53.8513ms | 53.3629ms | 18.7396 Ops/s | 18.7358 Ops/s | $\color{#35bf28}+0.02\%$ |
| test_values[vec_td1_return_estimate-False-False] | 2.0813ms | 1.7682ms | 565.5353 Ops/s | 565.5376 Ops/s | $-0.00\%$ |
| test_values[td_lambda_return_estimate-True-False] | 85.4243ms | 84.9818ms | 11.7672 Ops/s | 11.7206 Ops/s | $\color{#35bf28}+0.40\%$ |
| test_values[vec_td_lambda_return_estimate-True-False] | 2.1360ms | 1.7650ms | 566.5825 Ops/s | 565.6951 Ops/s | $\color{#35bf28}+0.16\%$ |
| test_gae_speed[generalized_advantage_estimate-False-1-512] | 23.9492ms | 23.7315ms | 42.1381 Ops/s | 41.9107 Ops/s | $\color{#35bf28}+0.54\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.8909ms | 0.6993ms | 1.4300 KOps/s | 1.4289 KOps/s | $\color{#35bf28}+0.08\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7236ms | 0.6529ms | 1.5316 KOps/s | 1.5201 KOps/s | $\color{#35bf28}+0.75\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.4906ms | 1.4548ms | 687.3836 Ops/s | 687.8456 Ops/s | $\color{#d91a1a}-0.07\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.9314ms | 0.6695ms | 1.4936 KOps/s | 1.4845 KOps/s | $\color{#35bf28}+0.61\%$ |
| test_dqn_speed | 1.8216ms | 1.4736ms | 678.6066 Ops/s | 690.9173 Ops/s | $\color{#d91a1a}-1.78\%$ |
| test_ddpg_speed | 3.3253ms | 3.0054ms | 332.7397 Ops/s | 337.0056 Ops/s | $\color{#d91a1a}-1.27\%$ |
| test_sac_speed | 8.8732ms | 8.6140ms | 116.0897 Ops/s | 117.5075 Ops/s | $\color{#d91a1a}-1.21\%$ |
| test_redq_speed | 11.3136ms | 10.5434ms | 94.8461 Ops/s | 97.8302 Ops/s | $\color{#d91a1a}-3.05\%$ |
| test_redq_deprec_speed | 12.1959ms | 11.6095ms | 86.1366 Ops/s | 85.8475 Ops/s | $\color{#35bf28}+0.34\%$ |
| test_td3_speed | 17.1526ms | 8.6951ms | 115.0074 Ops/s | 118.8552 Ops/s | $\color{#d91a1a}-3.24\%$ |
| test_cql_speed | 27.5573ms | 26.5550ms | 37.6577 Ops/s | 38.6885 Ops/s | $\color{#d91a1a}-2.66\%$ |
| test_a2c_speed | 5.9792ms | 5.7922ms | 172.6470 Ops/s | 174.8075 Ops/s | $\color{#d91a1a}-1.24\%$ |
| test_ppo_speed | 6.2594ms | 6.0622ms | 164.9578 Ops/s | 166.7314 Ops/s | $\color{#d91a1a}-1.06\%$ |
| test_reinforce_speed | 4.9918ms | 4.7654ms | 209.8445 Ops/s | 216.4252 Ops/s | $\color{#d91a1a}-3.04\%$ |
| test_iql_speed | 21.0907ms | 20.3950ms | 49.0316 Ops/s | 50.2914 Ops/s | $\color{#d91a1a}-2.50\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.0602ms | 2.8048ms | 356.5346 Ops/s | 362.6844 Ops/s | $\color{#d91a1a}-1.70\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7324ms | 0.6031ms | 1.6581 KOps/s | 1.6699 KOps/s | $\color{#d91a1a}-0.71\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.3135ms | 0.5785ms | 1.7286 KOps/s | 1.7423 KOps/s | $\color{#d91a1a}-0.79\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.1331ms | 2.8344ms | 352.8131 Ops/s | 361.0986 Ops/s | $\color{#d91a1a}-2.29\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7078ms | 0.5943ms | 1.6826 KOps/s | 1.6993 KOps/s | $\color{#d91a1a}-0.98\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.2340ms | 0.5736ms | 1.7435 KOps/s | 1.7554 KOps/s | $\color{#d91a1a}-0.68\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 1.7487ms | 1.5826ms | 631.8858 Ops/s | 647.7757 Ops/s | $\color{#d91a1a}-2.45\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 1.6338ms | 1.4960ms | 668.4339 Ops/s | 680.8206 Ops/s | $\color{#d91a1a}-1.82\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.1023ms | 2.9517ms | 338.7827 Ops/s | 343.1340 Ops/s | $\color{#d91a1a}-1.27\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.2772ms | 0.7368ms | 1.3573 KOps/s | 1.3717 KOps/s | $\color{#d91a1a}-1.05\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.1147s | 0.8156ms | 1.2261 KOps/s | 1.4151 KOps/s | $\textbf{\color{#d91a1a}-13.35\%}$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 2.9303ms | 2.8282ms | 353.5761 Ops/s | 358.1402 Ops/s | $\color{#d91a1a}-1.27\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.7316ms | 0.6040ms | 1.6556 KOps/s | 1.6727 KOps/s | $\color{#d91a1a}-1.02\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.4461ms | 0.5836ms | 1.7135 KOps/s | 1.7311 KOps/s | $\color{#d91a1a}-1.01\%$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.0297ms | 2.8511ms | 350.7400 Ops/s | 355.2399 Ops/s | $\color{#d91a1a}-1.27\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.1144s | 0.7800ms | 1.2820 KOps/s | 1.6830 KOps/s | $\textbf{\color{#d91a1a}-23.83\%}$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7218ms | 0.5700ms | 1.7542 KOps/s | 1.7547 KOps/s | $\color{#d91a1a}-0.03\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.0578ms | 2.9752ms | 336.1158 Ops/s | 340.8276 Ops/s | $\color{#d91a1a}-1.38\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 0.8294ms | 0.7377ms | 1.3556 KOps/s | 1.3702 KOps/s | $\color{#d91a1a}-1.07\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 4.5973ms | 0.7170ms | 1.3947 KOps/s | 1.4144 KOps/s | $\color{#d91a1a}-1.40\%$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1158s | 7.1636ms | 139.5955 Ops/s | 104.3140 Ops/s | $\textbf{\color{#35bf28}+33.82\%}$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 0.1239s | 18.0113ms | 55.5206 Ops/s | 64.1727 Ops/s | $\textbf{\color{#d91a1a}-13.48\%}$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.3040ms | 1.2550ms | 796.8359 Ops/s | 855.8339 Ops/s | $\textbf{\color{#d91a1a}-6.89\%}$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1139s | 7.1707ms | 139.4560 Ops/s | 139.7709 Ops/s | $\color{#d91a1a}-0.23\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 18.1749ms | 15.8805ms | 62.9702 Ops/s | 64.2679 Ops/s | $\color{#d91a1a}-2.02\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.3344ms | 1.2385ms | 807.4166 Ops/s | 854.3269 Ops/s | $\textbf{\color{#d91a1a}-5.49\%}$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1130s | 7.5053ms | 133.2385 Ops/s | 133.8682 Ops/s | $\color{#d91a1a}-0.47\%$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 18.7200ms | 16.2254ms | 61.6318 Ops/s | 55.3482 Ops/s | $\textbf{\color{#35bf28}+11.35\%}$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.5903ms | 1.5831ms | 631.6668 Ops/s | 614.9586 Ops/s | $\color{#35bf28}+2.72\%$ |