rl
rl copied to clipboard
Published
20 hours ago •
pytorch
pytorch
[BugFix] Fix isaac - bis
:link: Helpful Links
:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2119
- :page_facing_up: Preview Python docs built from this PR
Note: Links to docs will display an error until the docs builds have been completed.
:x: 2 New Failures, 25 Unrelated Failures
As of commit fb9b27367f4fdf9118c9664c398e4a8eac213f23 with merge base 0707e072236d955497b9f83de7c9fddd11093385 (
NEW FAILURES - The following jobs have failed:
-
Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
RuntimeError: Command docker exec -t 1435d1df081f1221056f8a1e8359a71c4e092b0262ad6a6391022a7060182711 /exec failed with exit code 139 -
Unit-tests on Windows / unittests-cpu / windows-job (gh)
The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128
FLAKY - The following jobs failed but were likely due to flakiness present on trunk:
-
Libs Tests on Linux / unittests-atari-dqn (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-brax (3.11, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-d4rl (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-envpool (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-gendgrl (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-jumanji (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-meltingpot / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-minari (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-openx (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-pettingzoo / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-robohive (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-roboset (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-smacv2 (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-vd4rl (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Libs Tests on Linux / unittests-vmas (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128 -
Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 128
BROKEN TRUNK - The following jobs failed but were present on the merge base:
👉 Rebase onto the `viable/strict` branch to avoid these failures
-
Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh) (trunk failure)
test/test_env.py::TestInfoDict::test_auto_register[device0] -
Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh) (trunk failure)
test/test_env.py::TestInfoDict::test_auto_register[device0] -
Unit-tests on Linux / tests-cpu (3.8) / linux-job (gh) (trunk failure)
test/test_env.py::TestInfoDict::test_auto_register[device0] -
Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh) (trunk failure)
test/test_env.py::TestInfoDict::test_auto_register[device0]
This comment was automatically generated by Dr. CI and updates every 15 minutes.
![pytorch-bot[bot] avatar](https://avatars.githubusercontent.com/in/40112?v=4 "Published by a pub.dev verified publisher"){kind=small}
$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests
Total Benchmarks: 91. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}7$.
Expand to view detailed results
| Name | Max | Mean | Ops | Ops on Repo HEAD |
Change |
|---|---|---|---|---|---|
| test_single | 54.5086ms | 54.1310ms | 18.4737 Ops/s | 18.6561 Ops/s | $\color{#d91a1a}-0.98\%$ |
| test_sync | 33.9116ms | 29.3339ms | 34.0903 Ops/s | 33.5732 Ops/s | $\color{#35bf28}+1.54\%$ |
| test_async | 55.6845ms | 28.2629ms | 35.3820 Ops/s | 35.3020 Ops/s | $\color{#35bf28}+0.23\%$ |
| test_simple | 0.4091s | 0.3551s | 2.8163 Ops/s | 2.9259 Ops/s | $\color{#d91a1a}-3.75\%$ |
| test_transformed | 0.5467s | 0.4986s | 2.0056 Ops/s | 2.0060 Ops/s | $\color{#d91a1a}-0.02\%$ |
| test_serial | 1.2781s | 1.2255s | 0.8160 Ops/s | 0.8165 Ops/s | $\color{#d91a1a}-0.06\%$ |
| test_parallel | 1.0539s | 1.0150s | 0.9852 Ops/s | 0.9782 Ops/s | $\color{#35bf28}+0.71\%$ |
| test_step_mdp_speed[True-True-True-True-True] | 0.2212ms | 21.4965μs | 46.5192 KOps/s | 46.4584 KOps/s | $\color{#35bf28}+0.13\%$ |
| test_step_mdp_speed[True-True-True-True-False] | 34.0340μs | 13.1656μs | 75.9553 KOps/s | 75.4123 KOps/s | $\color{#35bf28}+0.72\%$ |
| test_step_mdp_speed[True-True-True-False-True] | 41.9890μs | 12.5999μs | 79.3656 KOps/s | 79.3447 KOps/s | $\color{#35bf28}+0.03\%$ |
| test_step_mdp_speed[True-True-True-False-False] | 0.1320ms | 7.8224μs | 127.8373 KOps/s | 129.9896 KOps/s | $\color{#d91a1a}-1.66\%$ |
| test_step_mdp_speed[True-True-False-True-True] | 76.4600μs | 22.6573μs | 44.1358 KOps/s | 43.4510 KOps/s | $\color{#35bf28}+1.58\%$ |
| test_step_mdp_speed[True-True-False-True-False] | 38.3520μs | 14.4295μs | 69.3023 KOps/s | 68.8892 KOps/s | $\color{#35bf28}+0.60\%$ |
| test_step_mdp_speed[True-True-False-False-True] | 40.6060μs | 13.7956μs | 72.4867 KOps/s | 71.3746 KOps/s | $\color{#35bf28}+1.56\%$ |
| test_step_mdp_speed[True-True-False-False-False] | 29.5050μs | 8.9798μs | 111.3616 KOps/s | 110.7851 KOps/s | $\color{#35bf28}+0.52\%$ |
| test_step_mdp_speed[True-False-True-True-True] | 51.1360μs | 24.5340μs | 40.7598 KOps/s | 40.4957 KOps/s | $\color{#35bf28}+0.65\%$ |
| test_step_mdp_speed[True-False-True-True-False] | 58.8710μs | 15.8527μs | 63.0807 KOps/s | 62.6240 KOps/s | $\color{#35bf28}+0.73\%$ |
| test_step_mdp_speed[True-False-True-False-True] | 0.1564ms | 14.0963μs | 70.9406 KOps/s | 71.8572 KOps/s | $\color{#d91a1a}-1.28\%$ |
| test_step_mdp_speed[True-False-True-False-False] | 40.6160μs | 8.9373μs | 111.8901 KOps/s | 110.6246 KOps/s | $\color{#35bf28}+1.14\%$ |
| test_step_mdp_speed[True-False-False-True-True] | 52.1680μs | 25.5673μs | 39.1125 KOps/s | 38.8536 KOps/s | $\color{#35bf28}+0.67\%$ |
| test_step_mdp_speed[True-False-False-True-False] | 42.9410μs | 17.1316μs | 58.3718 KOps/s | 58.5893 KOps/s | $\color{#d91a1a}-0.37\%$ |
| test_step_mdp_speed[True-False-False-False-True] | 46.0360μs | 15.0840μs | 66.2956 KOps/s | 66.0150 KOps/s | $\color{#35bf28}+0.43\%$ |
| test_step_mdp_speed[True-False-False-False-False] | 37.5400μs | 10.1601μs | 98.4243 KOps/s | 98.1646 KOps/s | $\color{#35bf28}+0.26\%$ |
| test_step_mdp_speed[False-True-True-True-True] | 59.6310μs | 24.5102μs | 40.7993 KOps/s | 41.0031 KOps/s | $\color{#d91a1a}-0.50\%$ |
| test_step_mdp_speed[False-True-True-True-False] | 51.9880μs | 15.7442μs | 63.5154 KOps/s | 63.0690 KOps/s | $\color{#35bf28}+0.71\%$ |
| test_step_mdp_speed[False-True-True-False-True] | 44.1820μs | 16.2338μs | 61.5999 KOps/s | 61.4507 KOps/s | $\color{#35bf28}+0.24\%$ |
| test_step_mdp_speed[False-True-True-False-False] | 32.6010μs | 10.1893μs | 98.1424 KOps/s | 98.3488 KOps/s | $\color{#d91a1a}-0.21\%$ |
| test_step_mdp_speed[False-True-False-True-True] | 72.8170μs | 25.5661μs | 39.1144 KOps/s | 39.1357 KOps/s | $\color{#d91a1a}-0.05\%$ |
| test_step_mdp_speed[False-True-False-True-False] | 50.7750μs | 17.0576μs | 58.6249 KOps/s | 58.4621 KOps/s | $\color{#35bf28}+0.28\%$ |
| test_step_mdp_speed[False-True-False-False-True] | 71.3140μs | 17.4188μs | 57.4091 KOps/s | 57.7593 KOps/s | $\color{#d91a1a}-0.61\%$ |
| test_step_mdp_speed[False-True-False-False-False] | 37.1900μs | 11.4742μs | 87.1517 KOps/s | 88.0829 KOps/s | $\color{#d91a1a}-1.06\%$ |
| test_step_mdp_speed[False-False-True-True-True] | 77.6620μs | 26.7855μs | 37.3336 KOps/s | 36.8952 KOps/s | $\color{#35bf28}+1.19\%$ |
| test_step_mdp_speed[False-False-True-True-False] | 53.4000μs | 18.3662μs | 54.4480 KOps/s | 54.4372 KOps/s | $\color{#35bf28}+0.02\%$ |
| test_step_mdp_speed[False-False-True-False-True] | 68.5510μs | 17.4455μs | 57.3213 KOps/s | 57.3156 KOps/s | $+0.01\%$ |
| test_step_mdp_speed[False-False-True-False-False] | 42.6300μs | 11.4041μs | 87.6881 KOps/s | 87.9615 KOps/s | $\color{#d91a1a}-0.31\%$ |
| test_step_mdp_speed[False-False-False-True-True] | 39.4440μs | 28.0631μs | 35.6340 KOps/s | 34.8968 KOps/s | $\color{#35bf28}+2.11\%$ |
| test_step_mdp_speed[False-False-False-True-False] | 53.0600μs | 19.3426μs | 51.6993 KOps/s | 50.8450 KOps/s | $\color{#35bf28}+1.68\%$ |
| test_step_mdp_speed[False-False-False-False-True] | 60.0730μs | 18.4043μs | 54.3352 KOps/s | 53.5644 KOps/s | $\color{#35bf28}+1.44\%$ |
| test_step_mdp_speed[False-False-False-False-False] | 32.1900μs | 12.5729μs | 79.5360 KOps/s | 79.7276 KOps/s | $\color{#d91a1a}-0.24\%$ |
| test_values[generalized_advantage_estimate-True-True] | 11.6274ms | 9.3679ms | 106.7473 Ops/s | 107.0455 Ops/s | $\color{#d91a1a}-0.28\%$ |
| test_values[vec_generalized_advantage_estimate-True-True] | 42.6090ms | 35.5474ms | 28.1315 Ops/s | 28.5195 Ops/s | $\color{#d91a1a}-1.36\%$ |
| test_values[td0_return_estimate-False-False] | 0.2769ms | 0.1798ms | 5.5618 KOps/s | 5.8903 KOps/s | $\textbf{\color{#d91a1a}-5.58\%}$ |
| test_values[td1_return_estimate-False-False] | 23.5977ms | 23.1056ms | 43.2795 Ops/s | 41.8584 Ops/s | $\color{#35bf28}+3.39\%$ |
| test_values[vec_td1_return_estimate-False-False] | 37.9246ms | 35.2557ms | 28.3642 Ops/s | 28.0763 Ops/s | $\color{#35bf28}+1.03\%$ |
| test_values[td_lambda_return_estimate-True-False] | 35.2720ms | 33.0436ms | 30.2630 Ops/s | 30.1711 Ops/s | $\color{#35bf28}+0.30\%$ |
| test_values[vec_td_lambda_return_estimate-True-False] | 48.6045ms | 35.6154ms | 28.0777 Ops/s | 27.9026 Ops/s | $\color{#35bf28}+0.63\%$ |
| test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.2381ms | 8.0916ms | 123.5848 Ops/s | 123.7355 Ops/s | $\color{#d91a1a}-0.12\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 2.4913ms | 1.9245ms | 519.6275 Ops/s | 525.5709 Ops/s | $\color{#d91a1a}-1.13\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.4359ms | 0.3498ms | 2.8590 KOps/s | 2.8616 KOps/s | $\color{#d91a1a}-0.09\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 49.3274ms | 46.6846ms | 21.4203 Ops/s | 22.6575 Ops/s | $\textbf{\color{#d91a1a}-5.46\%}$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 3.6839ms | 3.0193ms | 331.2067 Ops/s | 329.0436 Ops/s | $\color{#35bf28}+0.66\%$ |
| test_dqn_speed | 7.2411ms | 1.3571ms | 736.8732 Ops/s | 743.1759 Ops/s | $\color{#d91a1a}-0.85\%$ |
| test_ddpg_speed | 4.3304ms | 2.8529ms | 350.5184 Ops/s | 350.5804 Ops/s | $\color{#d91a1a}-0.02\%$ |
| test_sac_speed | 9.1969ms | 8.6813ms | 115.1898 Ops/s | 118.5330 Ops/s | $\color{#d91a1a}-2.82\%$ |
| test_redq_speed | 14.7309ms | 13.3246ms | 75.0493 Ops/s | 76.2059 Ops/s | $\color{#d91a1a}-1.52\%$ |
| test_redq_deprec_speed | 88.7213ms | 14.3895ms | 69.4951 Ops/s | 74.6135 Ops/s | $\textbf{\color{#d91a1a}-6.86\%}$ |
| test_td3_speed | 9.8708ms | 8.4117ms | 118.8820 Ops/s | 118.7692 Ops/s | $\color{#35bf28}+0.10\%$ |
| test_cql_speed | 37.9035ms | 36.6310ms | 27.2993 Ops/s | 27.0806 Ops/s | $\color{#35bf28}+0.81\%$ |
| test_a2c_speed | 8.8698ms | 7.5200ms | 132.9785 Ops/s | 133.2072 Ops/s | $\color{#d91a1a}-0.17\%$ |
| test_ppo_speed | 8.7367ms | 7.7439ms | 129.1339 Ops/s | 129.7870 Ops/s | $\color{#d91a1a}-0.50\%$ |
| test_reinforce_speed | 7.5717ms | 6.7335ms | 148.5119 Ops/s | 148.7550 Ops/s | $\color{#d91a1a}-0.16\%$ |
| test_iql_speed | 39.4765ms | 33.7452ms | 29.6339 Ops/s | 30.2117 Ops/s | $\color{#d91a1a}-1.91\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 2.5675ms | 2.2187ms | 450.7098 Ops/s | 465.5942 Ops/s | $\color{#d91a1a}-3.20\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.0416ms | 0.4930ms | 2.0283 KOps/s | 2.0235 KOps/s | $\color{#35bf28}+0.24\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.8481ms | 0.4672ms | 2.1403 KOps/s | 2.1267 KOps/s | $\color{#35bf28}+0.64\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.3248ms | 2.1816ms | 458.3786 Ops/s | 472.4446 Ops/s | $\color{#d91a1a}-2.98\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.0055ms | 0.4837ms | 2.0672 KOps/s | 2.0452 KOps/s | $\color{#35bf28}+1.07\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.8112ms | 0.4642ms | 2.1544 KOps/s | 2.1456 KOps/s | $\color{#35bf28}+0.41\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 1.3648ms | 1.2249ms | 816.3944 Ops/s | 821.2192 Ops/s | $\color{#d91a1a}-0.59\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 1.3648ms | 1.1669ms | 856.9397 Ops/s | 865.6436 Ops/s | $\color{#d91a1a}-1.01\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 2.8028ms | 2.4458ms | 408.8589 Ops/s | 448.1380 Ops/s | $\textbf{\color{#d91a1a}-8.76\%}$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.0248ms | 0.6151ms | 1.6257 KOps/s | 1.6419 KOps/s | $\color{#d91a1a}-0.99\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8424ms | 0.5831ms | 1.7149 KOps/s | 1.7184 KOps/s | $\color{#d91a1a}-0.20\%$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.3212ms | 2.1209ms | 471.5081 Ops/s | 473.2591 Ops/s | $\color{#d91a1a}-0.37\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 0.9281ms | 0.4903ms | 2.0394 KOps/s | 2.0285 KOps/s | $\color{#35bf28}+0.53\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7067ms | 0.4735ms | 2.1118 KOps/s | 2.1101 KOps/s | $\color{#35bf28}+0.08\%$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.3602ms | 2.1514ms | 464.8194 Ops/s | 470.9781 Ops/s | $\color{#d91a1a}-1.31\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.6064ms | 0.4845ms | 2.0641 KOps/s | 2.0298 KOps/s | $\color{#35bf28}+1.69\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.4522ms | 0.4686ms | 2.1341 KOps/s | 2.1405 KOps/s | $\color{#d91a1a}-0.30\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 2.5116ms | 2.2506ms | 444.3205 Ops/s | 449.0909 Ops/s | $\color{#d91a1a}-1.06\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.1657ms | 0.6130ms | 1.6313 KOps/s | 1.6394 KOps/s | $\color{#d91a1a}-0.49\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.7209ms | 0.5829ms | 1.7155 KOps/s | 1.7141 KOps/s | $\color{#35bf28}+0.09\%$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1051s | 7.5471ms | 132.5004 Ops/s | 130.3624 Ops/s | $\color{#35bf28}+1.64\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 14.1590ms | 11.9038ms | 84.0067 Ops/s | 82.5598 Ops/s | $\color{#35bf28}+1.75\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 4.4123ms | 1.1721ms | 853.1619 Ops/s | 901.4317 Ops/s | $\textbf{\color{#d91a1a}-5.35\%}$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1025s | 5.6574ms | 176.7595 Ops/s | 182.5994 Ops/s | $\color{#d91a1a}-3.20\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 14.2290ms | 11.9095ms | 83.9663 Ops/s | 71.2432 Ops/s | $\textbf{\color{#35bf28}+17.86\%}$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 3.5892ms | 1.1316ms | 883.7245 Ops/s | 945.6518 Ops/s | $\textbf{\color{#d91a1a}-6.55\%}$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 98.0735ms | 7.7621ms | 128.8316 Ops/s | 168.9890 Ops/s | $\textbf{\color{#d91a1a}-23.76\%}$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 14.4117ms | 12.2367ms | 81.7211 Ops/s | 80.1964 Ops/s | $\color{#35bf28}+1.90\%$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 3.9036ms | 1.4197ms | 704.3572 Ops/s | 720.4087 Ops/s | $\color{#d91a1a}-2.23\%$ |
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests
Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}5$.
Expand to view detailed results
| Name | Max | Mean | Ops | Ops on Repo HEAD |
Change |
|---|---|---|---|---|---|
| test_single | 0.1178s | 0.1155s | 8.6584 Ops/s | 8.6610 Ops/s | $\color{#d91a1a}-0.03\%$ |
| test_sync | 0.1030s | 0.1022s | 9.7860 Ops/s | 9.8566 Ops/s | $\color{#d91a1a}-0.72\%$ |
| test_async | 0.1889s | 95.5421ms | 10.4666 Ops/s | 10.3928 Ops/s | $\color{#35bf28}+0.71\%$ |
| test_single_pixels | 0.1978s | 0.1338s | 7.4734 Ops/s | 8.0041 Ops/s | $\textbf{\color{#d91a1a}-6.63\%}$ |
| test_sync_pixels | 0.1336s | 85.6700ms | 11.6727 Ops/s | 12.4078 Ops/s | $\textbf{\color{#d91a1a}-5.92\%}$ |
| test_async_pixels | 0.1547s | 78.1907ms | 12.7892 Ops/s | 12.4282 Ops/s | $\color{#35bf28}+2.91\%$ |
| test_simple | 0.8253s | 0.7638s | 1.3093 Ops/s | 1.3092 Ops/s | $+0.01\%$ |
| test_transformed | 1.0613s | 1.0049s | 0.9952 Ops/s | 0.9954 Ops/s | $\color{#d91a1a}-0.02\%$ |
| test_serial | 2.4637s | 2.4092s | 0.4151 Ops/s | 0.4158 Ops/s | $\color{#d91a1a}-0.18\%$ |
| test_parallel | 2.3787s | 2.2993s | 0.4349 Ops/s | 0.4321 Ops/s | $\color{#35bf28}+0.65\%$ |
| test_step_mdp_speed[True-True-True-True-True] | 94.6720μs | 33.6458μs | 29.7214 KOps/s | 30.5200 KOps/s | $\color{#d91a1a}-2.62\%$ |
| test_step_mdp_speed[True-True-True-True-False] | 88.9410μs | 19.9364μs | 50.1595 KOps/s | 52.4031 KOps/s | $\color{#d91a1a}-4.28\%$ |
| test_step_mdp_speed[True-True-True-False-True] | 43.9910μs | 19.0505μs | 52.4921 KOps/s | 53.6521 KOps/s | $\color{#d91a1a}-2.16\%$ |
| test_step_mdp_speed[True-True-True-False-False] | 35.2010μs | 11.3757μs | 87.9064 KOps/s | 90.9102 KOps/s | $\color{#d91a1a}-3.30\%$ |
| test_step_mdp_speed[True-True-False-True-True] | 96.5220μs | 35.4166μs | 28.2353 KOps/s | 29.2000 KOps/s | $\color{#d91a1a}-3.30\%$ |
| test_step_mdp_speed[True-True-False-True-False] | 51.1610μs | 21.9656μs | 45.5257 KOps/s | 47.6199 KOps/s | $\color{#d91a1a}-4.40\%$ |
| test_step_mdp_speed[True-True-False-False-True] | 46.7710μs | 20.8655μs | 47.9260 KOps/s | 49.7975 KOps/s | $\color{#d91a1a}-3.76\%$ |
| test_step_mdp_speed[True-True-False-False-False] | 28.8900μs | 13.3555μs | 74.8756 KOps/s | 77.4939 KOps/s | $\color{#d91a1a}-3.38\%$ |
| test_step_mdp_speed[True-False-True-True-True] | 89.1530μs | 37.2253μs | 26.8634 KOps/s | 27.7796 KOps/s | $\color{#d91a1a}-3.30\%$ |
| test_step_mdp_speed[True-False-True-True-False] | 55.6720μs | 23.6668μs | 42.2532 KOps/s | 43.1733 KOps/s | $\color{#d91a1a}-2.13\%$ |
| test_step_mdp_speed[True-False-True-False-True] | 61.3610μs | 21.1401μs | 47.3034 KOps/s | 50.2910 KOps/s | $\textbf{\color{#d91a1a}-5.94\%}$ |
| test_step_mdp_speed[True-False-True-False-False] | 34.7410μs | 13.3563μs | 74.8711 KOps/s | 78.1456 KOps/s | $\color{#d91a1a}-4.19\%$ |
| test_step_mdp_speed[True-False-False-True-True] | 55.0310μs | 38.6826μs | 25.8514 KOps/s | 26.8916 KOps/s | $\color{#d91a1a}-3.87\%$ |
| test_step_mdp_speed[True-False-False-True-False] | 45.4010μs | 25.2621μs | 39.5850 KOps/s | 40.7757 KOps/s | $\color{#d91a1a}-2.92\%$ |
| test_step_mdp_speed[True-False-False-False-True] | 42.6110μs | 22.4927μs | 44.4589 KOps/s | 46.2212 KOps/s | $\color{#d91a1a}-3.81\%$ |
| test_step_mdp_speed[True-False-False-False-False] | 45.3310μs | 15.1480μs | 66.0152 KOps/s | 68.2860 KOps/s | $\color{#d91a1a}-3.33\%$ |
| test_step_mdp_speed[False-True-True-True-True] | 66.5710μs | 37.2299μs | 26.8601 KOps/s | 27.7523 KOps/s | $\color{#d91a1a}-3.21\%$ |
| test_step_mdp_speed[False-True-True-True-False] | 45.2420μs | 23.5690μs | 42.4285 KOps/s | 43.5393 KOps/s | $\color{#d91a1a}-2.55\%$ |
| test_step_mdp_speed[False-True-True-False-True] | 57.5810μs | 24.7070μs | 40.4744 KOps/s | 40.9311 KOps/s | $\color{#d91a1a}-1.12\%$ |
| test_step_mdp_speed[False-True-True-False-False] | 38.3610μs | 15.2604μs | 65.5291 KOps/s | 68.3584 KOps/s | $\color{#d91a1a}-4.14\%$ |
| test_step_mdp_speed[False-True-False-True-True] | 69.9120μs | 39.0560μs | 25.6043 KOps/s | 26.7104 KOps/s | $\color{#d91a1a}-4.14\%$ |
| test_step_mdp_speed[False-True-False-True-False] | 72.4810μs | 25.8221μs | 38.7265 KOps/s | 40.1077 KOps/s | $\color{#d91a1a}-3.44\%$ |
| test_step_mdp_speed[False-True-False-False-True] | 49.7510μs | 27.0747μs | 36.9348 KOps/s | 37.8481 KOps/s | $\color{#d91a1a}-2.41\%$ |
| test_step_mdp_speed[False-True-False-False-False] | 38.3400μs | 17.0688μs | 58.5864 KOps/s | 59.7196 KOps/s | $\color{#d91a1a}-1.90\%$ |
| test_step_mdp_speed[False-False-True-True-True] | 74.2420μs | 41.0550μs | 24.3576 KOps/s | 25.2172 KOps/s | $\color{#d91a1a}-3.41\%$ |
| test_step_mdp_speed[False-False-True-True-False] | 58.8810μs | 27.9079μs | 35.8321 KOps/s | 37.8629 KOps/s | $\textbf{\color{#d91a1a}-5.36\%}$ |
| test_step_mdp_speed[False-False-True-False-True] | 50.6310μs | 27.0769μs | 36.9319 KOps/s | 37.9759 KOps/s | $\color{#d91a1a}-2.75\%$ |
| test_step_mdp_speed[False-False-True-False-False] | 37.5110μs | 16.9470μs | 59.0075 KOps/s | 60.4367 KOps/s | $\color{#d91a1a}-2.36\%$ |
| test_step_mdp_speed[False-False-False-True-True] | 57.3310μs | 42.8373μs | 23.3441 KOps/s | 23.8601 KOps/s | $\color{#d91a1a}-2.16\%$ |
| test_step_mdp_speed[False-False-False-True-False] | 86.5610μs | 29.5056μs | 33.8918 KOps/s | 34.9838 KOps/s | $\color{#d91a1a}-3.12\%$ |
| test_step_mdp_speed[False-False-False-False-True] | 52.3310μs | 28.9052μs | 34.5959 KOps/s | 36.0525 KOps/s | $\color{#d91a1a}-4.04\%$ |
| test_step_mdp_speed[False-False-False-False-False] | 52.2310μs | 18.8734μs | 52.9846 KOps/s | 54.4239 KOps/s | $\color{#d91a1a}-2.64\%$ |
| test_values[generalized_advantage_estimate-True-True] | 24.9432ms | 24.3890ms | 41.0020 Ops/s | 40.8966 Ops/s | $\color{#35bf28}+0.26\%$ |
| test_values[vec_generalized_advantage_estimate-True-True] | 81.3088ms | 3.1877ms | 313.7053 Ops/s | 310.3363 Ops/s | $\color{#35bf28}+1.09\%$ |
| test_values[td0_return_estimate-False-False] | 93.1420μs | 62.7925μs | 15.9255 KOps/s | 15.8128 KOps/s | $\color{#35bf28}+0.71\%$ |
| test_values[td1_return_estimate-False-False] | 52.0306ms | 51.7912ms | 19.3083 Ops/s | 19.2545 Ops/s | $\color{#35bf28}+0.28\%$ |
| test_values[vec_td1_return_estimate-False-False] | 2.0348ms | 1.7514ms | 570.9680 Ops/s | 570.4587 Ops/s | $\color{#35bf28}+0.09\%$ |
| test_values[td_lambda_return_estimate-True-False] | 82.6412ms | 82.3681ms | 12.1406 Ops/s | 11.9972 Ops/s | $\color{#35bf28}+1.20\%$ |
| test_values[vec_td_lambda_return_estimate-True-False] | 2.1010ms | 1.7519ms | 570.8007 Ops/s | 570.8901 Ops/s | $\color{#d91a1a}-0.02\%$ |
| test_gae_speed[generalized_advantage_estimate-False-1-512] | 23.1828ms | 22.8455ms | 43.7723 Ops/s | 43.0891 Ops/s | $\color{#35bf28}+1.59\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.8846ms | 0.6863ms | 1.4572 KOps/s | 1.4630 KOps/s | $\color{#d91a1a}-0.40\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7147ms | 0.6386ms | 1.5660 KOps/s | 1.5642 KOps/s | $\color{#35bf28}+0.12\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.4677ms | 1.4410ms | 693.9595 Ops/s | 693.5377 Ops/s | $\color{#35bf28}+0.06\%$ |
| test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 0.9474ms | 0.6565ms | 1.5231 KOps/s | 1.5231 KOps/s | $-0.00\%$ |
| test_dqn_speed | 1.6727ms | 1.4042ms | 712.1434 Ops/s | 682.7578 Ops/s | $\color{#35bf28}+4.30\%$ |
| test_ddpg_speed | 3.8535ms | 2.9437ms | 339.7084 Ops/s | 336.5861 Ops/s | $\color{#35bf28}+0.93\%$ |
| test_sac_speed | 8.6171ms | 8.4069ms | 118.9501 Ops/s | 118.7136 Ops/s | $\color{#35bf28}+0.20\%$ |
| test_redq_speed | 11.1712ms | 10.2502ms | 97.5595 Ops/s | 97.5528 Ops/s | $+0.01\%$ |
| test_redq_deprec_speed | 12.2391ms | 11.4342ms | 87.4570 Ops/s | 86.5809 Ops/s | $\color{#35bf28}+1.01\%$ |
| test_td3_speed | 8.6031ms | 8.2619ms | 121.0373 Ops/s | 119.7637 Ops/s | $\color{#35bf28}+1.06\%$ |
| test_cql_speed | 26.8561ms | 26.0200ms | 38.4320 Ops/s | 38.4502 Ops/s | $\color{#d91a1a}-0.05\%$ |
| test_a2c_speed | 6.0876ms | 5.7473ms | 173.9944 Ops/s | 173.1086 Ops/s | $\color{#35bf28}+0.51\%$ |
| test_ppo_speed | 6.2878ms | 6.0426ms | 165.4914 Ops/s | 166.6360 Ops/s | $\color{#d91a1a}-0.69\%$ |
| test_reinforce_speed | 7.2521ms | 4.6839ms | 213.4959 Ops/s | 213.4226 Ops/s | $\color{#35bf28}+0.03\%$ |
| test_iql_speed | 20.2532ms | 19.5587ms | 51.1281 Ops/s | 50.5862 Ops/s | $\color{#35bf28}+1.07\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 2.9658ms | 2.7937ms | 357.9478 Ops/s | 362.4385 Ops/s | $\color{#d91a1a}-1.24\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.2971ms | 0.6034ms | 1.6573 KOps/s | 1.6860 KOps/s | $\color{#d91a1a}-1.70\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7969ms | 0.5778ms | 1.7308 KOps/s | 1.7534 KOps/s | $\color{#d91a1a}-1.29\%$ |
| test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.0929ms | 2.8151ms | 355.2263 Ops/s | 361.4713 Ops/s | $\color{#d91a1a}-1.73\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 1.2338ms | 0.5951ms | 1.6805 KOps/s | 1.7065 KOps/s | $\color{#d91a1a}-1.52\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7444ms | 0.5685ms | 1.7589 KOps/s | 1.7730 KOps/s | $\color{#d91a1a}-0.80\%$ |
| test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] | 2.2894ms | 1.6283ms | 614.1227 Ops/s | 658.9169 Ops/s | $\textbf{\color{#d91a1a}-6.80\%}$ |
| test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] | 5.7911ms | 1.4894ms | 671.4318 Ops/s | 684.0527 Ops/s | $\color{#d91a1a}-1.85\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.0568ms | 2.9129ms | 343.2961 Ops/s | 348.0129 Ops/s | $\color{#d91a1a}-1.36\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.6102ms | 0.7248ms | 1.3797 KOps/s | 1.3874 KOps/s | $\color{#d91a1a}-0.56\%$ |
| test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8737ms | 0.6953ms | 1.4383 KOps/s | 1.4398 KOps/s | $\color{#d91a1a}-0.11\%$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 2.9696ms | 2.7935ms | 357.9722 Ops/s | 363.5777 Ops/s | $\color{#d91a1a}-1.54\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 1.3050ms | 0.5935ms | 1.6849 KOps/s | 1.6795 KOps/s | $\color{#35bf28}+0.32\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 0.7084ms | 0.5697ms | 1.7554 KOps/s | 1.7504 KOps/s | $\color{#35bf28}+0.28\%$ |
| test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.1107ms | 2.8069ms | 356.2629 Ops/s | 362.5204 Ops/s | $\color{#d91a1a}-1.73\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 0.7318ms | 0.5855ms | 1.7079 KOps/s | 1.6962 KOps/s | $\color{#35bf28}+0.69\%$ |
| test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 0.7356ms | 0.5616ms | 1.7807 KOps/s | 1.7440 KOps/s | $\color{#35bf28}+2.10\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.0995ms | 2.9287ms | 341.4449 Ops/s | 346.2766 Ops/s | $\color{#d91a1a}-1.40\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 1.4157ms | 0.7200ms | 1.3889 KOps/s | 1.3825 KOps/s | $\color{#35bf28}+0.46\%$ |
| test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 0.8529ms | 0.6904ms | 1.4485 KOps/s | 1.4283 KOps/s | $\color{#35bf28}+1.41\%$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1288s | 9.5372ms | 104.8527 Ops/s | 104.5486 Ops/s | $\color{#35bf28}+0.29\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 17.1109ms | 14.8548ms | 67.3184 Ops/s | 66.9932 Ops/s | $\color{#35bf28}+0.49\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 2.4128ms | 1.2069ms | 828.5462 Ops/s | 779.4136 Ops/s | $\textbf{\color{#35bf28}+6.30\%}$ |
| test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1167s | 7.0232ms | 142.3856 Ops/s | 141.9782 Ops/s | $\color{#35bf28}+0.29\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 17.2377ms | 14.9455ms | 66.9096 Ops/s | 66.9749 Ops/s | $\color{#d91a1a}-0.10\%$ |
| test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 2.1908ms | 1.1919ms | 839.0186 Ops/s | 719.7541 Ops/s | $\textbf{\color{#35bf28}+16.57\%}$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1163s | 7.3887ms | 135.3411 Ops/s | 134.6452 Ops/s | $\color{#35bf28}+0.52\%$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 17.3712ms | 15.2221ms | 65.6941 Ops/s | 65.4607 Ops/s | $\color{#35bf28}+0.36\%$ |
| test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 2.5876ms | 1.5330ms | 652.2947 Ops/s | 619.9342 Ops/s | $\textbf{\color{#35bf28}+5.22\%}$ |