[Example] Comprehensive dataset rendering examples

[vmoens]

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2141

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:x: 8 New Failures, 2 Unrelated Failures

As of commit 739eac6c01563886b5fc48a960d85fe8b2a5dda4 with merge base f613eefb0f44b6a9ccd79f82703bb309aa165009 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh) Process completed with exit code 4.
• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh) RuntimeError: Command docker exec -t d47edf0cd5a4e9b63c1f63ad6b95fc831e289f134eb4c501e79def9bae64acb3 /exec failed with exit code 1
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh) RuntimeError: Command docker exec -t 488abbc22adc2ad46a290102eb07f12ede71815df9f1e3e68c4ce0ff5a1e22e6 /exec failed with exit code 1
• Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job (gh) RuntimeError: Command docker exec -t ace67acce4592ebab5151a232e4732ef4f469fe67a56b80e1062aa870271ba70 /exec failed with exit code 1
• Lint / python-source-and-configs / linux-job (gh) torchrl/envs/libs/jumanji.py:12:1: F401 'torchvision.transforms.v2.functional' imported but unused
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh) RuntimeError: Command docker exec -t f73dd8f0d5b235d032150c69c75a53ea6ed99bc5f5b7cce989c61f067f7d05f7 /exec failed with exit code 1
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh) RuntimeError: Command docker exec -t 467ae0239552ce7ea887fde2899c9061f6a515a7f6e6380bde16362028e2d59f /exec failed with exit code 1
• Unit-tests on Windows / unittests-cpu / windows-job (gh) ModuleNotFoundError: No module named 'torchvision'

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh) (trunk failure) ##[error]The operation was canceled.
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh) (trunk failure) ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions[bot]]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1170s	0.1159s	8.6246 Ops/s	8.5174 Ops/s	$\color{#35bf28}+1.26\%$
test_sync	99.0960ms	97.5079ms	10.2556 Ops/s	9.7162 Ops/s	$\textbf{\color{#35bf28}+5.55\%}$
test_async	0.2709s	97.4224ms	10.2646 Ops/s	10.3690 Ops/s	$\color{#d91a1a}-1.01\%$
test_single_pixels	0.1257s	0.1256s	7.9642 Ops/s	7.7250 Ops/s	$\color{#35bf28}+3.10\%$
test_sync_pixels	83.4182ms	79.0104ms	12.6566 Ops/s	11.8392 Ops/s	$\textbf{\color{#35bf28}+6.90\%}$
test_async_pixels	0.1594s	79.2651ms	12.6159 Ops/s	14.4170 Ops/s	$\textbf{\color{#d91a1a}-12.49\%}$
test_simple	0.7894s	0.7887s	1.2679 Ops/s	1.2315 Ops/s	$\color{#35bf28}+2.95\%$
test_transformed	1.1345s	1.0638s	0.9400 Ops/s	0.9354 Ops/s	$\color{#35bf28}+0.49\%$
test_serial	2.5046s	2.4335s	0.4109 Ops/s	0.4078 Ops/s	$\color{#35bf28}+0.78\%$
test_parallel	2.4176s	2.3460s	0.4263 Ops/s	0.4252 Ops/s	$\color{#35bf28}+0.25\%$
test_step_mdp_speed[True-True-True-True-True]	0.1064ms	32.4380μs	30.8280 KOps/s	29.7107 KOps/s	$\color{#35bf28}+3.76\%$
test_step_mdp_speed[True-True-True-True-False]	0.1237ms	19.5323μs	51.1972 KOps/s	49.3577 KOps/s	$\color{#35bf28}+3.73\%$
test_step_mdp_speed[True-True-True-False-True]	90.6810μs	18.7669μs	53.2852 KOps/s	50.7808 KOps/s	$\color{#35bf28}+4.93\%$
test_step_mdp_speed[True-True-True-False-False]	30.1100μs	11.2464μs	88.9174 KOps/s	87.5264 KOps/s	$\color{#35bf28}+1.59\%$
test_step_mdp_speed[True-True-False-True-True]	59.5600μs	34.7016μs	28.8172 KOps/s	27.7614 KOps/s	$\color{#35bf28}+3.80\%$
test_step_mdp_speed[True-True-False-True-False]	46.6610μs	21.4262μs	46.6719 KOps/s	44.6534 KOps/s	$\color{#35bf28}+4.52\%$
test_step_mdp_speed[True-True-False-False-True]	50.6210μs	20.9149μs	47.8128 KOps/s	47.6979 KOps/s	$\color{#35bf28}+0.24\%$
test_step_mdp_speed[True-True-False-False-False]	32.7100μs	13.1431μs	76.0856 KOps/s	75.9784 KOps/s	$\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-False-True-True-True]	61.0900μs	36.4040μs	27.4695 KOps/s	26.2877 KOps/s	$\color{#35bf28}+4.50\%$
test_step_mdp_speed[True-False-True-True-False]	51.5710μs	23.5960μs	42.3801 KOps/s	40.6707 KOps/s	$\color{#35bf28}+4.20\%$
test_step_mdp_speed[True-False-True-False-True]	41.5900μs	20.3799μs	49.0680 KOps/s	46.6760 KOps/s	$\textbf{\color{#35bf28}+5.12\%}$
test_step_mdp_speed[True-False-True-False-False]	33.9200μs	13.1435μs	76.0832 KOps/s	75.1566 KOps/s	$\color{#35bf28}+1.23\%$
test_step_mdp_speed[True-False-False-True-True]	62.6510μs	37.5701μs	26.6169 KOps/s	25.2507 KOps/s	$\textbf{\color{#35bf28}+5.41\%}$
test_step_mdp_speed[True-False-False-True-False]	41.5700μs	24.7918μs	40.3359 KOps/s	38.1581 KOps/s	$\textbf{\color{#35bf28}+5.71\%}$
test_step_mdp_speed[True-False-False-False-True]	42.7910μs	21.7247μs	46.0306 KOps/s	43.7932 KOps/s	$\textbf{\color{#35bf28}+5.11\%}$
test_step_mdp_speed[True-False-False-False-False]	0.1069ms	14.6664μs	68.1830 KOps/s	65.1782 KOps/s	$\color{#35bf28}+4.61\%$
test_step_mdp_speed[False-True-True-True-True]	58.2410μs	35.8097μs	27.9254 KOps/s	26.5517 KOps/s	$\textbf{\color{#35bf28}+5.17\%}$
test_step_mdp_speed[False-True-True-True-False]	84.7200μs	22.7840μs	43.8904 KOps/s	41.1994 KOps/s	$\textbf{\color{#35bf28}+6.53\%}$
test_step_mdp_speed[False-True-True-False-True]	0.1282ms	24.2608μs	41.2187 KOps/s	39.3858 KOps/s	$\color{#35bf28}+4.65\%$
test_step_mdp_speed[False-True-True-False-False]	41.5710μs	14.7468μs	67.8114 KOps/s	65.4272 KOps/s	$\color{#35bf28}+3.64\%$
test_step_mdp_speed[False-True-False-True-True]	56.4900μs	37.8820μs	26.3978 KOps/s	25.5729 KOps/s	$\color{#35bf28}+3.23\%$
test_step_mdp_speed[False-True-False-True-False]	47.5800μs	25.3483μs	39.4503 KOps/s	37.6466 KOps/s	$\color{#35bf28}+4.79\%$
test_step_mdp_speed[False-True-False-False-True]	66.8810μs	26.1663μs	38.2171 KOps/s	36.3780 KOps/s	$\textbf{\color{#35bf28}+5.06\%}$
test_step_mdp_speed[False-True-False-False-False]	31.2300μs	16.2982μs	61.3564 KOps/s	58.0893 KOps/s	$\textbf{\color{#35bf28}+5.62\%}$
test_step_mdp_speed[False-False-True-True-True]	56.5400μs	39.3035μs	25.4430 KOps/s	24.1983 KOps/s	$\textbf{\color{#35bf28}+5.14\%}$
test_step_mdp_speed[False-False-True-True-False]	43.7300μs	26.4066μs	37.8693 KOps/s	35.4968 KOps/s	$\textbf{\color{#35bf28}+6.68\%}$
test_step_mdp_speed[False-False-True-False-True]	56.4110μs	26.0817μs	38.3411 KOps/s	36.3104 KOps/s	$\textbf{\color{#35bf28}+5.59\%}$
test_step_mdp_speed[False-False-True-False-False]	36.4810μs	16.5882μs	60.2838 KOps/s	57.9273 KOps/s	$\color{#35bf28}+4.07\%$
test_step_mdp_speed[False-False-False-True-True]	0.1825ms	42.7206μs	23.4079 KOps/s	23.3529 KOps/s	$\color{#35bf28}+0.24\%$
test_step_mdp_speed[False-False-False-True-False]	0.1217ms	28.8071μs	34.7136 KOps/s	33.5451 KOps/s	$\color{#35bf28}+3.48\%$
test_step_mdp_speed[False-False-False-False-True]	52.0610μs	28.0056μs	35.7071 KOps/s	34.9452 KOps/s	$\color{#35bf28}+2.18\%$
test_step_mdp_speed[False-False-False-False-False]	0.1971ms	18.4241μs	54.2768 KOps/s	52.8411 KOps/s	$\color{#35bf28}+2.72\%$
test_values[generalized_advantage_estimate-True-True]	24.4727ms	23.3542ms	42.8188 Ops/s	42.6754 Ops/s	$\color{#35bf28}+0.34\%$
test_values[vec_generalized_advantage_estimate-True-True]	92.2848ms	2.7307ms	366.2121 Ops/s	375.2731 Ops/s	$\color{#d91a1a}-2.41\%$
test_values[td0_return_estimate-False-False]	90.8310μs	65.0734μs	15.3673 KOps/s	15.5863 KOps/s	$\color{#d91a1a}-1.41\%$
test_values[td1_return_estimate-False-False]	54.2131ms	52.7562ms	18.9551 Ops/s	18.9347 Ops/s	$\color{#35bf28}+0.11\%$
test_values[vec_td1_return_estimate-False-False]	1.4111ms	1.0718ms	933.0153 Ops/s	927.6805 Ops/s	$\color{#35bf28}+0.58\%$
test_values[td_lambda_return_estimate-True-False]	84.6032ms	83.2762ms	12.0082 Ops/s	11.7166 Ops/s	$\color{#35bf28}+2.49\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.3990ms	1.0671ms	937.1552 Ops/s	928.6635 Ops/s	$\color{#35bf28}+0.91\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	24.2472ms	23.5024ms	42.5488 Ops/s	42.7474 Ops/s	$\color{#d91a1a}-0.46\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9163ms	0.6953ms	1.4383 KOps/s	1.4234 KOps/s	$\color{#35bf28}+1.05\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.8123ms	0.6508ms	1.5366 KOps/s	1.4928 KOps/s	$\color{#35bf28}+2.93\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.6198ms	1.4499ms	689.7118 Ops/s	687.1193 Ops/s	$\color{#35bf28}+0.38\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.8418ms	0.6662ms	1.5011 KOps/s	1.4615 KOps/s	$\color{#35bf28}+2.72\%$
test_dqn_speed	7.7513ms	1.4397ms	694.5680 Ops/s	702.1763 Ops/s	$\color{#d91a1a}-1.08\%$
test_ddpg_speed	3.1641ms	2.9184ms	342.6496 Ops/s	344.9976 Ops/s	$\color{#d91a1a}-0.68\%$
test_sac_speed	8.7112ms	8.3793ms	119.3412 Ops/s	119.5778 Ops/s	$\color{#d91a1a}-0.20\%$
test_redq_speed	12.8073ms	10.5608ms	94.6900 Ops/s	93.4101 Ops/s	$\color{#35bf28}+1.37\%$
test_redq_deprec_speed	13.0302ms	11.5029ms	86.9345 Ops/s	86.1911 Ops/s	$\color{#35bf28}+0.86\%$
test_td3_speed	8.5364ms	8.3265ms	120.0988 Ops/s	120.0865 Ops/s	$\color{#35bf28}+0.01\%$
test_cql_speed	27.1495ms	25.7460ms	38.8409 Ops/s	38.8829 Ops/s	$\color{#d91a1a}-0.11\%$
test_a2c_speed	5.8376ms	5.5744ms	179.3930 Ops/s	174.9330 Ops/s	$\color{#35bf28}+2.55\%$
test_ppo_speed	7.1036ms	6.0009ms	166.6403 Ops/s	164.4680 Ops/s	$\color{#35bf28}+1.32\%$
test_reinforce_speed	5.4270ms	4.5945ms	217.6539 Ops/s	214.5364 Ops/s	$\color{#35bf28}+1.45\%$
test_iql_speed	20.2648ms	19.6420ms	50.9113 Ops/s	50.1146 Ops/s	$\color{#35bf28}+1.59\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.9081ms	4.7635ms	209.9284 Ops/s	209.2085 Ops/s	$\color{#35bf28}+0.34\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.4299ms	0.5949ms	1.6809 KOps/s	1.6809 KOps/s	$-0.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7622ms	0.5695ms	1.7558 KOps/s	1.7435 KOps/s	$\color{#35bf28}+0.71\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.5265ms	4.6993ms	212.7984 Ops/s	210.2700 Ops/s	$\color{#35bf28}+1.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.3018ms	0.5894ms	1.6967 KOps/s	1.7071 KOps/s	$\color{#d91a1a}-0.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7405ms	0.5642ms	1.7723 KOps/s	1.4986 KOps/s	$\textbf{\color{#35bf28}+18.27\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	5.2799ms	2.0556ms	486.4802 Ops/s	484.6462 Ops/s	$\color{#35bf28}+0.38\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.1100ms	1.9526ms	512.1293 Ops/s	509.5730 Ops/s	$\color{#35bf28}+0.50\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.1717ms	4.8933ms	204.3601 Ops/s	203.0581 Ops/s	$\color{#35bf28}+0.64\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9873ms	0.7248ms	1.3798 KOps/s	1.3967 KOps/s	$\color{#d91a1a}-1.21\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.9508ms	0.7007ms	1.4271 KOps/s	1.4474 KOps/s	$\color{#d91a1a}-1.40\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.9704ms	4.7629ms	209.9580 Ops/s	208.3933 Ops/s	$\color{#35bf28}+0.75\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7913ms	0.5942ms	1.6829 KOps/s	1.6771 KOps/s	$\color{#35bf28}+0.35\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7528ms	0.5721ms	1.7481 KOps/s	1.7369 KOps/s	$\color{#35bf28}+0.64\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.0030ms	4.7392ms	211.0064 Ops/s	210.4316 Ops/s	$\color{#35bf28}+0.27\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1359s	0.7860ms	1.2723 KOps/s	1.6967 KOps/s	$\textbf{\color{#d91a1a}-25.01\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7276ms	0.5633ms	1.7753 KOps/s	1.7672 KOps/s	$\color{#35bf28}+0.46\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.0727ms	4.8808ms	204.8838 Ops/s	202.1973 Ops/s	$\color{#35bf28}+1.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9227ms	0.7210ms	1.3870 KOps/s	1.3954 KOps/s	$\color{#d91a1a}-0.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8791ms	0.6973ms	1.4342 KOps/s	1.4363 KOps/s	$\color{#d91a1a}-0.15\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1341s	7.7214ms	129.5098 Ops/s	134.7574 Ops/s	$\color{#d91a1a}-3.89\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	18.3377ms	15.9248ms	62.7952 Ops/s	62.4851 Ops/s	$\color{#35bf28}+0.50\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.2647ms	1.2766ms	783.3240 Ops/s	760.5122 Ops/s	$\color{#35bf28}+3.00\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1266s	9.8572ms	101.4487 Ops/s	140.5986 Ops/s	$\textbf{\color{#d91a1a}-27.85\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	18.4528ms	16.0723ms	62.2189 Ops/s	62.4100 Ops/s	$\color{#d91a1a}-0.31\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.3703ms	1.3026ms	767.7066 Ops/s	699.1779 Ops/s	$\textbf{\color{#35bf28}+9.80\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1264s	7.6828ms	130.1613 Ops/s	108.1123 Ops/s	$\textbf{\color{#35bf28}+20.39\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	19.0859ms	16.2393ms	61.5789 Ops/s	61.9049 Ops/s	$\color{#d91a1a}-0.53\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.5234ms	1.5721ms	636.1032 Ops/s	677.9601 Ops/s	$\textbf{\color{#d91a1a}-6.17\%}$