v0 param server (using collectives not object store)

[mikaylagawarecki]

Stack from ghstack (oldest at bottom):

• -> #2865

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2865

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:x: 7 New Failures, 1 Cancelled Job, 1 Unrelated Failure

As of commit 32c10d7a93b199658e7755e2dabdf8a57114ec30 with merge base 04d70c1e47855763b83b51c6339a13a29fb214af ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh) Process completed with exit code 1.
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh) Process completed with exit code 1.
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh) RuntimeError: Command docker exec -t 4dbc541ccd54ef94d84a3cc0540d7a13ee841f3b02373a45eb358c29eb1fcaee /exec failed with exit code 1
• Lint / python-source-and-configs / linux-job (gh) torchrl/collectors/vllm_weight_update.py:191:9: F401 'transformers.AutoModel' imported but unused
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh) test/test_env.py::TestNonTensorEnv::test_parallel[False-True]
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh) RuntimeError: Command docker exec -t 7a0121bec6daf52cfb9989f88871ca4d4764cf10fe4824304174f3bea5425632 /exec failed with exit code 1
• Unit-tests on Linux / tests-optdeps (3.11, 12.8) / linux-job (gh) test/test_cost.py::TestPPO4LLMs::test_hf[False]

CANCELLED JOB - The following job was cancelled. Please retry:

• LLM Tests on Linux / unittests (3.9, 12.8) / linux-job (gh) ##[error]The operation was canceled.

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh) (trunk failure) test/test_transforms.py::TestTimer::test_transform_env

This comment was automatically generated by Dr. CI and updates every 15 minutes.