Gym
Gym copied to clipboard
Published
6 days ago
•
NVIDIA-NeMo
NVIDIA-NeMo
python flag for colab venv installation
need to set uv pip install python flag in colab environments when launching servers
usage: ng_run "+config_paths=[...]" +uv_pip_set_python=true
defaults to false
For https://github.com/NVIDIA-NeMo/Gym/issues/370
Needed for notebook here: https://docs.unsloth.ai/models/nemotron-3#reinforcement-learning--nemo-gym
This pull request requires additional validation before any workflows can run on NVIDIA's runners.
Pull request vetters can view their responsibilities here.
Contributors can view more details about this message here.
![copy-pr-bot[bot] avatar](https://avatars.githubusercontent.com/in/354886?v=4 "Published by a pub.dev verified publisher"){kind=small}
Colab error without this feature:
(Gym) /content/Gym# ng_run "+config_paths=[resources_servers/reasoning_gym/configs/resources_only.yaml]"
Starting Ray cluster...
2025-12-19 19:16:42,045 INFO worker.py:2004 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8265
/content/Gym/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
Started Ray cluster at 172.28.0.12:54843
Waiting for head server to spin up
Head server is not up yet (status `connection_error`). Sleeping 3s
warning: The `--system` flag has no effect, a system Python interpreter is always used in `uv venv`
INFO: Started server process [893]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:11000 (Press CTRL+C to quit)
Using CPython 3.12.12 interpreter at: /usr/bin/python3.12
Creating virtual environment with seed packages at: .venv
+ pip==25.3
Activate with: source .venv/bin/activate
Using Python 3.12.12 environment at: /usr
⠸ reasoning-gym==0.1.24
INFO: 127.0.0.1:35006 - "GET / HTTP/1.1" 404 Not Found
Waiting for servers to spin up
Checking for HTTP server statuses (you should see some HTTP requests to `/` that may 404. This is expected.)
0 / 1 servers ready (0 timed out, 0 connection errored, 0 had unknown errors).
Waiting for servers to spin up: ['reasoning_gym']
Sleeping 3s...
Resolved 186 packages in 4.20s
Building accumulation-tree==0.6.4
Building pyudorandom==1.0.0
Building nemo-gym @ file:///content/Gym
Building pycosat==0.6.6
Building cellpylib==2.4.0
Building zss==1.2.0
Building accumulation-tree==0.6.4
Built pyudorandom==1.0.0
Building nemo-gym @ file:///content/Gym
Building accumulation-tree==0.6.4
Built pyudorandom==1.0.0
Building nemo-gym @ file:///content/Gym
Building pycosat==0.6.6
Building accumulation-tree==0.6.4
Built pyudorandom==1.0.0
Building nemo-gym @ file:///content/Gym
Building pycosat==0.6.6
Built cellpylib==2.4.0
Built zss==1.2.0
Building accumulation-tree==0.6.4
Built pyudorandom==1.0.0
Built nemo-gym @ file:///content/Gym
Building pycosat==0.6.6
Built cellpylib==2.4.0
Built zss==1.2.0
Building accumulation-tree==0.6.4
Built pyudorandom==1.0.0
Built nemo-gym @ file:///content/Gym
Building pycosat==0.6.6
Built cellpylib==2.4.0
Built zss==1.2.0
Building accumulation-tree==0.6.4
Built pyudorandom==1.0.0
Built nemo-gym @ file:///content/Gym
Building pycosat==0.6.6
Built cellpylib==2.4.0
Built zss==1.2.0
Building accumulation-tree==0.6.4
Built pyudorandom==1.0.0
Built nemo-gym @ file:///content/Gym
Built pycosat==0.6.6
Built cellpylib==2.4.0
Built zss==1.2.0
Building accumulation-tree==0.6.4
Built pyudorandom==1.0.0
Built nemo-gym @ file:///content/Gym
Built pycosat==0.6.6
Built cellpylib==2.4.0
Built zss==1.2.0
Built accumulation-tree==0.6.4
Built pyudorandom==1.0.0
Built nemo-gym @ file:///content/Gym
Built pycosat==0.6.6
Built cellpylib==2.4.0
Built zss==1.2.0
Prepared 54 packages in 21.59s
Uninstalled 1 package in 51ms
Installed 54 packages in 440ms
+ accumulation-tree==0.6.4
+ aiohttp-cors==0.8.1
+ arckit==0.1.0
+ asttokens==2.4.1
+ bfi==1.0.4
+ cellpylib==2.4.0
+ cfgv==3.5.0
+ colorful==0.5.8
+ coverage==7.13.0
+ databricks-sdk==0.76.0
+ devtools==0.12.2
+ distlib==0.4.0
+ docker==7.1.0
+ drawsvg==2.4.0
+ execnet==2.1.2
+ executing==2.2.1
+ flask-cors==6.0.2
+ graphene==3.4.3
+ graphql-core==3.2.7
+ graphql-relay==3.2.0
+ gunicorn==23.0.0
+ huey==2.5.5
+ hydra-core==1.3.2
+ identify==2.6.15
+ librt==0.7.4
+ magiccube==0.3.0
+ mlflow==3.7.0
+ mlflow-skinny==3.7.0
+ mlflow-tracing==3.7.0
+ mypy==1.19.1
+ mypy-extensions==1.1.0
+ nemo-gym==0.2.0rc0 (from file:///content/Gym)
+ nodeenv==1.9.1
- openai==2.12.0
+ openai==2.6.1
+ opencensus==0.11.4
+ opencensus-context==0.1.3
+ opentelemetry-exporter-prometheus==0.58b0
+ pathspec==0.12.1
+ pre-commit==4.5.1
+ py-spy==0.4.1
+ pycosat==0.6.6
+ pyfiglet==1.0.2
+ pytest-asyncio==1.3.0
+ pytest-cov==7.0.0
+ pytest-xdist==3.8.0
+ pyudorandom==1.0.0
+ ray==2.50.1
+ reasoning-gym==0.1.24
+ requests-mock==1.12.1
+ tdigest==0.5.2.2
+ uvloop==0.22.1
+ virtualenv==20.35.4
+ yappi==1.7.3
+ zss==1.2.0
Traceback (most recent call last):
File "/content/Gym/resources_servers/reasoning_gym/app.py", line 18, in <module>
import reasoning_gym
ModuleNotFoundError: No module named 'reasoning_gym'
Traceback (most recent call last):
File "/content/Gym/.venv/bin/ng_run", line 10, in <module>
sys.exit(run())
^^^^^
File "/content/Gym/nemo_gym/cli.py", line 393, in run
rh.start(global_config_dict_parser_config)
File "/content/Gym/nemo_gym/cli.py", line 250, in start
self.wait_for_spinup()
File "/content/Gym/nemo_gym/cli.py", line 298, in wait_for_spinup
self.poll()
File "/content/Gym/nemo_gym/cli.py", line 291, in poll
raise RuntimeError(print_str)
RuntimeError: Process `reasoning_gym` finished unexpectedly!
(Gym) /content/Gym#
resolved issue using this feature:
(Gym) /content/Gym# git checkout remotes/origin/cmunley1/colab
Note: switching to 'remotes/origin/cmunley1/colab'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 2100bf1 gate behind flag
(Gym) /content/Gym# ng_run "+config_paths=[resources_servers/reasoning_gym/configs/resources_only.yaml]" +uv_pip_set_python=true
Starting Ray cluster...
2025-12-19 19:18:27,854 INFO worker.py:2004 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8265
/content/Gym/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
Started Ray cluster at 172.28.0.12:60910
Waiting for head server to spin up
Head server is not up yet (status `connection_error`). Sleeping 3s
warning: The `--system` flag has no effect, a system Python interpreter is always used in `uv venv`
INFO: Started server process [1995]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:11000 (Press CTRL+C to quit)
Using CPython 3.12.12 interpreter at: /usr/bin/python3.12
Creating virtual environment with seed packages at: .venv
Activate with: source .venv/bin/activate
Resolved 186 packages in 2.09s
Building nemo-gym @ file:///content/Gym
⠼ Preparing packages... (10/131)
dill ------------------------------ 30.88 KiB/116.86 KiB
wrapt ------------------------------ 92.18 KiB/118.64 KiB
urllib3 ------------------------------ 63.07 KiB/128.11 KiB
opentelemetry-sdk ------------------------------ 32.00 KiB/129.46 KiB
jinja2 ------------------------------ 92.80 KiB/131.74 KiB
multiprocess ------------------------------ 62.69 KiB/146.76 KiB
Built nemo-gym @ file:///content/Gym
Built antlr4-python3-runtime==4.9.3
⠙ Preparing packages... (121/131)
sympy ------------------------------ 2.95 MiB/6.01 MiB
matplotlib ------------------------------ 7.09 MiB/8.31 MiB
scikit-learn ------------------------------ 6.52 MiB/8.52 MiB
pandas ------------------------------ 7.42 MiB/11.79 MiB
ruff ------------------------------ 7.42 MiB/13.57 MiB
⠧ Preparing packages... (127/131)
pandas ------------------------------ 10.87 MiB/11.79 MiB
gradio ------------------------------ 16.56 MiB/21.92 MiB
scipy ------------------------------ 33.93 MiB/34.02 MiB
pyarrow ------------------------------ 39.73 MiB/45.52 MiB
Checking for HTTP server statuses (you should see some HTTP requests to `/` that may 404. T
Prepared 131 packages in 6.96s
Installed 186 packages in 537ms
+ accumulation-tree==0.6.4
+ aiofiles==24.1.0
+ aiohappyeyeballs==2.6.1
+ aiohttp==3.13.2
+ aiohttp-cors==0.8.1
+ aiosignal==1.4.0
+ alembic==1.17.2
+ annotated-doc==0.0.4
+ annotated-types==0.7.0
+ antlr4-python3-runtime==4.9.3
+ anyio==4.12.0
+ arckit==0.1.0
+ asttokens==2.4.1
+ attrs==25.4.0
+ bfi==1.0.4
+ blinker==1.9.0
+ brotli==1.2.0
+ cachetools==6.2.4
+ cellpylib==2.4.0
+ certifi==2025.11.12
+ cffi==2.0.0
+ cfgv==3.5.0
+ charset-normalizer==3.4.4
+ click==8.3.1
+ cloudpickle==3.1.2
+ colorful==0.5.8
+ contourpy==1.3.3
+ coverage==7.13.0
+ cryptography==46.0.3
+ cycler==0.12.1
+ databricks-sdk==0.76.0
+ datasets==4.4.2
+ devtools==0.12.2
+ dill==0.4.0
+ distlib==0.4.0
+ distro==1.9.0
+ docker==7.1.0
+ drawsvg==2.4.0
+ execnet==2.1.2
+ executing==2.2.1
+ fastapi==0.125.0
+ ffmpy==1.0.0
+ filelock==3.20.1
+ flask==3.1.2
+ flask-cors==6.0.2
+ fonttools==4.61.1
+ frozenlist==1.8.0
+ fsspec==2025.10.0
+ gitdb==4.0.12
+ gitpython==3.1.45
+ google-api-core==2.28.1
+ google-auth==2.45.0
+ googleapis-common-protos==1.72.0
+ gradio==6.2.0
+ gradio-client==2.0.2
+ graphene==3.4.3
+ graphql-core==3.2.7
+ graphql-relay==3.2.0
+ greenlet==3.3.0
+ groovy==0.1.2
+ grpcio==1.76.0
+ gunicorn==23.0.0
+ h11==0.16.0
+ hf-xet==1.2.0
+ httpcore==1.0.9
+ httpx==0.28.1
+ huey==2.5.5
+ huggingface-hub==1.2.3
+ hydra-core==1.3.2
+ identify==2.6.15
+ idna==3.11
+ importlib-metadata==8.7.0
+ iniconfig==2.3.0
+ itsdangerous==2.2.0
+ jinja2==3.1.6
+ jiter==0.12.0
+ joblib==1.5.3
+ jsonschema==4.25.1
+ jsonschema-specifications==2025.9.1
+ kiwisolver==1.4.9
+ librt==0.7.4
+ magiccube==0.3.0
+ mako==1.3.10
+ markdown-it-py==4.0.0
+ markupsafe==3.0.3
+ matplotlib==3.10.8
+ mdurl==0.1.2
+ mlflow==3.7.0
+ mlflow-skinny==3.7.0
+ mlflow-tracing==3.7.0
+ mpmath==1.3.0
+ msgpack==1.1.2
+ multidict==6.7.0
+ multiprocess==0.70.18
+ mypy==1.19.1
+ mypy-extensions==1.1.0
+ nemo-gym==0.2.0rc0 (from file:///content/Gym)
+ nodeenv==1.9.1
+ numpy==2.3.5
+ omegaconf==2.3.0
+ openai==2.6.1
+ opencensus==0.11.4
+ opencensus-context==0.1.3
+ opentelemetry-api==1.39.1
+ opentelemetry-exporter-prometheus==0.60b1
+ opentelemetry-proto==1.39.1
+ opentelemetry-sdk==1.39.1
+ opentelemetry-semantic-conventions==0.60b1
+ orjson==3.11.5
+ packaging==25.0
+ pandas==2.3.3
+ pathspec==0.12.1
+ pillow==12.0.0
+ platformdirs==4.5.1
+ pluggy==1.6.0
+ pre-commit==4.5.1
+ prometheus-client==0.23.1
+ propcache==0.4.1
+ proto-plus==1.27.0
+ protobuf==6.33.2
+ psutil==7.1.3
+ py-spy==0.4.1
+ pyarrow==22.0.0
+ pyasn1==0.6.1
+ pyasn1-modules==0.4.2
+ pycosat==0.6.6
+ pycparser==2.23
+ pydantic==2.12.5
+ pydantic-core==2.41.5
+ pydub==0.25.1
+ pyfiglet==1.0.2
+ pygments==2.19.2
+ pyparsing==3.2.5
+ pytest==9.0.2
+ pytest-asyncio==1.3.0
+ pytest-cov==7.0.0
+ pytest-xdist==3.8.0
+ python-dateutil==2.9.0.post0
+ python-dotenv==1.2.1
+ python-multipart==0.0.21
+ pytz==2025.2
+ pyudorandom==1.0.0
+ pyyaml==6.0.3
+ ray==2.50.1
+ reasoning-gym==0.1.24
+ referencing==0.37.0
+ requests==2.32.5
+ requests-mock==1.12.1
+ rich==14.2.0
+ rpds-py==0.30.0
+ rsa==4.9.1
+ ruff==0.14.10
+ safehttpx==0.1.7
+ scikit-learn==1.8.0
+ scipy==1.16.3
+ semantic-version==2.10.0
+ shellingham==1.5.4
+ six==1.17.0
+ smart-open==7.5.0
+ smmap==5.0.2
+ sniffio==1.3.1
+ sqlalchemy==2.0.45
+ sqlparse==0.5.5
+ starlette==0.50.0
+ sympy==1.14.0
+ tabulate==0.9.0
+ tdigest==0.5.2.2
+ threadpoolctl==3.6.0
+ tomlkit==0.13.3
+ tqdm==4.67.1
+ typer==0.20.1
+ typer-slim==0.20.1
+ typing-extensions==4.15.0
+ typing-inspection==0.4.2
+ tzdata==2025.3
+ urllib3==2.6.2
+ uvicorn==0.38.0
+ uvloop==0.22.1
+ virtualenv==20.35.4
+ werkzeug==3.1.4
+ wrapt==2.0.1
+ xxhash==3.6.0
+ yappi==1.7.3
+ yarl==1.22.0
+ zipp==3.23.0
+ zss==1.2.0
Checking for HTTP server statuses (you should see some HTTP requests to `/` that may 404. This is expected.)
0 / 1 servers ready (0 timed out, 0 connection errored, 0 had unknown errors).
Waiting for servers to spin up: ['reasoning_gym']
Sleeping 3s...
/content/Gym/resources_servers/reasoning_gym/.venv/lib/python3.12/site-packages/cellpylib/ca_functions.py:194: SyntaxWarning: "is" with 'str' literal. Did you mean "=="?
if memoize is "recursive":
/content/Gym/resources_servers/reasoning_gym/.venv/lib/python3.12/site-packages/cellpylib/ca_functions.py:246: SyntaxWarning: "is" with 'str' literal. Did you mean "=="?
if memoize is "recursive":
/content/Gym/resources_servers/reasoning_gym/.venv/lib/python3.12/site-packages/cellpylib/ca_functions2d.py:431: SyntaxWarning: "is" with 'str' literal. Did you mean "=="?
if memoize is "recursive":
/content/Gym/resources_servers/reasoning_gym/.venv/lib/python3.12/site-packages/cellpylib/ca_functions2d.py:501: SyntaxWarning: "is" with 'str' literal. Did you mean "=="?
if memoize is "recursive":
Checking for HTTP server statuses (you should see some HTTP requests to `/` that may 404. This is expected.)
0 / 1 servers ready (0 timed out, 0 connection errored, 0 had unknown errors).
Waiting for servers to spin up: ['reasoning_gym']
Sleeping 3s...
Connecting to Ray cluster at specified address: 172.28.0.12:60910
2025-12-19 19:18:46,429 INFO worker.py:1833 -- Connecting to existing Ray cluster at address: 172.28.0.12:60910...
2025-12-19 19:18:46,441 INFO worker.py:2004 -- Connected to Ray cluster. View the dashboard at http://127.0.0.1:8265
/content/Gym/resources_servers/reasoning_gym/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
Adding a uvicorn logging filter so that the logs aren't spammed with 200 OK messages. This is to help errors pop up better and filter out noise.
INFO: Started server process [2514]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:59761 (Press CTRL+C to quit)
Checking for HTTP server statuses (you should see some HTTP requests to `/` that may 404. This is expected.)
INFO: 127.0.0.1:38332 - "GET / HTTP/1.1" 404 Not Found
All 1 / 1 servers ready! Polling every 60s
####################################################################################################
#
# Server Instances
#
####################################################################################################
[1] reasoning_gym (resources_servers/reasoning_gym)
{
'process_name': 'reasoning_gym',
'server_type': 'resources_servers',
'name': 'reasoning_gym',
'dir_path': '/content/Gym/resources_servers/reasoning_gym',
'entrypoint': 'app.py',
'host': '127.0.0.1',
'port': 59761,
'pid': 2514,
'config_path': 'reasoning_gym',
'url': 'http://127.0.0.1:59761',
}
####################################################################################################
testing ng_run with flag on regular (brev) node looks fine:
(gym-test-colab) ubuntu@brev-64egbpb2a:~/gym-test-colab$ ng_run "+config_paths=[resources_servers/reasoning_gym/configs/reasoning_gym.yaml,responses_api_models/vllm_model/configs/vllm_model.yaml]" +uv_pip_set_python=false
Starting Ray cluster...
2025-12-19 19:11:13,738 INFO worker.py:2004 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8265
/home/ubuntu/gym-test-colab/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
Started Ray cluster at 172.27.16.211:54985
Waiting for head server to spin up
Head server is not up yet (status `connection_error`). Sleeping 3s
INFO: Started server process [3366069]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:11000 (Press CTRL+C to quit)
Using CPython 3.12.12
Creating virtual environment with seed packages at: .venv
Using CPython 3.12.12
Creating virtual environment with seed packages at: .venv
Using CPython 3.12.12
Creating virtual environment with seed packages at: .venv
Activate with: source .venv/bin/activate
Activate with: source .venv/bin/activate
Activate with: source .venv/bin/activate
⠙ nemo-gym==0.2.0rc0 Audited 3 packages in 57ms
Audited 4 packages in 58ms
Resolved 263 packages in 69ms
Built vllm-model @ file:///home/ubuntu/gym-test-colab/responses_api_models/vllm_model
Prepared 1 package in 799ms
Uninstalled 1 package in 0.36ms
Installed 1 package in 2ms
~ vllm-model==0.2.0rc0 (from file:///home/ubuntu/gym-test-colab/responses_api_models/vllm_model)
Connecting to Ray cluster at specified address: 172.27.16.211:54985
2025-12-19 19:11:15,899 INFO worker.py:1833 -- Connecting to existing Ray cluster at address: 172.27.16.211:54985...
2025-12-19 19:11:15,911 INFO worker.py:2004 -- Connected to Ray cluster. View the dashboard at http://127.0.0.1:8265
/home/ubuntu/gym-test-colab/responses_api_agents/simple_agent/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
Adding a uvicorn logging filter so that the logs aren't spammed with 200 OK messages. This is to help errors pop up better and filter out noise.
INFO: Started server process [3377518]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:57685 (Press CTRL+C to quit)
Connecting to Ray cluster at specified address: 172.27.16.211:54985
2025-12-19 19:11:16,612 INFO worker.py:1833 -- Connecting to existing Ray cluster at address: 172.27.16.211:54985...
2025-12-19 19:11:16,625 INFO worker.py:2004 -- Connected to Ray cluster. View the dashboard at http://127.0.0.1:8265
/home/ubuntu/gym-test-colab/resources_servers/reasoning_gym/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
Adding a uvicorn logging filter so that the logs aren't spammed with 200 OK messages. This is to help errors pop up better and filter out noise.
INFO: Started server process [3377516]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:59951 (Press CTRL+C to quit)
Connecting to Ray cluster at specified address: 172.27.16.211:54985
2025-12-19 19:11:16,929 INFO worker.py:1833 -- Connecting to existing Ray cluster at address: 172.27.16.211:54985...
2025-12-19 19:11:16,941 INFO worker.py:2004 -- Connected to Ray cluster. View the dashboard at http://127.0.0.1:8265
/home/ubuntu/gym-test-colab/responses_api_models/vllm_model/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
Adding a uvicorn logging filter so that the logs aren't spammed with 200 OK messages. This is to help errors pop up better and filter out noise.
INFO: Started server process [3377520]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:45093 (Press CTRL+C to quit)
INFO: 127.0.0.1:48754 - "GET / HTTP/1.1" 404 Not Found
Waiting for servers to spin up
Checking for HTTP server statuses (you should see some HTTP requests to `/` that may 404. This is expected.)
INFO: 127.0.0.1:43114 - "GET / HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:42604 - "GET / HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:52144 - "GET / HTTP/1.1" 404 Not Found
All 3 / 3 servers ready! Polling every 60s
####################################################################################################
#
# Server Instances
#
####################################################################################################
[1] reasoning_gym (resources_servers/reasoning_gym)
{
'process_name': 'reasoning_gym',
'server_type': 'resources_servers',
'name': 'reasoning_gym',
'dir_path': '/home/ubuntu/gym-test-colab/resources_servers/reasoning_gym',
'entrypoint': 'app.py',
'host': '127.0.0.1',
'port': 59951,
'pid': 3377516,
'config_path': 'reasoning_gym',
'url': 'http://127.0.0.1:59951',
}
[2] reasoning_gym_simple_agent (responses_api_agents/simple_agent)
{
'process_name': 'reasoning_gym_simple_agent',
'server_type': 'responses_api_agents',
'name': 'simple_agent',
'dir_path': '/home/ubuntu/gym-test-colab/responses_api_agents/simple_agent',
'entrypoint': 'app.py',
'host': '127.0.0.1',
'port': 57685,
'pid': 3377518,
'config_path': 'reasoning_gym_simple_agent',
'url': 'http://127.0.0.1:57685',
}
[3] policy_model (responses_api_models/vllm_model)
{
'process_name': 'policy_model',
'server_type': 'responses_api_models',
'name': 'vllm_model',
'dir_path': '/home/ubuntu/gym-test-colab/responses_api_models/vllm_model',
'entrypoint': 'app.py',
'host': '127.0.0.1',
'port': 45093,
'pid': 3377520,
'config_path': 'policy_model',
'url': 'http://127.0.0.1:45093',
}
####################################################################################################
another test outside of colab:
(gym-test-colab) ubuntu@brev-64egbpb2a:~/gym-test-colab$ ng_run "+config_paths=[resources_servers/instruction_following[2025-12-19 19:33:35,726 E 3584055 3584848] rpc_client.h:201: Failed to connect to GCS within 60 seconds. GCS may have been killed. It's either GCS is terminated by `ray stop` or is killed unexpectedly. If it is killed unexpectedly, see the log file gcs_server.out. https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#logging-directory-structure. The program will terminate.
[2025-12-19 19:33:36,099 E 3584056 3585134] rpc_client.h:201: Failed to connect to GCS within 60 seconds. GCS may have been killed. It's either GCS is terminated by `ray stop` or is killed unexpectedly. If it is killed unexpectedly, see the log file gcs_server.out. https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#logging-directory-structure. The program will terminate.
Starting Ray cluster...
2025-12-19 19:33:39,103 INFO worker.py:2004 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8265
/home/ubuntu/gym-test-colab/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
Started Ray cluster at 172.27.16.211:54033
Waiting for head server to spin up
Head server is not up yet (status `connection_error`). Sleeping 3s
INFO: Started server process [3587461]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:11000 (Press CTRL+C to quit)
Using CPython 3.12.12
Creating virtual environment with seed packages at: .venv
Using CPython 3.12.12
Creating virtual environment with seed packages at: .venv
Using CPython 3.12.12
Creating virtual environment with seed packages at: .venv
Activate with: source .venv/bin/activate
Activate with: source .venv/bin/activate
Audited 3 packages in 12ms
+ pip==25.3
Activate with: source .venv/bin/activate
⠙ ray==2.50.1
Resolved 263 packages in 70ms
Building vllm-model @ file:///home/ubuntu/gym-test-colab/responses_api_models/vllm_model
Built vllm-model @ file:///home/ubuntu/gym-test-colab/responses_api_models/vllm_model
Prepared 1 package in 822ms
Uninstalled 1 package in 0.50ms
Installed 1 package in 1ms
~ vllm-model==0.2.0rc0 (from file:///home/ubuntu/gym-test-colab/responses_api_models/vllm_model)
⠙ gradio-client==2.0.2 Connecting to Ray cluster at specified address: 172.27.16.211:54033
2025-12-19 19:33:41,228 INFO worker.py:1833 -- Connecting to existing Ray cluster at address: 172.27.16.211:54033...
2025-12-19 19:33:41,241 INFO worker.py:2004 -- Connected to Ray cluster. View the dashboard at http://127.0.0.1:8265
Resolved 181 packages in 963ms
/home/ubuntu/gym-test-colab/responses_api_agents/simple_agent/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
⠙ Preparing packages... (0/6)
immutabledict ------------------------------ 4.62 KiB/4.62 KiB
Building verifiable-instructions @ git+https://github.com/abukharin-nv/verifiable-instructions.git@f46a5ac87b1400a4f8973039844b6be9b56e3faf
⠙ Preparing packages... (0/6)
absl-py ------------------------------ 32.00 KiB/132.63 KiB
nltk ------------------------------ 30.92 KiB/1.44 MiB
en-core-web-sm ------------------------------ 47.26 KiB/12.21 MiB
Built verifiable-instructions @ git+https://github.com/abukharin-nv/verifiable-instructions.git@f46a5ac87b1400a4f8973039844b6be9b56e3faf
Built langdetect==1.0.9
Prepared 6 packages in 833ms
░░░░░░░░░░░░░░░░░░░░ [4/181] aiohappyeyeballs==2.6.1 Connecting to Ray cluster at specified address: 172.27.16.211:54033
░░░░░░░░░░░░░░░░░░░░ [6/181] jiter==0.12.0 2025-12-19 19:33:42,156 INFO worker.py:1833 -- Connecting to existing Ray cluster at address: 172.27.16.211:54033...
░░░░░░░░░░░░░░░░░░░░ [9/181] msgpack==1.1.2 2025-12-19 19:33:42,171 INFO worker.py:2004 -- Connected to Ray cluster. View the dashboard at http://127.0.0.1:8265
/home/ubuntu/gym-test-colab/responses_api_models/vllm_model/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
Adding a uvicorn logging filter so that the logs aren't spammed with 200 OK messages. This is to help errors pop up better and filter out noise.
INFO: Started server process [3598819]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:37653 (Press CTRL+C to quit)
Installed 181 packages in 341ms
+ absl-py==2.3.1
+ accumulation-tree==0.6.4
+ aiofiles==24.1.0
+ aiohappyeyeballs==2.6.1
+ aiohttp==3.13.2
+ aiohttp-cors==0.8.1
+ aiosignal==1.4.0
+ alembic==1.17.2
+ annotated-doc==0.0.4
+ annotated-types==0.7.0
+ antlr4-python3-runtime==4.9.3
+ anyio==4.12.0
+ asttokens==2.4.1
+ attrs==25.4.0
+ blinker==1.9.0
+ brotli==1.2.0
+ cachetools==6.2.4
+ certifi==2025.11.12
+ cffi==2.0.0
+ cfgv==3.5.0
+ charset-normalizer==3.4.4
+ click==8.3.1
+ cloudpickle==3.1.2
+ colorful==0.5.8
+ contourpy==1.3.3
+ coverage==7.13.0
+ cryptography==46.0.3
+ cycler==0.12.1
+ databricks-sdk==0.76.0
+ datasets==4.4.2
+ devtools==0.12.2
+ dill==0.4.0
+ distlib==0.4.0
+ distro==1.9.0
+ docker==7.1.0
+ en-core-web-sm==3.8.0 (from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl)
+ execnet==2.1.2
+ executing==2.2.1
+ fastapi==0.125.0
+ ffmpy==1.0.0
+ filelock==3.20.1
+ flask==3.1.2
+ flask-cors==6.0.2
+ fonttools==4.61.1
+ frozenlist==1.8.0
+ fsspec==2025.10.0
+ gitdb==4.0.12
+ gitpython==3.1.45
+ google-api-core==2.28.1
+ google-auth==2.45.0
+ googleapis-common-protos==1.72.0
+ gradio==6.2.0
+ gradio-client==2.0.2
+ graphene==3.4.3
+ graphql-core==3.2.7
+ graphql-relay==3.2.0
+ greenlet==3.3.0
+ groovy==0.1.2
+ grpcio==1.76.0
+ gunicorn==23.0.0
+ h11==0.16.0
+ hf-xet==1.2.0
+ httpcore==1.0.9
+ httpx==0.28.1
+ huey==2.5.5
+ huggingface-hub==1.2.3
+ hydra-core==1.3.2
+ identify==2.6.15
+ idna==3.11
+ immutabledict==4.2.2
+ importlib-metadata==8.7.0
+ iniconfig==2.3.0
+ itsdangerous==2.2.0
+ jinja2==3.1.6
+ jiter==0.12.0
+ joblib==1.5.3
+ jsonschema==4.25.1
+ jsonschema-specifications==2025.9.1
+ kiwisolver==1.4.9
+ langdetect==1.0.9
+ librt==0.7.4
+ mako==1.3.10
+ markdown-it-py==4.0.0
+ markupsafe==3.0.3
+ matplotlib==3.10.8
+ mdurl==0.1.2
+ mlflow==3.7.0
+ mlflow-skinny==3.7.0
+ mlflow-tracing==3.7.0
+ msgpack==1.1.2
+ multidict==6.7.0
+ multiprocess==0.70.18
+ mypy==1.19.1
+ mypy-extensions==1.1.0
+ nemo-gym==0.2.0rc0 (from file:///home/ubuntu/gym-test-colab)
+ nltk==3.9.2
+ nodeenv==1.9.1
+ numpy==2.3.5
+ omegaconf==2.3.0
+ openai==2.6.1
+ opencensus==0.11.4
+ opencensus-context==0.1.3
+ opentelemetry-api==1.39.1
+ opentelemetry-exporter-prometheus==0.60b1
+ opentelemetry-proto==1.39.1
+ opentelemetry-sdk==1.39.1
+ opentelemetry-semantic-conventions==0.60b1
+ orjson==3.11.5
+ packaging==25.0
+ pandas==2.3.3
+ pathspec==0.12.1
+ pillow==12.0.0
+ platformdirs==4.5.1
+ pluggy==1.6.0
+ pre-commit==4.5.1
+ prometheus-client==0.23.1
+ propcache==0.4.1
+ proto-plus==1.27.0
+ protobuf==6.33.2
+ psutil==7.1.3
+ py-spy==0.4.1
+ pyarrow==22.0.0
+ pyasn1==0.6.1
+ pyasn1-modules==0.4.2
+ pycparser==2.23
+ pydantic==2.12.5
+ pydantic-core==2.41.5
+ pydub==0.25.1
+ pygments==2.19.2
+ pyparsing==3.2.5
+ pytest==9.0.2
+ pytest-asyncio==1.3.0
+ pytest-cov==7.0.0
+ pytest-xdist==3.8.0
+ python-dateutil==2.9.0.post0
+ python-dotenv==1.2.1
+ python-multipart==0.0.21
+ pytz==2025.2
+ pyudorandom==1.0.0
+ pyyaml==6.0.3
+ ray==2.50.1
+ referencing==0.37.0
+ regex==2025.11.3
+ requests==2.32.5
+ requests-mock==1.12.1
+ rich==14.2.0
+ rpds-py==0.30.0
+ rsa==4.9.1
+ ruff==0.14.10
+ safehttpx==0.1.7
+ scikit-learn==1.8.0
+ scipy==1.16.3
+ semantic-version==2.10.0
+ shellingham==1.5.4
+ six==1.17.0
+ smart-open==7.5.0
+ smmap==5.0.2
+ sniffio==1.3.1
+ sqlalchemy==2.0.45
+ sqlparse==0.5.5
+ starlette==0.50.0
+ tdigest==0.5.2.2
+ threadpoolctl==3.6.0
+ tomlkit==0.13.3
+ tqdm==4.67.1
+ typer==0.20.1
+ typer-slim==0.20.1
+ typing-extensions==4.15.0
+ typing-inspection==0.4.2
+ tzdata==2025.3
+ urllib3==2.6.2
+ uvicorn==0.38.0
+ uvloop==0.22.1
+ verifiable-instructions==0.1.0 (from git+https://github.com/abukharin-nv/verifiable-instructions.git@f46a5ac87b1400a4f8973039844b6be9b56e3faf)
+ virtualenv==20.35.4
+ werkzeug==3.1.4
+ wrapt==2.0.1
+ xxhash==3.6.0
+ yappi==1.7.3
+ yarl==1.22.0
+ zipp==3.23.0
INFO: 127.0.0.1:38612 - "GET / HTTP/1.1" 404 Not Found
Waiting for servers to spin up
Checking for HTTP server statuses (you should see some HTTP requests to `/` that may 404. This is expected.)
INFO: 127.0.0.1:54708 - "GET / HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:37352 - "GET / HTTP/1.1" 404 Not Found
2 / 3 servers ready (0 timed out, 0 connection errored, 0 had unknown errors).
Waiting for servers to spin up: ['instruction_following']
Sleeping 3s...
Checking for HTTP server statuses (you should see some HTTP requests to `/` that may 404. This is expected.)
INFO: 127.0.0.1:40724 - "GET / HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:37752 - "GET / HTTP/1.1" 404 Not Found
2 / 3 servers ready (0 timed out, 0 connection errored, 0 had unknown errors).
Waiting for servers to spin up: ['instruction_following']
Sleeping 3s...
Connecting to Ray cluster at specified address: 172.27.16.211:54033
2025-12-19 19:33:48,784 INFO worker.py:1833 -- Connecting to existing Ray cluster at address: 172.27.16.211:54033...
2025-12-19 19:33:48,796 INFO worker.py:2004 -- Connected to Ray cluster. View the dashboard at http://127.0.0.1:8265
Adding a uvicorn logging filter so that the logs aren't spammed with 200 OK messages. This is to help errors pop up better and filter out noise.
INFO: Started server process [3598815]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:47835 (Press CTRL+C to quit)
(raylet) [2025-12-19 19:33:49,017 E 3588327 3588353] (raylet) file_system_monitor.cc:116: /tmp/ray/session_2025-12-19_19-33-36_893511_3587461 is over 95% full, available space: 2.66402 GB; capacity: 123.869 GB. Object creation will fail if spilling is required.
(raylet) [2025-12-19 19:33:49,017 E 3588327 3588353] (raylet) file_system_monitor.cc:116: /tmp/ray/session_2025-12-19_19-33-36_893511_3587461 is over 95% full, available space: 2.66402 GB; capacity: 123.869 GB. Object creation will fail if spilling is required.
(raylet) [2025-12-19 19:33:49,017 E 3588327 3588353] (raylet) file_system_monitor.cc:116: /tmp/ray/session_2025-12-19_19-33-36_893511_3587461 is over 95% full, available space: 2.66402 GB; capacity: 123.869 GB. Object creation will fail if spilling is required.
(raylet) [2025-12-19 19:33:49,017 E 3588327 3588353] (raylet) file_system_monitor.cc:116: /tmp/ray/session_2025-12-19_19-33-36_893511_3587461 is over 95% full, available space: 2.66402 GB; capacity: 123.869 GB. Object creation will fail if spilling is required.
Checking for HTTP server statuses (you should see some HTTP requests to `/` that may 404. This is expected.)
INFO: 127.0.0.1:39816 - "GET / HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:40738 - "GET / HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:37766 - "GET / HTTP/1.1" 404 Not Found
All 3 / 3 servers ready! Polling every 60s
####################################################################################################
#
# Server Instances
#
####################################################################################################
[1] instruction_following (resources_servers/instruction_following)
{
'process_name': 'instruction_following',
'server_type': 'resources_servers',
'name': 'instruction_following',
'dir_path': '/home/ubuntu/gym-test-colab/resources_servers/instruction_following',
'entrypoint': 'app.py',
'host': '127.0.0.1',
'port': 47835,
'pid': 3598815,
'config_path': 'instruction_following',
'url': 'http://127.0.0.1:47835',
}
[2] instruction_following_simple_agent (responses_api_agents/simple_agent)
{
'process_name': 'instruction_following_simple_agent',
'server_type': 'responses_api_agents',
'name': 'simple_agent',
'dir_path': '/home/ubuntu/gym-test-colab/responses_api_agents/simple_agent',
'entrypoint': 'app.py',
'host': '127.0.0.1',
'port': 44091,
'pid': 3598818,
'config_path': 'instruction_following_simple_agent',
'url': 'http://127.0.0.1:44091',
}
[3] policy_model (responses_api_models/vllm_model)
{
'process_name': 'policy_model',
'server_type': 'responses_api_models',
'name': 'vllm_model',
'dir_path': '/home/ubuntu/gym-test-colab/responses_api_models/vllm_model',
'entrypoint': 'app.py',
'host': '127.0.0.1',
'port': 37653,
'pid': 3598819,
'config_path': 'policy_model',
'url': 'http://127.0.0.1:37653',
}
####################################################################################################
im pretty sure its fine to default this to true