ART
ART copied to clipboard
enforce_eager=True not being respected in test scripts
Problem
When running test scripts with enforce_eager=True specified, the logs still show enforce_eager=False and CUDA graphs are being calculated. This makes startup slower and leads to a slower feedback cycle during testing.
Reproduction
In the test script src/art/test/test_step_skipping.py, we're passing enforce_eager=True:
# Register the model
await model.register(
backend,
_openai_client_config={"engine_args": {"enforce_eager": True}},
)
However, when running the script, the logs show that enforce_eager is still False and CUDA graphs are being compiled.
Expected Behavior
When enforce_eager=True is passed in the configuration, it should:
- Skip CUDA graph compilation
- Start up faster
- Provide quicker feedback during testing
Impact
This issue affects development velocity as tests take longer to start and provide feedback than necessary.
Environment
- The issue can be reproduced by running:
./src/art/test/test_step_skipping.py - The script is configured to run on GPU with sky launch