datarobot-user-models
datarobot-user-models copied to clipboard
Minor improvements to GPU examples
This repository is public. Do not put here any private DataRobot or customer's data: code, datasets, model artifacts, .etc.
Summary
- Remove concurrency param from example
- Set vLLM model via engine config
Rationale
This makes some minor improvements to our examples. First it is better to set the vLLM model param via the config file because then it is versioned and can't be overriden at runtime with no auditing. Also don't showcase the concurrency flag yet as we haven't tested it extensively.