vllm
vllm copied to clipboard
Enable Pydantic mypy checks and convert configs to Pydantic dataclasses
- Enables the mypy extension for Pydantic
- Fix the new mypy errors created by doing that
- Swap out the builtin dataclasses for Pydantic dataclasses (free input validation!)
$ vllm serve meta-llama/Llama-3.2-1B-Instruct --gpu-memory-utilization 0.45 --compilation-config '{"level":2.5}' INFO 05-12 12:59:58 [__init__.py:248] Automatically detected platform cuda. WARNING 05-12 13:00:02 [config.py:2034] max_model_len was is not set. Defaulting to arbitrary value of 8192. WARNING 05-12 13:00:02 [config.py:2040] max_num_seqs was is not set. Defaulting to arbitrary value of 128. usage: vllm serve [model_tag] [options] vllm serve: error: argument --compilation-config/-O: 1 validation error for CompilationConfig level Input should be a valid integer, got a number with a fractional part [type=int_from_float, input_value=2.5, input_type=float] For further information visit https://errors.pydantic.dev/2.11/v/int_from_float
N.B. Pydantic base models handle mutable defaults correctly. x: list = [] looks much nicer than the alternative mypy will allow x: list = Field(default_factory=lambda: [])
👋 Hi! Thank you for contributing to the vLLM project.
💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.
Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.
To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.
🚀
I think this PR must wait for https://github.com/vllm-project/vllm/pull/17485. Hopefully the switch to static analysis in the docs build will prevent the Pydantic errors
This pull request has merge conflicts that must be resolved before it can be merged. Please rebase the PR, @hmellor.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork
This pull request has merge conflicts that must be resolved before it can be merged. Please rebase the PR, @hmellor.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork
Instead of fighting Sphinx, I will try and merge https://github.com/vllm-project/vllm/pull/18145 so that we never have to think about breaking the docs build again 🎉
This pull request has merge conflicts that must be resolved before it can be merged. Please rebase the PR, @hmellor.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork
Getting this error on Python 3.10 after this PR:
Traceback (most recent call last):
File "/home/luka/neuralmagic-vllm/examples/offline_inference/basic/generate.py", line 3, in <module>
from vllm import LLM, EngineArgs
File "/home/luka/neuralmagic-vllm/vllm/__init__.py", line 12, in <module>
from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs
File "/home/luka/neuralmagic-vllm/vllm/engine/arg_utils.py", line 21, in <module>
from vllm.config import (BlockSize, CacheConfig, CacheDType, CompilationConfig,
File "/home/luka/neuralmagic-vllm/vllm/config.py", line 1704, in <module>
class ParallelConfig:
File "/home/luka/neuralmagic-vllm/venv/lib/python3.10/site-packages/pydantic/dataclasses.py", line 264, in dataclass
return create_dataclass if _cls is None else create_dataclass(_cls)
File "/home/luka/neuralmagic-vllm/venv/lib/python3.10/site-packages/pydantic/dataclasses.py", line 261, in create_dataclass
_pydantic_dataclasses.complete_dataclass(cls, config_wrapper, raise_errors=False, types_namespace=None)
File "/home/luka/neuralmagic-vllm/venv/lib/python3.10/site-packages/pydantic/_internal/_dataclasses.py", line 181, in complete_dataclass
cls.__pydantic_validator__ = validator = create_schema_validator(
File "/home/luka/neuralmagic-vllm/venv/lib/python3.10/site-packages/pydantic/plugin/_schema_validator.py", line 50, in create_schema_validator
return SchemaValidator(schema, config)
pydantic_core._pydantic_core.SchemaError: Error building "dataclass" validator:
SchemaError: Error building "dataclass-args" validator:
SchemaError: Field 'distributed_executor_backend':
SchemaError: Error building "default" validator:
SchemaError: Error building "nullable" validator:
SchemaError: Error building "union" validator:
SchemaError: Error building "is-subclass" validator:
TypeError: 'str' object cannot be converted to 'PyType'
Discussion in https://vllm-dev.slack.com/archives/C07R5Q1Q2BB/p1748456500324789