in python 3.11 and release 0.8.0: `ValueError: mutable default <class 'datasets.utils.version.Version'> for field version is not allowed: use default_factory`

[bm-synth]

System Info

In python 3.11 and TensorRT-LLM release 0.8.0, when running import tensorrt_llm, the following error message is displayed:

ValueError: mutable default <class 'datasets.utils.version.Version'> for field version is not allowed: use default_factory

Im running this from the docker image. full stack:

 python3 -c "import tensorrt_llm; print(tensorrt_llm.__version__)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/root/TensorRT-LLM_rel/tensorrt_llm/__init__.py", line 31, in <module>
    import tensorrt_llm.models as models
  File "/root/TensorRT-LLM_rel/tensorrt_llm/models/__init__.py", line 25, in <module>
    from .llama.model import LLaMAForCausalLM, LLaMAModel
  File "/root/TensorRT-LLM_rel/tensorrt_llm/models/llama/model.py", line 21, in <module>
    from tensorrt_llm.models.llama.weight import (load_from_awq_llama,
  File "/root/TensorRT-LLM_rel/tensorrt_llm/models/llama/weight.py", line 32, in <module>
    from tensorrt_llm.runtime.lora_manager import LoraConfig
  File "/root/TensorRT-LLM_rel/tensorrt_llm/runtime/__init__.py", line 23, in <module>
    from .model_runner import ModelRunner
  File "/root/TensorRT-LLM_rel/tensorrt_llm/runtime/model_runner.py", line 30, in <module>
    from .engine import Engine, get_engine_version
  File "/root/TensorRT-LLM_rel/tensorrt_llm/runtime/engine.py", line 9, in <module>
    from ..builder import BuildConfig
  File "/root/TensorRT-LLM_rel/tensorrt_llm/builder.py", line 396, in <module>
    @dataclass
     ^^^^^^^^^
  File "/usr/lib/python3.11/dataclasses.py", line 1230, in dataclass
    return wrap(cls)
           ^^^^^^^^^
  File "/usr/lib/python3.11/dataclasses.py", line 1220, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/dataclasses.py", line 958, in _process_class
    cls_fields.append(_get_field(cls, name, type, kw_only))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/dataclasses.py", line 815, in _get_field
    raise ValueError(f'mutable default {type(f.default)} for field '
ValueError: mutable default <class 'tensorrt_llm.plugin.plugin.PluginConfig'> for field plugin_config is not allowed: use default_factory

the system installation is the same as the ones in docker/Dockerfile.multi. TensorRT installed with ./docker/common/install_tensorrt.sh and TensorRT-LLM installed with python3 ./scripts/build_wheel.py -D TRT_LIB_DIR=${TENSORRT_HOME}/lib -D TRT_INCLUDE_DIR=${TENSORRT_HOME}/include, where TENSORRT_HOME="/usr/local/tensorrt"

A similar error has been discussed here: https://github.com/huggingface/datasets/issues/5230 and is due to one of the following changes in the dataclasses standard library in version 3.11:

• Changed in version 3.11: Instead of looking for and disallowing objects of type list, dict, or set, unhashable objects are now not allowed as default values. Unhashability is used to approximate mutability.
• fields may optionally specify a default value, using normal Python syntax:

Who can help?

No response

Information

• [ ] The official example scripts
• [X] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

1. install TensorRT following docker/Dockerfile.multi with python 3.11
2. run python3 -c "import tensorrt_llm; print(tensorrt_llm.__version__)"

Expected behavior

0.8.0

actual behavior

see stack trace on description

additional notes

commenting @dataclass in /root/TensorRT-LLM_rel/tensorrt_llm/builder.py, line 396 seems to fix the import, but may lead to errors at runtime.

[brunomaga]

also

from dataclasses import field

and

plugin_config: PluginConfig = field(default_factory=PluginConfig)

should fix it.

[mirekphd]

I thought only Python 3.10 was supported?:) That's what the docs say anyway here (I know that this Python version is bundled with the current LTS version of Ubuntu, but we and others have skipped Python 3.10 when moving from Python 3.9, because Python 3.11 is faster than both of them).

[nathanael-perraudin-synth]

@byshiue Would it be possible to have an update on this issue? Is there a PR with @brunomaga solution?

[bm-synth]

AFAIK, the python version version of TensorRT-LLM matches the one shipped by the default base image. So far (docker/Makefile):

BASE_TAG = 12.4.1-devel-ubuntu22.04

And ubuntu 22.04 ships with python 3.10. When TensorRT-LLM starts using Ubuntu 24.04 as base image instead, it will ship with python 3.12, and this bug fix should be included in the new TensorRT-LLM release. For now:

commenting @dataclass in /root/TensorRT-LLM_rel/tensorrt_llm/builder.py, line 396 seems to fix the import, but may lead to errors at runtime.

[nathanael-perraudin-synth]

@bm-synth It might take some time before TensorRT-LLM starts using Ubuntu 24.04... Thanks for the update.