FileNotFoundError: [Errno 2] No such file or directory: '/root/dolphin-2_6-phi-2/config.json' when running self build docker image

[OliverWalter]

When running a newly self build docker image, I'm getting the error message: FileNotFoundError: [Errno 2] No such file or directory: '/root/dolphin-2_6-phi-2/config.json'

Some more context:

s6-rc: info: service legacy-services successfully started /usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _torch_pytree._register_pytree_node( /usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _torch_pytree._register_pytree_node( /usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _torch_pytree._register_pytree_node( /usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _torch_pytree._register_pytree_node( /usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _torch_pytree._register_pytree_node( Process Process-3: Traceback (most recent call last): File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/root/WhisperFusion/llm_service.py", line 205, in run self.initialize_model( File "/root/WhisperFusion/llm_service.py", line 98, in initialize_model model_name = read_model_name(engine_dir) File "/root/WhisperFusion/llm_service.py", line 23, in read_model_name engine_version = tensorrt_llm.runtime.engine.get_engine_version(engine_dir) File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/runtime/engine.py", line 81, in get_engine_version with open(config_path, 'r') as f: FileNotFoundError: [Errno 2] No such file or directory: '/root/dolphin-2_6-phi-2/config.json' /usr/local/lib/python3.10/dist-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm. warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.") [2024-02-09 13:49:45,605] [0/0] torch._dynamo.variables.torch: [WARNING] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored

I had to rebuild for compute capability 8.9 with: bash build.sh 89-real

I can't immediately see any obvious error messages in the build (also not guaranteed that I missed something) I used the master branch at c90a694

Could you provide an image with compute capability 8.9? Or any ideas for the fix? Should I use another checkout to rebuild the docker image?

[makaveli10]

Hello @OliverWalter, the docker image mentioned in the readme is actually for 4090 i.e. 89-real. So, you should be able to use:

 docker run --gpus all --shm-size 64G -p 6006:6006 -p 8888:8888 -it ghcr.io/collabora/whisperfusion:latest

[OliverWalter]

Hi @makaveli10, thanks for the reply. The Current image does not seem to run for me. It shows a different error (undefined symbol) See Issue #40 for details.