text-generation-inference
text-generation-inference copied to clipboard
Fix Numba Cache Error
What does this PR do?
Fix for an error I received while trying to utilize the intel GPU TGI image.
Fixes
2024-08-21T22:35:19.551153Z INFO download: text_generation_launcher: Starting check and download process for Intel/neural-chat-7b-v3-3
Error: DownloadError
2024-08-21T22:35:24.217869Z ERROR download: text_generation_launcher: Download encountered an error:
/usr/local/lib/python3.10/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 'libpng16.so.16: cannot open shared object file: No such file or directory'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
warn(
2024-08-21 22:35:23.250 | INFO | text_generation_server.utils.import_utils:<module>:75 - Detected system ipex
Traceback (most recent call last):
File "/usr/local/bin/text-generation-server", line 5, in <module>
from text_generation_server.cli import app
File "/usr/local/lib/python3.10/dist-packages/text_generation_server/cli.py", line 10, in <module>
from text_generation_server.utils.adapter import parse_lora_adapters
File "/usr/local/lib/python3.10/dist-packages/text_generation_server/utils/__init__.py", line 13, in <module>
from text_generation_server.utils.tokens import (
File "/usr/local/lib/python3.10/dist-packages/text_generation_server/utils/tokens.py", line 7, in <module>
from text_generation_server.utils.logits_process import (
File "/usr/local/lib/python3.10/dist-packages/text_generation_server/utils/logits_process.py", line 8, in <module>
from outlines.fsm.fsm import RegexFSM
File "/usr/local/lib/python3.10/dist-packages/outlines/__init__.py", line 2, in <module>
import outlines.generate
File "/usr/local/lib/python3.10/dist-packages/outlines/generate/__init__.py", line 1, in <module>
from .api import SequenceGenerator
File "/usr/local/lib/python3.10/dist-packages/outlines/generate/api.py", line 5, in <module>
from outlines.fsm.fsm import FSMState
File "/usr/local/lib/python3.10/dist-packages/outlines/fsm/fsm.py", line 9, in <module>
from outlines.fsm.regex import create_fsm_index_tokenizer, make_deterministic_fsm
File "/usr/local/lib/python3.10/dist-packages/outlines/fsm/regex.py", line 96, in <module>
def create_fsm_info(
File "/usr/local/lib/python3.10/dist-packages/numba/core/decorators.py", line 225, in wrapper
disp.enable_caching()
File "/usr/local/lib/python3.10/dist-packages/numba/core/dispatcher.py", line 808, in enable_caching
self._cache = FunctionCache(self.py_func)
File "/usr/local/lib/python3.10/dist-packages/numba/core/caching.py", line 601, in __init__
self._impl = self._impl_class(py_func)
File "/usr/local/lib/python3.10/dist-packages/numba/core/caching.py", line 337, in __init__
raise RuntimeError("cannot cache function %r: no locator available "
RuntimeError: cannot cache function 'create_fsm_info': no locator available for file '/usr/local/lib/python3.10/dist-packages/outlines/fsm/regex.py'
Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the contributor guideline, Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
- [ ] Did you write any new necessary tests?
Who can review?
@OlivierDehaene @Narsil