jupyter-ai
jupyter-ai copied to clipboard
jupyterlab
Jupyternaut fails when using ChatNVIDIAProvider
Description
Jupyternaut fails when trying to use NVIDIA::playground_llama2_70b (and other models from NVIDIA) with:
ValueError: Model meta/llama2-70b is incompatible with client ChatNVIDIAProvider. Please check `ChatNVIDIAProvider.get_available_models()`.
When suggested command run it fails with other error:
AssertionError: model_id was not specified. Please specify it as a keyword argument.
Reproduce
To reproduce:
python -m venv /tmp/venv
source /tmp/venv/bin/activate
pip install jupyter-ai[all]==2.28.2 langchain==0.2.17 langchain-core==0.2.43 langchain-nvidia-ai-endpoints==0.2.2
export NVIDIA_API_KEY="key"
python -c 'from jupyter_ai_magics.partner_providers.nvidia import ChatNVIDIAProvider; ChatNVIDIAProvider(model_id="playground_llama2_70b")'
Output will be
/tmp/venv/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_statics.py:659: UserWarning: Model playground_llama2_70b is deprecated. Using meta/llama2-70b instead.
warnings.warn(
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/venv/lib/python3.12/site-packages/jupyter_ai_magics/providers.py", line 354, in __init__
super().__init__(*args, **kwargs, **model_kwargs)
File "/tmp/venv/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/chat_models.py", line 243, in __init__
self._client = _NVIDIAClient(
^^^^^^^^^^^^^^
File "/tmp/venv/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py", line 175, in __init__
raise ValueError(
ValueError: Model meta/llama2-70b is incompatible with client ChatNVIDIAProvider. Please check `ChatNVIDIAProvider.get_available_models()`.
Context
Troubleshoot Output
PATH: /tmp/venv/bin /usr/sbin /usr/bin /bin /sbinsys.path: /tmp/venv/bin /usr/lib/python312.zip /usr/lib/python3.12 /usr/lib/python3.12/lib-dynload /usr/python3.12/site-packages
sys.executable: /tmp/venv/bin/python
sys.version: 3.12.7 (main, Nov 6 2024, 21:09:52) [GCC 14.1.0]
platform.platform(): Linux-5.14.0-427.13.1.el9_4.x86_64-x86_64-with-glibc2.34
which -a jupyter: /tmp/venv/bin/jupyter
pip list: Package Version ----------------------------- --------------- ai21 3.0.0 ai21-tokenizer 0.12.0 aiohappyeyeballs 2.4.3 aiohttp 3.11.7 aiolimiter 1.1.0 aiosignal 1.3.1 aiosqlite 0.20.0 annotated-types 0.7.0 anthropic 0.39.0 anyio 4.6.2.post1 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 arxiv 2.1.3 asttokens 2.4.1 async-lru 2.0.4 attrs 24.2.0 babel 2.16.0 bce-python-sdk 0.9.23 beautifulsoup4 4.12.3 bleach 6.2.0 boto3 1.34.162 botocore 1.34.162 cachetools 5.5.0 certifi 2024.8.30 cffi 1.17.1 charset-normalizer 3.4.0 click 8.1.7 cloudpickle 3.1.0 cohere 5.11.4 comm 0.2.2 dask 2024.11.2 dataclasses-json 0.6.7 debugpy 1.8.9 decorator 5.1.1 deepmerge 2.0 defusedxml 0.7.1 dill 0.3.9 diskcache 5.6.3 distributed 2024.11.2 distro 1.9.0 eval_type_backport 0.2.0 executing 2.1.0 faiss-cpu 1.9.0.post1 fastavro 1.9.7 fastjsonschema 2.20.0 feedparser 6.0.11 filelock 3.16.1 fqdn 1.5.1 frozenlist 1.5.0 fsspec 2024.10.0 future 1.0.0 google-ai-generativelanguage 0.6.6 google-api-core 2.23.0 google-api-python-client 2.154.0 google-auth 2.36.0 google-auth-httplib2 0.2.0 google-generativeai 0.7.2 googleapis-common-protos 1.66.0 gpt4all 2.8.2 greenlet 3.1.1 grpcio 1.68.0 grpcio-status 1.62.3 h11 0.14.0 httpcore 1.0.7 httplib2 0.22.0 httpx 0.27.2 httpx-sse 0.4.0 huggingface-hub 0.26.2 idna 3.10 importlib_metadata 8.5.0 ipykernel 6.29.5 ipython 8.29.0 ipywidgets 8.1.5 isoduration 20.11.0 jedi 0.19.2 Jinja2 3.1.4 jiter 0.7.1 jmespath 1.0.1 json5 0.9.28 jsonpatch 1.33 jsonpath-ng 1.7.0 jsonpointer 3.0.0 jsonschema 4.23.0 jsonschema-specifications 2024.10.1 jupyter_ai 2.28.2 jupyter_ai_magics 2.28.2 jupyter_client 8.6.3 jupyter_core 5.7.2 jupyter-events 0.10.0 jupyter-lsp 2.2.5 jupyter_server 2.14.2 jupyter_server_terminals 0.5.3 jupyterlab 4.3.1 jupyterlab_pygments 0.3.0 jupyterlab_server 2.27.3 jupyterlab_widgets 3.0.13 langchain 0.2.17 langchain-anthropic 0.1.23 langchain-aws 0.1.18 langchain-cohere 0.2.4 langchain-community 0.2.19 langchain-core 0.2.43 langchain-experimental 0.0.65 langchain-google-genai 1.0.10 langchain-mistralai 0.1.13 langchain-nvidia-ai-endpoints 0.2.2 langchain-ollama 0.1.3 langchain-openai 0.1.25 langchain-text-splitters 0.2.4 langsmith 0.1.146 locket 1.0.0 markdown-it-py 3.0.0 MarkupSafe 3.0.2 marshmallow 3.23.1 matplotlib-inline 0.1.7 mdurl 0.1.2 mistune 3.0.2 msgpack 1.1.0 multidict 6.1.0 multiprocess 0.70.17 mypy-extensions 1.0.0 nbclient 0.10.0 nbconvert 7.16.4 nbformat 5.10.4 nest-asyncio 1.6.0 notebook_shim 0.2.4 numpy 1.26.4 ollama 0.4.1 openai 1.55.1 orjson 3.10.12 overrides 7.7.0 packaging 24.2 pandas 2.2.3 pandocfilters 1.5.1 parameterized 0.9.0 parso 0.8.4 partd 1.4.2 pexpect 4.9.0 pillow 10.4.0 pip 24.2 platformdirs 4.3.6 ply 3.11 prometheus_client 0.21.0 prompt_toolkit 3.0.48 propcache 0.2.0 proto-plus 1.25.0 protobuf 4.25.5 psutil 6.1.0 ptyprocess 0.7.0 pure_eval 0.2.3 pyarrow 18.1.0 pyasn1 0.6.1 pyasn1_modules 0.4.1 pycparser 2.22 pycryptodome 3.21.0 pydantic 2.10.2 pydantic_core 2.27.1 Pygments 2.18.0 pyparsing 3.2.0 pypdf 5.1.0 python-dateutil 2.9.0.post0 python-dotenv 1.0.1 python-json-logger 2.0.7 pytz 2024.2 PyYAML 6.0.2 pyzmq 26.2.0 qianfan 0.4.12.1 referencing 0.35.1 regex 2024.11.6 requests 2.32.3 requests-toolbelt 1.0.0 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rich 13.9.4 rpds-py 0.21.0 rsa 4.9 s3transfer 0.10.4 Send2Trash 1.8.3 sentencepiece 0.2.0 setuptools 75.6.0 sgmllib3k 1.0.0 shellingham 1.5.4 six 1.16.0 sniffio 1.3.1 sortedcontainers 2.4.0 soupsieve 2.6 SQLAlchemy 2.0.36 stack-data 0.6.3 tabulate 0.9.0 tblib 3.0.0 tenacity 8.5.0 terminado 0.18.1 tiktoken 0.8.0 tinycss2 1.4.0 together 1.3.5 tokenizers 0.20.4 toolz 1.0.0 tornado 6.4.2 tqdm 4.67.1 traitlets 5.14.3 typer 0.13.1 types-python-dateutil 2.9.0.20241003 types-requests 2.32.0.20241016 typing_extensions 4.12.2 typing-inspect 0.9.0 tzdata 2024.2 uri-template 1.3.0 uritemplate 4.1.1 urllib3 2.2.3 wcwidth 0.2.13 webcolors 24.11.1 webencodings 0.5.1 websocket-client 1.8.0 widgetsnbextension 4.0.13 yarl 1.18.0 zict 3.0.0 zipp 3.21.0
