Error message when upload file

[alexng88]

Hello, I can load the interface but when I upload a PDF file, it shows: Chroma.init() got an unexpected keyword argument 'anonymized_telemetry'

any clues?

[pseudotensor]

Been reported before, and in each case I think it's clear the version of langchain was wrong. Please ensure you are using a fresh env. If it's fresh, please share result of pip freeze &> freeze.txt.

[alexng88]

yes im using a fresh env. Mac with M1.

absl-py==1.4.0 accelerate==0.20.3 aiofiles==23.1.0 aiohttp==3.8.4 aiosignal==1.3.1 alt-profanity-check==1.2.2 altair==5.0.1 anyio==3.7.1 appdirs==1.4.4 APScheduler==3.10.1 arrow==1.2.3 async-timeout==4.0.2 attrs==23.1.0 backoff==2.2.1 beautifulsoup4==4.12.2 better-profanity==0.7.0 bioc==2.0 bitsandbytes==0.39.0 boto3==1.26.101 botocore==1.29.101 bravado==11.0.3 bravado-core==5.17.1 cachetools==5.3.1 certifi==2023.5.7 charset-normalizer==3.2.0 click==8.1.5 colorama==0.4.6 contourpy==1.1.0 cycler==0.11.0 dataclasses-json==0.5.9 datasets==2.13.0 dill==0.3.6 distlib==0.3.6 docutils==0.20.1 einops==0.6.1 et-xmlfile==1.1.0 evaluate==0.4.0 exceptiongroup==1.1.2 execnet==2.0.2 fastapi==0.100.0 ffmpy==0.3.0 filelock==3.12.0 fire==0.5.0 fonttools==4.41.0 fqdn==1.5.1 frozenlist==1.4.0 fsspec==2023.6.0 future==0.18.3 gitdb==4.0.10 GitPython==3.1.32 google-auth==2.22.0 google-auth-oauthlib==1.0.0 gradio==3.35.2 gradio_client==0.2.7 greenlet==2.0.2 grpcio==1.56.0 h11==0.14.0 httpcore==0.17.3 httpx==0.24.1 huggingface-hub==0.15.1 idna==3.4 iniconfig==2.0.0 InstructorEmbedding==1.0.1 intervaltree==3.1.0 isoduration==20.11.0 Jinja2==3.1.2 jmespath==1.0.1 joblib==1.3.1 jsonlines==3.1.0 jsonpointer==2.4 jsonref==1.1.0 jsonschema==4.18.3 jsonschema-specifications==2023.6.1 kiwisolver==1.4.4 langchain==0.0.234 langsmith==0.0.5 linkify-it-py==2.0.2 lm-dataformat==0.0.20 loralib==0.1.1 lxml==4.9.3 Markdown==3.4.3 markdown-it-py==2.2.0 MarkupSafe==2.1.3 marshmallow==3.19.0 marshmallow-enum==1.5.1 matplotlib==3.7.1 mdit-py-plugins==0.3.3 mdurl==0.1.2 monotonic==1.6 mpmath==1.3.0 msgpack==1.0.5 multidict==6.0.4 multiprocess==0.70.14 mypy-extensions==1.0.0 neptune==1.2.0 networkx==3.1 nltk==3.8.1 numexpr==2.8.4 numpy==1.24.3 oauthlib==3.2.2 openai==0.27.8 openapi-schema-pydantic==1.2.4 openpyxl==3.1.2 orjson==3.9.2 packaging==23.1 pandas==2.0.2 peft @ git+https://github.com/huggingface/peft.git@06fd06a4d2e8ed8c3a253c67d9c3cb23e0f497ad Pillow==10.0.0 platformdirs==3.5.0 pluggy==1.2.0 portalocker==2.7.0 posthog==3.0.1 protobuf==4.23.4 psutil==5.9.5 pyarrow==12.0.1 pyasn1==0.5.0 pyasn1-modules==0.3.0 pydantic==1.10.11 pydub==0.25.1 Pygments==2.15.1 PyJWT==2.7.0 pynvml==11.5.0 pypandoc-binary==1.11 pyparsing==3.1.0 pypdf==3.12.1 pyphen==0.14.0 pytest==7.2.2 pytest-xdist==3.2.1 python-dateutil==2.8.2 python-dotenv==1.0.0 python-multipart==0.0.6 pytz==2023.3 PyYAML==6.0 referencing==0.29.1 regex==2023.6.3 requests==2.31.0 requests-oauthlib==1.3.1 responses==0.18.0 rfc3339-validator==0.1.4 rfc3987==1.3.8 rouge-score==0.1.2 rpds-py==0.8.10 rsa==4.9 s3transfer==0.6.1 sacrebleu==2.3.1 safetensors==0.3.1 scikit-learn==1.2.2 scipy==1.11.1 semantic-version==2.10.0 sentence-transformers==2.2.2 sentencepiece==0.1.99 simplejson==3.19.1 six==1.16.0 smmap==5.0.0 sniffio==1.3.0 sortedcontainers==2.4.0 soupsieve==2.4.1 SQLAlchemy==2.0.19 starlette==0.27.0 swagger-spec-validator==3.0.3 sympy==1.12 tabulate==0.9.0 tenacity==8.2.2 tensorboard==2.13.0 tensorboard-data-server==0.7.1 termcolor==2.3.0 text-generation==0.6.0 textstat==0.7.3 threadpoolctl==3.2.0 tiktoken==0.4.0 tokenizers==0.13.3 tomli==2.0.1 toolz==0.12.0 torch==2.1.0.dev20230714 torchaudio==2.1.0.dev20230714 torchvision==0.16.0.dev20230714 tqdm==4.65.0 transformers==4.30.2 typing-inspect==0.9.0 typing_extensions==4.7.1 tzdata==2023.3 tzlocal==5.0.1 uc-micro-py==1.0.2 ujson==5.8.0 uri-template==1.3.0 urllib3==1.26.16 uvicorn==0.23.0 virtualenv==20.23.0 webcolors==1.13 websocket-client==1.6.1 websockets==11.0.3 Werkzeug==2.3.6 xxhash==3.2.0 yarl==1.9.2 zstandard==0.21.0

[pseudotensor]

Thanks. The langchain version is off. It should be langchain==0.0.202. Perhaps some other package updates it due to the order of installation.

If you have a chance, could you start with fresh env, run the pip install steps, and share output for each step? Or at least check which package updated langchain? Thanks!

[pseudotensor]

BTW, you can run: pytest -s -v tests/test_requirements.py::test_requirements to see which dependencies are wrong (can ignore peft or things like numpy/pandas, but not rest).

[alexng88]

Thanks. The langchain version is off. It should be langchain==0.0.202. Perhaps some other package updates it due to the order of installation.

If you have a chance, could you start with fresh env, run the pip install steps, and share output for each step? Or at least check which package updated langchain? Thanks!

This seems works!!! The error is gone! By the way, how can I speed up the response? Now it requires a few seconds to produce a word

[alexng88]

pytest -s -v tests/test_requirements.py::test_requirements

(h2ogpt2) development@Developments-MacBook-Pro h2ogpt % pytest -s -v tests/test_requirements.py::test_requirements ==================================================================== test session starts ===================================================================== platform darwin -- Python 3.10.12, pytest-7.2.2, pluggy-1.2.0 -- /Users/development/opt/anaconda3/envs/h2ogpt2/bin/python cachedir: .pytest_cache rootdir: /Users/development/h2ogpt/h2ogpt plugins: xdist-3.2.1, anyio-3.7.1 collected 1 item

tests/test_requirements.py::test_requirements FAILED

========================================================================== FAILURES ========================================================================== _____________________________________________________________________ test_requirements ______________________________________________________________________ concurrent.futures.process._RemoteTraceback: """ Traceback (most recent call last): File "/Users/development/opt/anaconda3/envs/h2ogpt2/lib/python3.10/concurrent/futures/process.py", line 246, in _process_worker r = call_item.fn(*call_item.args, **call_item.kwargs) File "/Users/development/h2ogpt/h2ogpt/src/utils.py", line 777, in _traced_func func, args, kwargs = forkdatacontext.get_args_kwargs_for_traced_func(func, args, kwargs) File "/Users/development/h2ogpt/h2ogpt/src/utils.py", line 768, in get_args_kwargs_for_traced_func assert func is not None, "function should not be None, indicates original args[0] was None or args was None" AssertionError: function should not be None, indicates original args[0] was None or args was None """

The above exception was the direct cause of the following exception:

args = (), kwargs = {} func_new = functools.partial(<function call_subprocess_onetask at 0x7fd0d1b575b0>, <function test_requirements at 0x7fd0d0d96a70>, (), {})

@wraps(func)
def f(*args, **kwargs):
    func_new = partial(call_subprocess_onetask, func, args, kwargs)

  return run_test(func_new)

tests/utils.py:18:

---

tests/utils.py:24: in run_test return func(*args, **kwargs) src/utils.py:796: in call_subprocess_onetask return future.result() ../../opt/anaconda3/envs/h2ogpt2/lib/python3.10/concurrent/futures/_base.py:458: in result return self.__get_result()

---

self = None

def __get_result(self):
    if self._exception:
        try:

          raise self._exception

E AssertionError: function should not be None, indicates original args[0] was None or args was None

../../opt/anaconda3/envs/h2ogpt2/lib/python3.10/concurrent/futures/_base.py:403: AssertionError ====================================================================== warnings summary ====================================================================== ../../opt/anaconda3/envs/h2ogpt2/lib/python3.10/site-packages/pkg_resources/init.py:121 /Users/development/opt/anaconda3/envs/h2ogpt2/lib/python3.10/site-packages/pkg_resources/init.py:121: DeprecationWarning: pkg_resources is deprecated as an API warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)

../../opt/anaconda3/envs/h2ogpt2/lib/python3.10/site-packages/pkg_resources/init.py:2870 /Users/development/opt/anaconda3/envs/h2ogpt2/lib/python3.10/site-packages/pkg_resources/init.py:2870: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('mpl_toolkits'). Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================================================================== short test summary info =================================================================== FAILED tests/test_requirements.py::test_requirements - AssertionError: function should not be None, indicates original args[0] was None or args was None =============================================================== 1 failed, 2 warnings in 1.69s ================================================================ (h2ogpt2) development@Developments-MacBook-Pro h2ogpt %

I don't know how to read the output

[pseudotensor]

Thanks, mac/windows can't handle the forking pytest does, so I disabled that so it can be run.

e40ca19a37178ad3a98a8ead7cec3bedf74d8a83

[pseudotensor]

I tried on fresh env to do these steps:

pip install -r requirements.txt
pip install -r reqs_optional/requirements_optional_langchain.txt
pip install -r reqs_optional/requirements_optional_gpt4all.txt
pip install -r reqs_optional/requirements_optional_langchain.gpllike.txt
pip install -r reqs_optional/requirements_optional_langchain.urls.txt

and I still had langchain==0.0.202, so not sure how it happens for some that fresh env leads to new langchain.

The langchain you had is the latest version, but none of the packages install langchain except langchain itself. I also checked all dependencies with pipdeptree -r, and no langchain dependencies come from anything except h2ogpt.

Will close for now until further ideas or issues.