[BUG] - cannot process chat with the pdfs

[michaelyeung445]

trafficstars

Description

I uploaded a file to test but it occured error, and may I know what is the default llm to use in kotaemon?

the error message in cmd: File "C:\Users\Micha\Downloads\kotaemon-app\install_dir\env\lib\site-packages\tenacity_init_.py", line 326, in iter raise retry_exc from fut.exception() tenacity.RetryError: RetryError[<Future at 0x26fd6e69a50 state=finished raised PermissionDeniedError>]

Reproduction steps

I uploaded a file to test but it occured error

Screenshots

No response

Logs

No response

Browsers

No response

OS

No response

Additional information

No response

[michaelyeung445]

hi, anyone here?

[taprosoft]

Hi, please describe your issue in more detail. You need to setup your LLM (either OpenAI, Azure OpenAI or a local ones). See the README for instructions how to setup or go with local model route here.

[michaelyeung445]

I just uploaded a file and test it with "summarize this pdf" then error, I think maybe the LLM problem, I need to set up even I set default llm? Also, if use openai api, is this mean only gpt model or any api in openai api format?(Like openrouter and groq)

[khankaholic]

Also, if use openai api, is this mean only gpt model or any api in openai api format?(Like openrouter and groq)

Hi @michaelyeung445, currently, here are some LLM interfaces that we support, most of them are wrapped by Lanchain, and LlamaCpp. If you're interested in any kind of other interface/format, please feel free to add it on. https://github.com/Cinnamon/kotaemon/blob/c1e8c37e5e7ad8360509f71ecf1ba156e9f1791c/libs/kotaemon/kotaemon/llms/init.py#L30-L39

Btw, could you please provide details about your LLM settings? You will need to supply your own API key (e.g., for OpenAI or Azure OpenAI) or the endpoint of a hosted model.

[hlhlhl231]

same question T-T

[michaelyeung445]

now i can chat but it cannot takes my pdf as reference.

[anhkhoi2017]

I've got the same problem. I can chat with the model without the attached document. But if I select a document to chat, it will show error.

Here's the log:

Session reasoning type None Session LLM None Reasoning class <class 'ktem.reasoning.simple.FullQAPipeline'> Reasoning state {'app': {'regen': False}, 'pipeline': {}} Thinking ... Retrievers [DocumentRetrievalPipeline(DS=<kotaemon.storages.docstores.lancedb.LanceDBDocumentStore object at 0x000001E284331DB0>, FSPath=WindowsPath('D:/20240916-Kotaemon/kotaemon-app/ktem_app_data/user_data/files/index_1'), Index=<class 'ktem.index.file.index.IndexTable'>, Source=<class 'ktem.index.file.index.Source'>, VS=<kotaemon.storages.vectorstores.chroma.ChromaVectorStore object at 0x000001E284332050>, get_extra_table=False, llm_scorer=LLMTrulensScoring(concurrent=True, normalize=10, prompt_template=<kotaemon.llms.prompts.template.PromptTemplate object at 0x000001E286A30E50>, system_prompt_template=<kotaemon.llms.prompts.template.PromptTemplate object at 0x000001E286A32800>, top_k=3, user_prompt_template=<kotaemon.llms.prompts.template.PromptTemplate object at 0x000001E286A32BF0>), mmr=False, rerankers=[CohereReranking(cohere_api_key='', model_name='rerank-multilingual-v2.0')], retrieval_mode='hybrid', top_k=10, user_id=1), GraphRAGRetrieverPipeline(DS=<theflow.base.unset_ object at 0x000001E2EB6EC220>, FSPath=<theflow.base.unset_ object at 0x000001E2EB6EC220>, Index=<class 'ktem.index.file.index.IndexTable'>, Source=<theflow.base.unset_ object at 0x000001E2EB6EC220>, VS=<theflow.base.unset_ object at 0x000001E2EB6EC220>, file_ids=[], user_id=<theflow.base.unset_ object at 0x000001E2EB6EC220>)] searching in doc_ids ['4299411a-78cc-4d17-b7bb-8d1f59b7d9a8'] retrieval_kwargs: dict_keys(['do_extend', 'scope', 'filters']) Traceback (most recent call last): File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\gradio\queueing.py", line 575, in process_events response = await route_utils.call_process_api( File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\gradio\route_utils.py", line 276, in call_process_api output = await app.get_blocks().process_api( File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\gradio\blocks.py", line 1923, in process_api result = await self.call_function( File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\gradio\blocks.py", line 1520, in call_function prediction = await utils.async_iteration(iterator) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\gradio\utils.py", line 663, in async_iteration return await iterator.anext() File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\gradio\utils.py", line 656, in anext return await anyio.to_thread.run_sync( File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\anyio\to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\anyio_backends_asyncio.py", line 2177, in run_sync_in_worker_thread return await future File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\anyio_backends_asyncio.py", line 859, in run result = context.run(func, *args) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\gradio\utils.py", line 639, in run_sync_iterator_async return next(iterator) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\gradio\utils.py", line 801, in gen_wrapper response = next(iterator) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\ktem\pages\chat_init_.py", line 804, in chat_fn for response in pipeline.stream(chat_input, conversation_id, chat_history): File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\ktem\reasoning\simple.py", line 660, in stream docs, infos = self.retrieve(message, history) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\ktem\reasoning\simple.py", line 488, in retrieve retriever_docs = retriever_node(text=query) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1097, in call raise e from None File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1088, in call output = self.fl.exec(func, args, kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\backends\base.py", line 151, in exec return run(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 144, in call raise e from None File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 141, in call _output = self.next_call(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 117, in call return self.next_call(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1017, in _runx return self.run(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\ktem\index\file\pipelines.py", line 162, in run docs = self.vector_retrieval(text=text, top_k=self.top_k, **retrieval_kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1261, in exec return child(*args, **kwargs, fl_runstates=fl_runstates) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1097, in call raise e from None File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1088, in call output = self.fl.exec(func, args, kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\backends\base.py", line 151, in exec return run(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 144, in call raise e from None File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 141, in call _output = self.next_call(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 117, in call return self.next_call(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1017, in _runx return self.run(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\kotaemon\indices\vectorindex.py", line 188, in run emb = self.embedding(text)[0].embedding File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1261, in exec return child(*args, **kwargs, fl_runstates=fl_runstates) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1675, in call return self.create_callable(getattr(self.ff_original_obj, "call"))( File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1663, in wrapper raise e from None File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1661, in wrapper output = callable_obj(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 144, in call raise e from None File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 141, in call output = self.next_call(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 117, in call return self.next_call(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1261, in exec return child(*args, **kwargs, fl_runstates=fl_runstates) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1097, in call raise e from None File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1088, in call output = self.fl.exec(func, args, kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\backends\base.py", line 151, in exec return run(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 144, in call raise e from None File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 141, in call output = self.next_call(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\middleware.py", line 117, in call return self.next_call(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\theflow\base.py", line 1017, in runx return self.run(*args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\kotaemon\embeddings\base.py", line 10, in run return self.invoke(text, *args, **kwargs) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\kotaemon\embeddings\openai.py", line 104, in invoke resp = self.openai_response(client, input=input, **kwargs).dict() File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\tenacity_init.py", line 289, in wrapped_f return self(f, *args, **kw) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\tenacity_init.py", line 379, in call do = self.iter(retry_state=retry_state) File "D:\20240916-Kotaemon\kotaemon-app\install_dir\env\lib\site-packages\tenacity_init.py", line 326, in iter raise retry_exc from fut.exception() tenacity.RetryError: RetryError[<Future at 0x1e286782500 state=finished raised AuthenticationError>]

[mmcc007]

I get the same AuthenticationError. When running locally it works fine. When I deploy to a AWS EC2, I get the AuthenticationError

[mmcc007]

I get the same AuthenticationError. When running locally it works fine. When I deploy to a AWS EC2, I get the AuthenticationError

Since I'm only using cohere.com it was fixed by making it the default for LLM and embedding. (I think it prompts for that in the local install, but not on the remote install for whatever reason)