Kenneth Wong

Results 6 comments of Kenneth Wong

Pydantic error: extra fields not permitted for chain_type_kwargs

Can someone please provide the version of langchian that works? I tried it on version 0.0.340 and it still failed. llm = Ollama(model="codellama:7b", temperature=0.1, callback_manager=CallbackManager([StreamingStdOutCallbackHandler()])) a_chain = RetrievalQA.from_chain_type( llm, chain_type="map_reduce",...

`CUDA out of memory` when using long prompts and context sizes

Is there anyway to recover from this issue other than reboot the computer? Restarting the ollama server does not work for me. I am hoping to find an workaround until...

`CUDA out of memory` when using long prompts and context sizes

I kicked off the test yesterday at around Tue Mar 5 19:18:25 2024 and by this morning I see that cpu load is very high and gpu load is low....

`CUDA out of memory` when using long prompts and context sizes

Able to reproduce the issue on: 0.1.28 grep "CPU only" outollama_11.txt Mar 06 10:26:15 kenneth-MS-7E06 ollama[2968]: time=2024-03-06T10:26:15.445-08:00 level=INFO source=llm.go:111 msg="not enough vram available, falling back to CPU only" Mar 06...

Not enough vram available, falling back to CPU only, AMD 16 GB VRAM

I ran into the same issue while running a set of tests using ollama version is 0.1.25. Note each test loads a different LLM and this is reproduceable but only...

Not enough vram available, falling back to CPU only, AMD 16 GB VRAM

Reproduce this issue on ubuntu 22.04.1 with RTX 4070 TI uname -a Linux kenneth-MS-7E06 6.5.0-21-generic #21~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 9 13:32:52 UTC 2 x86_64 x86_64 x86_64 GNU/Linux kenneth@kenneth-MS-7E06:~$ ollama...