CHesketh76
CHesketh76
### Describe the bug After setting up Agents and workflow with local endpoints. Getting this error message ```openai.OpenAIError: The api_key client option must be set either by passing api_key to...
### Checked other resources - [X] I added a very descriptive title to this issue. - [X] I searched the LangChain documentation with the integrated search. - [X] I used...
### Describe the bug After loading in GGUF model into RAM or vRAM (doesn't matter if it is split between the two). After selecting **Unload Model** in the UI the...
My work does not allow any Docker to run or be installed. Is there an option for me to run this in python environment?
Mamba has been showing very promising results for scaling and I was wonder how huge mamba + Unsloth could be by allowing consumer hardware to train + finetune Mamba.
### What model would you like? After creating a ModelFile for a custom model. Please add the ability to use curl to ping the model. ``` curl http://localhost:11434/api/chat -d '{...
After following the steps in the documentation: ```css -m /path/to/my/weights/mistral-7b-Q8_0.gguf --mmproj mistral-7b-mmproj-Q8_0.gguf --host 0.0.0.0 -ngl 9999 ... ``` I get this message: ```mistral-7b-mmproj-Q8_0.gguf: No such file or directory``` Is this...
### Describe the bug When given a long input (either from only user or from both user + assistant) the text will not truncation will not not remove older text...
### Describe the bug This happens when i run openllm. I am using a fresh ubutnu install with fresh venv running. ### To reproduce 1.) Create venv 2.) pip install...
I trying to setup the TensorRT-LLM on a google Colab environment, but after I run this line of code ``` !python3 ./scripts/build_wheel.py --trt_root /usr/local/tensorrt ``` I get this error message....