torchchat
torchchat copied to clipboard
pytorch
Run PyTorch LLMs locally on servers, desktop and mobile
### 🐛 Describe the bug I am running an Arch Linux system with a 4090/3090 w/ and up-to-date CUDA 12.5 (`Build cuda_12.5.r12.5/compiler.34385749_0`) I have created a new mamba env for...
The command `python3 torchchat.py where llama3` fails quietly presumably because I might not have the HF Token configured. I assumed the code was broken, though because I got a backtrace...
Implement JSON formatted responses using OpenAI API types for server completion requests. Rather than giving single tokens at a time, the server will respond with a JSON following the API...
### 🐛 Describe the bug I am trying to build the llama runner natively on a rasperry pi following the torchchat description, and the post at https://dev-discuss.pytorch.org/t/run-llama3-8b-on-a-raspberry-pi-5-with-executorch/2048 I was able...
### 🐛 Describe the bug For example `Memory used: 0.00 GB` ``` > python3 torchchat.py generate llama3.1 --dso-path exportedModels/llama3.1.so --prompt "Hello my name is" NumExpr defaulting to 10 threads. PyTorch...
Hi, I'm trying out the torchchat right now, started the streamlit application with llama3 model  I just texted Hi !! - Why is this text generation behaviour unusal ,...
### 🐛 Describe the bug `pip install` fails because it could not find `torch` even though, it's present in the environment: ``` % pip install git+https://github.com/pytorch/ao.git@d36de1b144b73bf753bd082109c2b5d0141abd5b Collecting git+https://github.com/pytorch/ao.git@d36de1b144b73bf753bd082109c2b5d0141abd5b Cloning https://github.com/pytorch/ao.git...
Adds set -x for all installation commands in install_requirements.sh so that users can see what's actually being installed and can help debug when they run into any issues. NOTE: This...
This PR aims to enable Llava in torchchat. TODOs: - [x] Create Model and ModelArgs as model definition entrances - [x] Support model definition with multiple transformers - [ ]...
### 🚀 The feature, motivation and pitch torchchat currently uses the hf hub which has it's own model cache, torchchat copies it into it's own model directory so you end...