chris-aeviator

I know this is/sounds a bit far fetched, but as a user I expect the health command check for not only the presence of a binary but also if the...

> I have the 65B (120GB) model working at 60 seconds/token on: > > ``` > GPU: Nvidia RTX 2070 super (8GB vram, 5946MB in use, only 18% utilization) >...

note: take a look at charm.sh for an kick ass TUI framework.

Could contribute a simple system to accept/decline/upsert the entries (Imagine each card in this kanban board beeing one instruction -> answer pair each)  Instead of category it would be...

here's a wip/ poc of loading an adapter model via Peft https://github.com/ohmytofu-ai/tgi-angryface/commit/aba56c1343aa77ba0a07d14327d3e52736334308 . This is adressing https://github.com/huggingface/text-generation-inference/issues/896#issuecomment-1691770960). I cannot test hot-swapping right now since I'm trying to finish my LlamaModel...

Good idea with the workflow. Maybe a sanity check for the folders and .bin files being present?

@edoakes I think this does not test real gradio queue usage. When trying to load a queue, gradio fails with a dns resolution error. ``` def gradio_summarizer_builder(): def run_tasks(text, progress=gr.Progress()):...

@stoperro according to https://github.com/mosaicml/llm-foundry/pull/416 just use the ordinary peft code (huggingface has ready to go PEFT notebooks) or with llm-foundry add 

python 3.11 debian affected, all projects relying on faiss fail to install ```` WARNING: Generating metadata for package faiss-gpu produced metadata for project name faiss-cpu. Fix your #egg=faiss-gpu fragments. Discarding...

``` ` version: 3.9 services: chat-registration: image: zeratax/matrix-registration:v0.9.1 volumes: - ./:/data command: --config-path /data/matrix_registration/config.yaml serve ``` ```