Darrin Hodges
Darrin Hodges
Thanks, it installed cleanly on Ubuntu 20.04. I do have it installed on a 22.04 instance but that was a two or three weeks ago, so perhaps something has changed...
Getting same error here.
some of the other parameters need to be adjusted for single gpu: < --num-layers 4 --embedding-dim 4096 \ < --world-size 1 Gets me: Initialize NCCLCommunicator: < pipeline_group_0 >; rank: 0...
It won't train on my 12GB GPU, it runs out of memory. It requires more VRAM than I currently have.
Thanks @orangetin , it starts but quickly runs out of memory. Thanks for the link, will check it out. thanks
I've tried it on a Intel(R) Xeon(R) E-2236 CPU @ 3.40GHz with 64G of RAM, was faster than my initial testing on an I5 with 32G RAM. when it does...
I tested on host (72G RAM, 36 cores AWS EC2 - c5.9xlarge), ingesting is super fast. On inference, I did not get the dreaded unknown token messages, so I assume...
GPU support will be awesome for inference, the ingestion is super fast as it is on CPU. Looking forward to trying it. thanks Darrin
Yes, I have the same issue, was saying that ingest works fine as it is on CPU.
I tried vicuna and got: ``` Using embedded DuckDB with persistence: data will be stored in: db gptj_model_load: loading model from 'models/wizard-vicuna-13B.ggmlv3.q4_1.bin' - please wait ... gptj_model_load: invalid model file...