pyllama

pyllama copied to clipboard

How to run llama_quant without downloading models from huggingface ?

1

How to run llama_quant without downloading model files from hugginface ? I tried ckpt_dir no luck I got the following error: "the following argument are required: dataset"

B2F

example.py FAILED

1

RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 261346) of binary: /usr/bin/python3

yangzhipeng1108

world size assertionerror

1

I try to make the 7B model on my single GPU server, and I have error: ``` Traceback (most recent call last): File "inference.py", line 82, in run( File "inference.py",...

sharlec

is py-itree available for Windows?

14

`hiq-python` depends on `py-itree`, but `py-itree` does not appear to have a windows compatible release.

vmajor

Model mismatch for 13B

3

CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2 webapp.py --ckpt_dir ../../../llama/ckpt/13B/ --tokenizer_path ../../../llama/ckpt/tokenizer.model `WARNING:torch.distributed.run: ***************************************** Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please...

BOB603049648

Model does not split for 65B

5

I have 8 80G A100 GPUs. I can't run correctly for the project， while I can run official example.py. ``` torchrun --nproc_per_node 8 webapp.py --ckpt_dir /nvme/syx/llama/model/65B/65B/ --tokenizer_path /nvme/syx/ll ama/model/tokenizer.model ```...

YixinSong-e

Error when download models

5

Hi, Here is the errors when i try download from my mac M1: ``` python3 -m llama.download Traceback (most recent call last): File "/Users/paulo/Library/Python/3.9/lib/python/site-packages/itree/__init__.py", line 5, in from . import...

paulocoutinhox

M1 inference

1

I can run original `llama` (with minimal changes) and `llama.cpp` on my Macbook M1 Max. I think it would be great if I could use `pyllama` with the same hardware...

zmactep

Struggle with training LLaMA with a single GPU using both PT v1 and v2

4

Hi, I love your code base and want to try how to train the LLaMA with a single GPU. This code I use is here https://github.com/juncongmoo/pyllama/blob/main/llama/model_single.py. However, I struggle with...

linhduongtuan

Running web_server.py on Multi GPU instance.

6

Hello. I started 8x A100 80G instance in Google Cloud and can't start 65B model: ``` root@llama:/pyllama/apps/flask# python3 web_server.py --ckpt_dir /var/llama/65B --tokenizer_path /var/llama/tokenizer.model Traceback (most recent call last): File "/pyllama/apps/flask/web_server.py",...

maxpain