absin-lt

Inference on multi-gpu

9

Tried to run: ``` torchrun --nproc_per_node 1 codellama/example_instructions.py \ --ckpt_dir /home/ubuntu/model/ \ --tokenizer_path /home/ubuntu/model/tokenizer.model \ --max_seq_len 4512 --max_batch_size 4 ``` I have a long prompt (4000 tokens). I have 4...