Ludobico

I recomend getting at least 20GB VRAM

> Had the same when training on 2 gpus, using just python finetune.py > > Got it running on both using torchrun > > `WORLD_SIZE=2 CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node=2 --master_port=1234 finetune.py`...

try to modify pretrained model and tokenizer in finetune.py ```py model = LLaMAForCausalLM.from_pretrained( "decapoda-research/llama-7b-hf", load_in_8bit=True, device_map=device_map, ) tokenizer = LLaMATokenizer.from_pretrained( "decapoda-research/llama-7b-hf", add_eos_token=True ) ```

if your code is written like this ```py assert ( "LlamaTokenizer" in transformers._import_structure["models.llama"] ), "LLaMA is now in HuggingFace's main branch.\nPlease reinstall it: pip uninstall transformers && pip install git+https://github.com/huggingface/transformers.git"...