Shrey Gupta

icon indicating an email [email protected]

icon location Mountain View, CA icon location Sometimes I write Python code and other times it runs.

Results 2 comments of Shrey Gupta

Unable to do inference on Falcon-40b fine-tuned using LORA

No, I just leave the strategy to "auto" which essentially means that I am not using "FSDP". Also, I tried using both 1 and multiple devices but the model weights...

Unable to do inference on Falcon-40b fine-tuned using LORA

I tried using FSDP and using 8 80GB A100 GPUs but still, it gets stuck while using the generate/lora.py script. The commands I am using are: 1. python generate/lora.py --lora_path...