Brett Young
Brett Young
What are the GPU requirements for fine tuning @hadipash ?
@hadipash thats not bad really. You tested it with sharding across multiple GPU's (eg multiple GPU's with
i got a tutorial coming soon, stay tuned.
Thanks for the quick response! I am writing an article for wandb comparing the two. I disabled activation checkpointing like you mentioned, and also the only real change between the...
gotcha. I'll make sure and make note of this!
https://wandb.ai/byyoung3/mlnews2/reports/Fine-Tuning-Llama-3-with-LoRA-TorchTune-vs-Huggingface--Vmlldzo3NjE3NzAz?utm_campaign=Fine-Tuning+Llama-3&utm_source=twitter&utm_medium=social&utm_content=Llama3 Heres the article. Hope you enjoy!
https://wandb.ai/byyoung3/ml-news?nw=nwuserbyyoung3
yeah sorry about the delay, I lost track of this. You are correct, I simply click the add to report button.
Sorry again for the delay, I dont get notifications for github issues..
ml-news project doesn't seem to show. I created a ml-news2 as a workaround for now.