ml-ferret
ml-ferret copied to clipboard
FEAT add auto mps support
- Add MPS support with auto DEVICE selection between cuda, mps and cpu
- Applied PEP refacto to the 3 modified files (single quotes, lines max length etc)
Happy to make Apple's ferret work in my Macbook, even if that's a bit slow.
Tested with ferret-13b-v1-3
, Macbook M3 Max 14" GPU 40 Core, 64Go RAM, using
export PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 # to disable upper limit for memory allocations
featuring my own photo, thank you ferret, even if I don't see much birds.
Thanks @gucxa2-mYjnyb-wyjvyg
@Haotian-Zhang if this looks interesting to merge for you, or you'd need any modification, please let me know