ml-ferret icon indicating copy to clipboard operation
ml-ferret copied to clipboard

FEAT add auto mps support

Open paulasquin opened this issue 1 year ago • 1 comments

  • Add MPS support with auto DEVICE selection between cuda, mps and cpu
  • Applied PEP refacto to the 3 modified files (single quotes, lines max length etc)

Happy to make Apple's ferret work in my Macbook, even if that's a bit slow.

Tested with ferret-13b-v1-3, Macbook M3 Max 14" GPU 40 Core, 64Go RAM, using

export PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 # to disable upper limit for memory allocations

featuring my own photo, thank you ferret, even if I don't see much birds.

image

paulasquin avatar Jan 08 '24 14:01 paulasquin

Thanks @gucxa2-mYjnyb-wyjvyg

@Haotian-Zhang if this looks interesting to merge for you, or you'd need any modification, please let me know

paulasquin avatar Feb 07 '24 16:02 paulasquin