moondream icon indicating copy to clipboard operation
moondream copied to clipboard

PIL Fallback

Open EthanReid opened this issue 7 months ago • 2 comments

Some users have had trouble with Pyvips. Added a fallback to use PIL inside of image_crops.py. When using PIL, there is a slight wall clock penalty of about 0.2 second (tested on m3 MBA).

Added a check before "torch._dynamo.mark_dynamic" which prevents it from running on Mac.

These changes have been tested on Mac and Ubuntu.

EthanReid avatar May 07 '25 01:05 EthanReid

Added a check before "torch._dynamo.mark_dynamic" which prevents it from running on Mac.

Why does this need to be blocked?

vikhyat avatar May 07 '25 01:05 vikhyat

When torch imports dynamo it runs a test floor divide. When using set_default_device this causes a runtime error if it was set to MPS ("RuntimeError: Placeholder storage has not been allocated on MPS device!"). If default is CPU, the error is not thrown. Removed my conditional use of torch._dynamo and instead removed set_default_device and instead moved the model to device.

EthanReid avatar May 07 '25 02:05 EthanReid