Blaizzy
libc++abi: terminating due to uncaught exception of type std::runtime_error: Attempting to allocate too much memory
vlm % python mytest.py
Model: mlx-community/Qwen2-VL-72B-Instruct-8bit
Fetching 25 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 16513.01it/s]
Fetching 25 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 10979.85it/s]
Image: 20241116-164155_L1010474-Edit.jpg
==========
Image: /Users/xxx/Pictures/Processed/20241116-164155_L1010474-Edit.jpg
Prompt: <|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be searched for easily<|vision_start|><|image_pad|><|vision_end|><|im_end|>
<|im_start|>assistant
libc++abi: terminating due to uncaught exception of type std::runtime_error: Attempting to allocate 135383101952 bytes which is greater than the maximum allowed buffer size of 77309411328 bytes.
zsh: abort python mytest.py
/opt/homebrew/Caskroom/miniconda/base/envs/mlx/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
I got a similar error with Qwen/Qwen2-VL-7B-Instruct
That's a really large image my friend.
Please use resize-shape to at least 1K to 2k
Well, it is the size that comes out of cameras these days!
Where does resize-shape come from? (ie, how do I use it? Is it a Pillow thing, or a vlm thing, or ...?)
I don't really want to edit the original jpeg., and generate only takes the file path. And a good few of the other models accept this size of picture. (I think that I submitted some transcripts last week.)
Where does resize-shape come from? (ie, how do I use it? Is it a Pillow thing, or a vlm thing, or ...?)
It's a mlx-vlm thing that uses pillow to resize your image while maintaining the aspect ratio.
The problem here is that the image processing operation is very taxing on your resources and some models are built in a way that makes things worst for on-device inference.
If you want to maintain original image size you either will run out memory or will have slow inference.
Nevertheless, I'm working on improving this as we speak.