apple
refacto(silicon): Refactor Ferret LLM for Apple Silicon
Hello, I've encountered this bug. When running the model_point_cls_single_image.py file, there's an error message saying "probabilty tensor contains either ‘inf’,'nan' or element<0". The error statement is in the following code:
output_ids = model.generate( input_ids, images=image_tensor.unsqueeze(0).half().cuda(), do_sample=True, temperature=args.temperature, max_new_tokens=1024, num_beams=1, use_cache=True, stopping_criteria=[stopping_criteria])
I'm hoping to get your reply as soon as possible.
Hello,
Thank you for reaching out and for highlighting this issue. I want to clarify that the Apple Silicon port of this project only supports the Gradio interface. The specific code you are trying to run relies on CUDA, which, as you may know, is incompatible with Apple's architecture.
If you'd like to use the Gradio interface on your Apple Silicon Mac, you can follow the procedure described here: https://scalastic.io/en/ferret-apple-mac-llm/.
As for your request, "I'm hoping to get your reply as soon as possible"—consider this your express delivery 🚀.
🥲