exo icon indicating copy to clipboard operation
exo copied to clipboard
Published 12 hours ago • verified publisher icon exo-explore

Issue with loading weights

Open JohnLee1360 opened this issue 1 year ago • 4 comments

I am trying to use two MacBooks to run a Llama 8B model, but I can't load weights from the model to inference and stuck in 0 progress... Screenshot 2024-09-24 at 12 03 28

Here's info of my equipments: node1: MacBook Air 16GB with M3 chip node2: MacBook Pro 16GB with M1 chip(Intel based)

Since my machine resource is limited, all of my machines run in tinygrad inference engine rather than in MLX. And I also wonder why my MacBook Pro shows 0TFLOPS?

I will be so appreciative if someone can offer me help~

JohnLee1360 avatar Sep 24 '24 04:09 JohnLee1360

Can you try running with SUPPORT_BF16=0

AlexCheema avatar Sep 24 '24 15:09 AlexCheema

Thanks for the reply! I assume that SUPPORT_BF16=0 means smaller quantized weights accuracy, right? I gave it a shot, but it doesn't work. Maybe the problem stems from the different architecture of chips?

JohnLee1360 avatar Sep 25 '24 08:09 JohnLee1360

Thanks for the reply! I assume that SUPPORT_BF16=0 means smaller quantized weights accuracy, right? I gave it a shot, but it doesn't work. Maybe the problem stems from the different architecture of chips?

No no this should definitely work. Can you run with DEBUG=6. Are there any errors?

AlexCheema avatar Sep 30 '24 17:09 AlexCheema

Thanks!!! DEBUG flag helps a lot! But I wanna know different DEBUG flags stand for what? lol

JohnLee1360 avatar Oct 07 '24 09:10 JohnLee1360