vik

Sorry about that! Can you try changing this line to force the model to load on CPU? Looks like I need to do some more testing on MPS. https://github.com/vikhyat/moondream/blob/main/moondream/text_model.py#L24 ```...

Agreed, it seems to work when it's fully on MPS or fully on CPU. Need to figure out why the disk offload is breaking it

Going to close this issue, we're no longer using disk offload in this repository.

Hi @monatis - would appreciate your support a ton! The vision encoder is SigLIP with the attention pool removed: ``` model.visual.trunk.attn_pool = nn.Identity() ``` Additionally the convolution used to create...

I retrained the model to use a LLaVA style MLP projector, running into another blocker because it appears clip.cpp doesn't support having a bias on the patch embedding: https://github.com/ggerganov/llama.cpp/blob/master/examples/llava/clip.cpp#L434

I'm training a new version of the model that will have an open license, will update the license in the repository to Apache 2.0 in a few days when it's...

Excellent news - though it looks like we're still limited by the OpenAI terms of use? Llama stipulation doesn't apply here since this is based on Phi-1.5.

I think I'll swap out the LLaVA dataset regardless, don't want people to have to worry about taking on legal risk. Pretty sure I can do it without hurting performance....

License is updated - Apache 2.0. Note that this applies to the code in this repository, and the moondream2 weights. moondream1 weights still have the old license. Closing this issue,...

Thank you for trying it out! Is the ask here to use the GPU when available, or are you seeing a failure when you try this?