Anthony
Anthony
Go ahead and build on my draft PR or create your own, since I won't have the capacity to finish this one.
Actually I think the chat template for some thinking models was updated so that the prompt starts with a `` tag to force a thinking block. You'll need to accommodate...
~~I tried to factor out `RMSNorm`, since several models use it, but I'm having trouble making it accessible everywhere.~~ Edit: This is now fixed.
I fixed some more errors, and now there are just a few errors and TODO comments left, which I'll need help resolving.
The config is working, although it can probably be improved (see TODO comment and possibly remove unneeded properties). But now I'm getting the following error when I run the model:...
Something is wrong with the image tokens that are being inserted by the tokenizer vs. what's expected in this implementation vs. what's in the config vs. what I see in...
Got it. Do you want to take on that part? I don't know if I'll be able to add anything else today.
I think I've replicated the processing code from transformers, and the model is now generating text without any errors, but the text is garbled. The debug output looks correct to...
I added the text-only model, but it's also generating strange output: ``` ```
Thanks for the tips, @davidkoski. I tried generating with text only as input to the vision model, and I got this output: ``` ``` I think I'll leave this for...