litellm
litellm copied to clipboard
BerriAI
Update ollama.py for image handling
ollama wants plain base64 jpeg/png images, and some clients send dataURI and/or webp. Remove prefixes and convert all non-jpeg/png images to jpeg.
Fixes https://github.com/BerriAI/litellm/issues/1427, https://github.com/BerriAI/litellm/issues/2661
The latest updates on your projects. Learn more about Vercel for Git ↗︎
| Name | Status | Preview | Comments | Updated (UTC) |
|---|---|---|---|---|
| litellm | ✅ Ready (Inspect) | Visit Preview | 💬 Add feedback | May 28, 2024 3:00pm |
![vercel[bot] avatar](https://avatars.githubusercontent.com/in/8329?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hi Jakob,
base64 encoded data shouldn't contain commas, it's restricted to the character set [A-Za-z0-9/+=]: https://en.wikipedia.org/wiki/Base64
On Thu, 11 Apr 2024 at 05:10, Jakob @.***> wrote:
I think this implementation is flawed since you're doing image.split(",") for the comma you're expecting at the beginning, but the base64 encoded data itself can also contain commas.
— Reply to this email directly, view it on GitHub https://github.com/BerriAI/litellm/pull/2888#issuecomment-2048855410, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADSBEJVKJCF77L263ZUULTTY4X5LTAVCNFSM6AAAAABF3HOAX6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBYHA2TKNBRGA . You are receiving this because you authored the thread.Message ID: @.***>
Yeah realized that afterwards and deleted my comment :) sorry about that. I was confusing "," with "/"
@rick-github thanks for this - can you add a test with a mock call for this to test_completions.py we've had a couple regressions with merged ollama pr's recently.
place to add test - https://github.com/BerriAI/litellm/blob/b29994a0ae0fc496f182167a8c8d4a1b689712fd/litellm/tests/test_completion.py#L4
Example of a mock call - https://github.com/BerriAI/litellm/blob/b29994a0ae0fc496f182167a8c8d4a1b689712fd/litellm/tests/test_completion.py#L1314
Thanks @rick-github just share a screenshot of it passing testing for your test + tests in test_ollama.py and we should be good to merge!
Thank you again for your awesome work here