vikhyat
Response cut off early, still very good though
Excellent speed improvements with this, thank you for the great work!
I noticed quite a few results are being cut off mid sentence. Using the image at the top of this article, the description gets cut off early.
"The image features a modern and well-lit home office with a large desk situated in the center of the room. The desk is equipped with a computer monitor, keyboard, and mouse, creating a functional workspace. A chair is placed in front of the desk, providing a comfortable seating option for the user.
In addition to the main desk, there is a bookshelf filled with various books, adding a touch of organization and intellectual ambiance to the room. A potted plant is also present, adding a touch of greenery and life to the office.
A chair is placed in front of the desk, and a"
There's a max token limit of 128 here, maybe we should bump it up.
https://github.com/vikhyat/moondream/blob/main/moondream/moondream.py#L55
It could be a parameter like --prompt and use the current as default? Like --chars=
Or something like splitting the result and if the last sentence runs over the token limit, then try not to print out anything after the last detected period / full stop.
Bumped it up to 256: https://github.com/vikhyat/moondream/blob/main/moondream/moondream.py#L98
Can go up further if it still keeps cutting off.