sglang
sglang copied to clipboard
Repetitive zeros for structuring JSON (float type)
There has been a number of times, where the float type will get an unlimited amount of zeros being generated.
Any idea what could be the cause? I am thinking it is most probably the issue with token representation along with LLM tendency. This also happens with whitespaces, it can go on a never-ending whitespaces too.
Most probably fixable at outlines, so I raised it there: https://github.com/outlines-dev/outlines/issues/847