sglang icon indicating copy to clipboard operation
sglang copied to clipboard

Repetitive zeros for structuring JSON (float type)

Open timothylimyl opened this issue 1 year ago • 0 comments

There has been a number of times, where the float type will get an unlimited amount of zeros being generated.

Any idea what could be the cause? I am thinking it is most probably the issue with token representation along with LLM tendency. This also happens with whitespaces, it can go on a never-ending whitespaces too.

Most probably fixable at outlines, so I raised it there: https://github.com/outlines-dev/outlines/issues/847

timothylimyl avatar Apr 30 '24 02:04 timothylimyl