outlines
outlines copied to clipboard
Make Outlines' cache reusable across startup by making the cache's key as string
When using vllm and outlines, in our use cases, it seems that the diskcache functionality is not working. Every time the server is started, it doesn't seem to be able to reuse the previously computed FSM cache.
One way that can fix this issue is to serialize the cache key object as a string.
See this issue.