outlines icon indicating copy to clipboard operation
outlines copied to clipboard

Make Outlines' cache reusable across startup by making the cache's key as string

Open Lap1n opened this issue 5 months ago • 22 comments

When using vllm and outlines, in our use cases, it seems that the diskcache functionality is not working. Every time the server is started, it doesn't seem to be able to reuse the previously computed FSM cache.

One way that can fix this issue is to serialize the cache key object as a string.

See this issue.

Lap1n avatar Sep 04 '24 19:09 Lap1n