Brandon T. Willard comments

Results 522 comments of


                                            Brandon T. Willard

Python 3.12 import failure: `ModuleNotFoundError: No module named 'numpy.distutils'`

Sorry, this project only supports Python

The logic of the disequality constraint with multiple variables is incorrect

Thanks for the report! From a quick read, it does look like a genuine bug. For reference, the constraint approach used here is based on the following fantastic paper by...

Make Outlines' cache reusable across startup by making the cache's key as string

> It seems that the issue is with using class method as the function to cache (vLLM use it similarly). What is it about a class method that's causing this?...

Make Outlines' cache reusable across startup by making the cache's key as string

> That is why it is most likely safer to serialize the args as a string instead of a binary key. So in short, I was able to reproduce the...

Make Outlines' cache reusable across startup by making the cache's key as string

> I see, but it is still better than the current implementation which won't use as well the underlying serialization implementations and will instead store it as a binary python...

Make Outlines' cache reusable across startup by making the cache's key as string

> Another alternative to quickly fix this would be to modify the usage of outlines's cache in vLLM. That sounds like the best approach, because—otherwise—this becomes a feature request to...

Make Outlines' cache reusable across startup by making the cache's key as string

> @brandonwillard seems like the error is not related to class methods after all. I've tried to extract the classmethod into an external function and it still doesn't work in...

Make Outlines' cache reusable across startup by making the cache's key as string

> I wonder if it could be related to [Diskcache's caveats of the default key serialization implementation](https://grantjenks.com/docs/diskcache/tutorial.html#caveats). This mostly emphasises the need to implement consistent serialization for the objects involved;...

Make Outlines' cache reusable across startup by making the cache's key as string

@Lap1n, I just noticed that `cache` is being applied to a `tokenizer` argument in `vllm` that's typed as `PreTrainedTokenizerBase`. In `outlines`, the `tokenizer` arguments are of `Tokenizer` type, and, more...

Make Outlines' cache reusable across startup by making the cache's key as string

> Thanks for investigating btw! Thank you for helping find and debug this! Also, don't hesitate to keep us in the loop regarding the vLLM side of things.