AlpinDale
AlpinDale
Sorry I totally forgot to get back to you 😅 Seems like this can be solved by setting `--guided-decoding-backend outlines`. Not sure what it is about lmfe that breaks mistral...
Thanks for the review @Yard1. It needs a bit more work, could probably use a mask and an unsqueeze for the factor/curve. I'll finish this up soon and add a...
@Yard1 Should be done, let me know if there's any issues. Correctness tests pass for the new sampler.
> @AlpinDale do you plan to continue this line of work? This sampler is quite outdated by now, so I'm not sure it'll be a useful addition anymore. There are...
It works, but will fail due to changed logit shapes when running without CFG.
Hi. GGUF kernels should theoretically work on AMD, but it's untested as I don't have regular access to AMD compute. Multi-gpu should work fine on AMD. Tensor parallelism will split...
@gitzaidi I've added range in #855. I did play around with using python lists, but I remember running into issues due to the batched nature of our samplers - we...
> @AlpinDale Thanks for your reactivity, very impressive ! I will look into it then. > > Also, do we agree that, as of now, the implementation considers the same...
Fixed the sequence breaker ID issue at PygmalionAI/aphrodite-engine@c6e0ae0515a7b6364a152fbe132747d94713f6e4
DRY should be faster now but still very slow. I'm attempting to write kernels to bypass this issue. Progress will be logged here: https://github.com/AlpinDale/dry_sampling_kernel