Gerben van V

The code in `model.py` essentially performs: ``` n_bins = 2^n_bits_x x = floor(x / 2^(8 - n_bits_x)) x = x / n_bins - 0.5 ``` Which can be more easily...

It's an optimization: `log(a) = log(1 / n_bins) = -log(n_bins)`

Giving that a normalizing flow gives you a correct log-likelihood of your data under your model it would be a shame to omit `c` even though technically not required for...

Some comments about the gist / PR: I think you should try to: - have the KV cache in **pinned** CPU memory at all times, - as well as reserve...

Cool, nice work btw and I will start with a static cache implementation. It's a bit easier to do and will be a good stepping stone to creating a dynamic...

Okay, I whipped something up :grin: Check https://gist.github.com/gerbenvv/282ed3c981a63ad71a301cdc1a705ef1 It's not finished yet, but a proof of concept: - Yet to add some docstrings - beam search re-indexing not optimized (haven't...

Yeah I will open one after I clean up the code a bit, have tested it with beamsearch and added some docstrings. It will probabably have some conflicts with the...

Opened PR but note that it's a WIP, it doesn't work properly as of yet (still debugging). Will move it out of WIP when ready to be reviewed which will...

Okay, I will create a new issue for this. Also: - Yes, I did do some benchmarks and they are looking pretty good! Will share them later in the new...