Émile Fugulin
Émile Fugulin
Just a little ping to get some feedback, thanks!
Another ping just to make sure I am not forgotten @smusali @jakedipity
@smusali Let me know if you prefer that I copy the props from the parent, otherwsie feel free to merge.
Dear @smusali, I hope you will the changes to you liking. Please see my comment above concerning a potential bug with defaults. I am also starting to consider that maybe...
@smusali I think it would be best if you take over, I do not have the bandwidth to go back and forth over this issue anymore. Thanks!
Like I said in the Pr I do not have the bandwidth nor the interest to do another revision of it. It is better that someone from logdna does it...
OK I will try to take a look if I have time.
We also see this in production, investigating @zurawiki There should be a way to re-use the BPE for sure in any case.
We did some investigation and the lib does a lot of allocations, specially around regexes (fancy_regex) in _encode_native. Given a 1-2MB input it can easily bubble up to 50mb of...
We did mainly manual tests with a codepath that had `tiktoken::num_tokens_from_messages`. I used dhat (https://docs.rs/dhat/latest/dhat/) and you can build automated memory tests with it. What we have seen is that...