Martin Dudek
Martin Dudek
>> Use a Dict to lookup token indices. This allows removing Quicksort and binary search code. did you check performance of this change - i am playing around with Tokenizer...
Great to hear about NARLL, looking forward. I find the benchmarks also surprising to the extend that i think there must be a mistake in the way i implemented it...
Thanks Patrick for this clarification. I use `jax.block_until_ready` and pre compile but there are other aspects I will have to look into. Looking forward to work with your library.