dbddv01

I have no tools to measure it. So i handle it manually. I prompted 'please produce 500 tokens story of starwars' then copy pasted the text produced to count the...

Ok thx ! now with the /stats. USER: /stats **encode: 35.4 tok/s, decode: 16.7 tok/s** USER: continue ASSISTANT: In this epic (... removed ...) USER: /stats **encode: 11.1 tok/s, decode:...

For the record, Intel® Core™ i5-7600K CPU @ 3.80GHz × 4, 16Gb ram, under Ubuntu, model 13B runs with acceptable response time. Note that as mentioned by previous comments, -t...