Armin Klačar

@manmay-nakhashi Quick question, Every tokenizer I train results in gibberish after training using DLAS. I am trying to create tokenizer that works, any tips here ? Only tokenizer that works...

@manmay-nakhashi What would be needed if we want to use larger tokenizer with DLAS and Tortoise ? e.g. 512 token ?