shoco icon indicating copy to clipboard operation
shoco copied to clipboard

Compress tokenized strings

Open eried opened this issue 4 years ago • 0 comments

Hi,

For compressing tokenized strings similar to:

//HEADER//CODE1//decimals_here//CODE4//decimals_here//CODE3//decimals_here//CODE5//decimals_here//ENDING//

Do I need to split them on the slashes for it to be effective? I ran the model generator with a giant example with lots of these and the output model does not actually compresses anything, I see the last array fill with 00s

Any hint? thanks!

eried avatar Jan 26 '21 15:01 eried