Sindre Rotnes Erichsen

Can reptition penalty be utilized in SynCode?

1

Hi! Can the huggingface's 'reptition penalty' parameter be utlized with SynCode? My output with grammar_mask is often stuck in generating high probability loops.

Grammar_mask vs grammar_strict mode

1

Hi :) Can someone please tell me the difference between the grammar_mask and grammar_strict mode? What are the difference in calculation of tokens etc. I am using constrained decoding for...

Option to use flash_attention

1

It would be very usefull to have the option to use flash attention to increase speed and lower memory usage.