syncode SynCode for fill-in-the-middle models

First of all, thank you for this great work! I would like to know if Syncode can also work with fill-in-the-middle models? If yes, how?

May 28 '24 07:05 AzizCode92

Hi @AzizCode92,

SynCode doesn't support FIM models at this point. Do you have any specific application in your mind?

May 28 '24 11:05 shubhamugare

Thanks @shubhamugare. Not really. I was just wondering how Syncode would find the most probable next token if we have prefix, suffix parts alongside with grammar rules. Right now, testing it for a fim model, it looks like the model is following the grammar by heart. It starts always by the first terminal I define in my grammar. IMH, I think it would take the prefix into consideration and produce the next token from there and at the same time respect the grammar rules. It is a challenging task I guess 😅

May 28 '24 12:05 AzizCode92

Yeah, it is definitely not straightforward extension of what currently exists within SynCode. In theory, one can use the constraints-based on suffix (in addition to the constraint from the prefix which SynCode currently uses) to find what is an appropriate stopping criterion for the MIM model.

May 28 '24 17:05 shubhamugare