Could somebody guide me how to fine-tune with fill-in-middle task based on StarCoderBase?

[FlyingPiggyKing]

trafficstars

As titled, I have thoudsands of SQL files, I wish to fine tune the base model with these sqls to achieve FIM task.

[loubnabnl]

You can check this repository which supports fine-tuning with FIM https://github.com/loubnabnl/santacoder-finetuning