Provide HF AutoModel interface

[QuietRocket]

I am interested in adding CTranslate2 support to LMQL similar to this PR which added autogptq support. In reality, it seems like the bulk of the work would have to be done in CTranslate2 to provide a high level HF Transformers "CTranslate2ForCausalLM" model interface.

I understand that the ctranslate2.Generator already provides very high level interfaces for language model text generation, but by providing the HF Transformers interface, CTranslate2 could be easily added to many more frameworks like LMQL and Microsoft's Guidance.

What do you think? Thank 😁

[arunpatro]

Seconded! The developers at https://github.com/normal-computing/outlines would also benefit with such support.

[QuietRocket]

Seems like other frontends are adding glue in their codebase to support ctranslate2 inference: https://github.com/Preemo-Inc/text-generation-inference/pull/1/files#diff-aae772a4b6c3433949bf62484be62e1c4c1af6af91276820697f71345d93dea7

I don't know how the licensing works, but might be worth getting/asking to get the relevant portion as a PR here.

[michaelfeil]

Seems like other frontends are adding glue in their codebase to support ctranslate2 inference: https://github.com/Preemo-Inc/text-generation-inference/pull/1/files#diff-aae772a4b6c3433949bf62484be62e1c4c1af6af91276820697f71345d93dea7

I don't know how the licensing works, but might be worth getting/asking to get the relevant portion as a PR here.

I developed this part. You are very welcome (from my side) to pick and integrate it in CTranslate2!