AdalFlow icon indicating copy to clipboard operation
AdalFlow copied to clipboard

Changed transformers_client design so it can support most (all?) HuggingFace models

Open adjahossoualexandre opened this issue 5 months ago • 0 comments

New design

TransformerClient into 3 client classes:

  1. TransformerEmbeddingModelClient
  • Merger of TransformerEmbedder and TransformerClient
  1. TransformerRerankerModelClient
  • Merger of TransformerReranker and TransformerClient
  1. TransformerLLMModelClient
  • Merger of TransformerLLM and TransformerClient

Pros:

  • Abstracts model/tokenizer initialisation as well as inference code
  • Hence, most of the time, the only customization needed is to change some parameters during client instantiation
  • Works with many different model architectures (example here)
  • Make the code easier to read and more maintainable since each class takes care of only one client
  • Easier to customize for users as it removes the need for model SDKs
  • Removes the need to check whether a model is supported or not

Cons:

  • The tradeoff here would be to couple inference code with the response and api kwargs handling.

adjahossoualexandre avatar Sep 11 '24 16:09 adjahossoualexandre