AdalFlow
AdalFlow copied to clipboard
Changed transformers_client design so it can support most (all?) HuggingFace models
New design
TransformerClient into 3 client classes:
- TransformerEmbeddingModelClient
- Merger of TransformerEmbedder and TransformerClient
- TransformerRerankerModelClient
- Merger of TransformerReranker and TransformerClient
- TransformerLLMModelClient
- Merger of TransformerLLM and TransformerClient
Pros:
- Abstracts model/tokenizer initialisation as well as inference code
- Hence, most of the time, the only customization needed is to change some parameters during client instantiation
- Works with many different model architectures (example here)
- Make the code easier to read and more maintainable since each class takes care of only one client
- Easier to customize for users as it removes the need for model SDKs
- Removes the need to check whether a model is supported or not
Cons:
- The tradeoff here would be to couple inference code with the response and api kwargs handling.