semantic-kernel icon indicating copy to clipboard operation
semantic-kernel copied to clipboard

Python/Local Hugging Face Inference for Completions and Embeddings

Open awharrison-28 opened this issue 2 years ago • 0 comments

Motivation and Context

This PR introduces native python support for Hugging Face models that can: complete text, generate new text, summarize, and that generate embeddings. Currently only supports downloading the models locally from the HF model hub. Future plans include supporting the HF inference API as well.

Description

  • Added 2 services: hf_text_completion and hf_text_embedding
  • hf_text_completion supports the following tasks: text-generation, text2text-generation, and summarization
  • hf_text_embedding supports any model supported by the sentence-transformers pip package
  • Added dependencies: pytorch, transformers, sentence-transformers to requirements.txt and poetry.lock
  • fixed typo: get_embedding_service_service_id -> get_embedding_service_id
  • Added a number of integration tests for supported HF models

Contribution Checklist

  • [x] The code builds clean without any errors or warnings
  • [x] The PR follows SK Contribution Guidelines (https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
  • [x] The code follows the .NET coding conventions (https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions) verified with dotnet format
  • [x] All unit tests pass, and I have added new tests where possible
  • [x] I didn't break anyone :smile:

awharrison-28 avatar Apr 26 '23 00:04 awharrison-28