elasticsearch-labs icon indicating copy to clipboard operation
elasticsearch-labs copied to clipboard

Add support for semantic search with the dense vector

Open daixque opened this issue 11 months ago • 0 comments

Intention

Current chatbot-rag-app is set up with only SparseVectorStrategy. I'd like to add DenseVectorStrategy because for those who speak non-English languages, DenseVectorStrategy with E5 models is good option.

I know this doesn't work for the data ingestion part, but it's great to have a dense vector option in the RAG example.

Changes

This pull request introduces several enhancements to the example-apps/chatbot-rag-app to support both sparse and dense vector strategies for Elasticsearch. The changes include updates to the environment configuration, the main chat API, and logging improvements.

Support for multiple vector strategies:

  • example-apps/chatbot-rag-app/api/chat.py: Added support for DenseVectorStrategy alongside the existing SparseVectorStrategy. Introduced new environment variables to configure the strategy type, model ID, vector field, and query field. [1] [2]

Environment configuration updates:

  • example-apps/chatbot-rag-app/env.example: Updated the example environment file to include configurations for both sparse and dense vector strategies, allowing users to easily switch between them.

Logging improvements:

  • example-apps/chatbot-rag-app/api/chat.py: Improved logging in the ask_question function to ensure that document metadata is logged only if the "name" field is present.

daixque avatar Nov 23 '24 08:11 daixque