chat-with-your-data-solution-accelerator icon indicating copy to clipboard operation
chat-with-your-data-solution-accelerator copied to clipboard

feat: Include vectorized text in search queries

Open cecheta opened this issue 1 year ago • 2 comments

Required by #750

Purpose

  • If advanced image processing is enabled, then the question will be vectorised with the same model as used for vectorising images
  • A second vector query is included when retrieving documents from AI search
  • Note that this is only for non-integrated vectorisation
  • Any retrieved images are not yet passed to the LLM, however the caption of the image is

Does this introduce a breaking change?

  • [ ] Yes
  • [x] No

How to Test

  • Enable advanced image processing
  • Deploy
  • Upload images
  • Ask questions based on images

cecheta avatar May 20 '24 15:05 cecheta

Coverage

Coverage Report •
FileStmtsMissCoverMissing
code/backend/batch/utilities/helpers
   azure_computer_vision_client.py530100% 
   azure_search_helper.py570100% 
code/backend/batch/utilities/helpers/embedders
   push_embedder.py730100% 
code/backend/batch/utilities/search
   azure_search_handler.py62296%24, 30
   integrated_vectorization_search_handler.py68395%35, 135, 144
   search.py14192%16
   search_handler_base.py512354%16–18, 21–23, 27, 31, 35, 39, 43, 47, 51, 54–55, 57–59, 61–62, 65–66, 68
TOTAL246067272% 

Tests Skipped Failures Errors Time
211 0 :zzz: 0 :x: 0 :fire: 11.437s :stopwatch:

github-actions[bot] avatar May 20 '24 15:05 github-actions[bot]

lgtm

I jumped the gun 😬

cecheta avatar May 20 '24 15:05 cecheta