azure-search-openai-demo icon indicating copy to clipboard operation
azure-search-openai-demo copied to clipboard

Creating an index with an already existing data in azure blob storage

Open samin-batra opened this issue 1 year ago • 2 comments

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [x] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

I am setting up the app with azd up command by specifying a new index name but referencing an already existing azure blob storage container and account. However, I have noticed that the sample documents are getting uploaded to the azure blob container and are being used to create the index. In the source code, I am trying to use the integrated vectorization approach to Azure search but it seems to still refer the local files to index. Is there a chance to have the functionality to use the already existing files uploaded to Azure Blob Storage, without using the sample data? I see there's a local file strategy object which is used when no Azure datalake account is specified, however I don't have any Azure Data Lake, only Azure Blob Storage.

Any log messages given by the failure

Expected/desired behavior

Indexing should be done only on the uploaded and existing documents in Azure blob storage

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

azd version?

run azd version and copy paste here.

Versions

Mention any other details that might be useful


Thanks! We'll be in touch soon.

samin-batra avatar Apr 09 '24 09:04 samin-batra

Hello,

Any update on this? Seems like only Azure data lake storage is supported if I want to reference existing data sources.

Thank You,

samin-batra avatar Apr 10 '24 15:04 samin-batra

You're correct that we don't have support for the situation you describe. I think it should be possible to set up with some code tweaks, but I won't have the time to dig into that myself in the near future.

pamelafox avatar Apr 13 '24 17:04 pamelafox