azure-search-openai-demo icon indicating copy to clipboard operation
azure-search-openai-demo copied to clipboard

Don't skip files if Index was deleted

Open ThomasLambertES opened this issue 2 years ago • 4 comments
trafficstars

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ x] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Delete existing index, then rerun azd up. It will recreate index but leave empty. I deleted manually the .md5, but maybe a check would be, if index not exist, rerun all files

Any log messages given by the failure

Skipping ./data/,,,,,,,,,,.pdf, no changes detected.

Expected/desired behavior

No skipping files if the index was deleted (in my clease to cleanup the data)

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

azd version?

run azd version and copy paste here.

Versions

Mention any other details that might be useful


Thanks! We'll be in touch soon.

ThomasLambertES avatar Oct 26 '23 06:10 ThomasLambertES

thanks for the suggestion. We should implement this once #862 is merged because that PR has rewritten the prepdocs script.

Suggested approach

  1. Create index if exists returns a signal if the index has been recreated
  2. The signal is used as a flag to upload (e.g. force reupload)

tonybaloney avatar Oct 26 '23 07:10 tonybaloney

Related issues:

  • Running --removeall leaves the md5 files (but probably shouldnt)
  • Switching azd environments leaves the md5 files (I have no solution for that right now)

pamelafox avatar Oct 26 '23 23:10 pamelafox

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.

github-actions[bot] avatar Jan 05 '24 01:01 github-actions[bot]

I am removing the "stale" label since this is still an issue with the current implementation. We hope to move prepdocs to a custom function on AI Search soon, away from the md5 files approach.

pamelafox avatar Sep 30 '25 00:09 pamelafox