azure-search-openai-demo icon indicating copy to clipboard operation
azure-search-openai-demo copied to clipboard

The prepdocs remove flag/instructions are confusing

Open notanaha opened this issue 1 year ago • 6 comments

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ x ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

As described in Removing documents section, added --remove flag in scripts/prepdocs.ps1, but all the documents were removed from the index, but those in blob storage were left.

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

azd version?

run azd version and copy paste here. img2

Versions

Mention any other details that might be useful


Thanks! We'll be in touch soon.

notanaha avatar Apr 09 '24 07:04 notanaha

You're saying that it did remove the blobs from blob storage, but did not remove them from the search index?

pamelafox avatar Apr 09 '24 22:04 pamelafox

No, opposite meaning, i.e., index was cleared, but blog wasn't. The more serious issue would be that while I used --remove flag, all data was removed from the index. Please not I didn't used --removeall flag.

notanaha avatar Apr 10 '24 00:04 notanaha

Did you modify the dataArg as well? I can't see from your screenshot.

pamelafox avatar Apr 10 '24 00:04 pamelafox

No, I just followed the instruction. If my modification was incorrect, please add such instruction.

image

notanaha avatar Apr 10 '24 12:04 notanaha

The second part of the instruction is to replace "/data/*" with "/data/your-filename.pdf", so you need to do that as well, otherwise it operates on all of "/data/*".

I think we need to make it possible to just specify --remove and --removeall from the command line, so that you don't have to mess with that file, as it is fairly confusing to wade through all that. We could also make it so that "remove" would only operate on a single file, but that would mean you couldn't use it with globs.

pamelafox avatar Apr 10 '24 19:04 pamelafox

So, with remove flag, did I need to move files from /data folder? I tried it prior to open this issue, but didn't work as intended like so. I agree that this is concerned about the explanation matter.

notanaha avatar Apr 13 '24 06:04 notanaha