Handle BlobDeleted events from Event Grid

[cecheta]

Describe the bug

The EventGrid topic sends messages to the Storage queue both when a blob is created or deleted, however the batch_push_results function is set up to only handle blob created items. When an item is deleted from the storage container, the corresponding message in the queue ends up in the poison queue, as it cannot be processed.

Expected behavior

One of the following:

• The Azure function does not respond to delete events
• The function handles delete events appropriately, e.g. also deletes from Azure Search (this point would need further discussion)

---

Debugging information

Steps to reproduce

Steps to reproduce the behavior:

1. Upload item to storage container (it should be processed successfully)
2. Delete item from storage container
3. View batch_push_results Azure function logs
4. View doc-processing-poison queue

Logs

If applicable, add logs to help the engineer debug the problem.

2024-03-12T10:17:57Z   [Error]   Error adding embeddings for https://*********.blob.core.windows.net/documents/PressReleaseFY23Q4.docx?*******************: File is not a zip file

---

Tasks

To be filled in by the engineer picking up the issue

[adamdougal]

As part of this we should probably move the deleting from the index from the admin app to the function. That way we can delete from storage without using the admin app, in the same way we can add documents to blob storage directly without using the admin app.

[matthieugi]

I like the idea of moving the business logic to the function. Did somebody start to work on this ?

[ross-p-smith]

Nobody is working on this - please feel free to take it . . .