Delete local entities that have been deleted/archived/unassigned from server

[seadowg]

~~Blocked by #6029~~

User stories

As a project administrator If I delete, archive or re-assign an entity, I expect it to be removed from offline clients' devices when they next update So that the entities they see reflect the current state of the project

As a data collector If I create and/or update an entity while offline, I expect it to remain available on my device until it's explicitly deleted, archived or re-assigned away from me

Context

When an entity is deleted on the server, it is not included in subsequent entity list updates. Currently, if an entity is created locally and then deleted on the server, that entity remains available in forms.

There are a few related scenarios:

1. The entity was created locally, came back to the device in an entity list update, was deleted on server: Entity is created/updated locally and then the relevant submissions are sent to the server, the device syncs and receives the new entity list (with the local entity duplicated), the entity is deleted on the server and then the device syncs and receives the new entity list again (without the local entity)
2. The entity was created locally and deleted on the server before it was included in an entity list update: The entity is created/updated locally and then the relevant submissions are sent to the server, the entity is then deleted on the server and then the device syncs and receives the new entity list (without the local entity)
3. The entity was created locally and an entity list update came in before that new entity was processed/approved: The entity is created/updated locally and then the relevant submissions are sent to the server. The server doesn't process the entity creation before the client requests a new entity list so the response doesn't include the local entity.

Scenarios 2 and 3 look identical to clients: they have a local entity that is not in the entity list update.

For scenario 2, we want the local entity deleted because it was deleted on the server.

For scenario 3, we want the local entity kept because it hasn't yet existed on the server.

This issue is for addressing scenario 1 only: if the last update to an entity was from the server, not local-only, and an entity list update does not include that entity, then the entity should be removed from the local working copy.

Acceptance

• [x] Given I've created an entity locally And submitted the form that created it And the entity has been approved And the entity list has been updated When the entity is deleted on the server And the entity list is then updated Then the entity does not appear in forms

• [x] Given I've created an entity locally And submitted the form that created it When the entity is deleted on the server And the entity list is then updated Then the entity still appears in follow up forms And the entity still appears in "View local entities" And I can manually delete the entity locally somehow

• [x] Given I've created an entity locally And submitted the form that created it And the entity has not be approved yet When the entity list is updated Then the entity still appears in follow up forms And the entity still appears in "View local entities"

• [x] Given I've updated an entity locally When the entity is deleted on the server And the entity list is then updated Then the entity does not appear in follow up forms

Questions

1. Is rejected the same as deleted?
   • Yes! Archive and unassign will also be treated the same when those are implemented
2. Are updates to deleted entities rejected by the server or will it "recreate" the entity?
   • It does not recreate the entity. It logs a failed attempt to update a non-existing entity.
3. Is there any reason that we need to know that a secondary instance is an entity list before we create/update locally? I'm not sure we actually need the server to indicate this to solve anything here.
   • https://github.com/getodk/collect/issues/6029

Notes

Brainstorming notes for related/upcoming cases

We'll need new API interactions with the server so that we can handle deletion automatically without needing a manual intervention for clean up ("And I can manually delete the entity locally somehow"), but this will be discussed elsewhere. Implementing this as-is will help us understand the needs there and also help us understand better what kind of storage we want to use for entities on device.