A failure in snapshot save can cause unobserved task exception

[stedmanm]

trafficstars

Version Information Version of Akka.NET? 1.5.28 Which Akka.NET Modules? Akka.Persistence

Describe the bug When snapshot save fails, Akka will then attempt to delete the specified snapshot, if the delete snapshot operation throws an exception, this exception will become unobserved.

To Reproduce This can most likely be reproduced in multiple ways, but this is what happened in our scenario.

1. Multiple load snapshot operations failed, causing an open circuit.
2. Save snapshot operation fails, because circuit is open.
3. Delete snapshot gets triggered, because snapshot save failed.
4. Delete snapshot operation returns a faulted task with an open circuit exception, this exception is never observed.

Expected behavior The exception should get observed by Akka framework.

Actual behavior Exception was not observed and caused our service to crash.

Screenshots

Environment Windows Service and .NET 7

[Aaronontheweb]

Weird, though I had commented on this and classified it yesterday. We'll look into it!

[Aaronontheweb]

ah wait, yes I did - is this a duplicate of https://github.com/akkadotnet/akka.net/issues/7358 ? or a different flavor of the same problem

[stedmanm]

@Aaronontheweb Yes this is different problem than https://github.com/akkadotnet/akka.net/issues/7358, that also caused us an unobserved task exception, but it is also related to the SnapshotStore.