azure-functions-extension
azure-functions-extension copied to clipboard
dapr-sandbox
DaprSaveStateAsyncCollector now uses ConcurrentQueue to ensure the order
DaprSaveStateAsyncCollector now uses ConcurrentQueue to ensure the order of changes (ConcurrentBag return reversed order using .Take()). DaprSaveStateAsyncCollector uses Dictionary on FlushAsync to avoid key repetition in json serialization
Hey - what problem is this PR trying to solve?
Lot of time passed, but if I remember right, the issues were 2:
- a wrong behavior when you try call AddAsync multiple time with same key because Concurrent Bag doesn't preserve order. The FlushAsync could (an would) store the wrong value instead of the last one.
- Flush async will save the same state key multiple time because it cycles on all items of Concurrent Bag (or Concurrent Queue). DaprStateRecord could "refer" to the same state key
Cool cheers!
Should we implement E-Tags if they're available? As it feels like we're wrapping around Dapr's concurrency handling.
https://docs.dapr.io/developing-applications/building-blocks/state-management/state-management-overview/#why-dapr-chooses-optimistic-concurrency-control-occ
If your application omits ETags in writing requests, Dapr skips ETag checks while handling the requests. This enables the last-write-wins pattern, compared to the first-write-wins pattern with ETags.
My thoughts are that, if the implementer has chosen a state store that doesn't support e-tags we can lean more on the Dapr runtime, then have our own methodology thus kinda forcing the choice over whether in-order writes are important to them, not the az function package.
In my opinion, the issue here is when the same service writes the same key multiple times, and Dapr's concurrency is related to multiple services writing the same key.
Moreover, the documentation you linked states that it is up to the application to decide whether to use tags or not. Therefore, I don't believe we are circumventing Dapr's concurrency handling.
I believe the issue lies in how the Azure Function extension buffers the write requests and how it handles the actual call within the flush process:
- AddAsync buffers the requests without preserving order, making it impossible for the same function to overwrite a value within the same execution.
- FlushAsync sends all buffered requests simultaneously, potentially resulting in multiple writes to the same key.
All of this occurs before Dapr becomes involved. While E-tags might mask the mishandling, in my opinion, they should primarily be used for managing concurrency between services rather than within the same execution of one service. In this context, I also believe the use of E-tags should be left to the discretion of the application developer.