dotnet
AWS SQS Message On-Demand Deserialization Support
Current Version: 3.2
To support production operational responsibilities, I need a way to deserialize SQS stream messages. These are serialized and deserialized in Orleans via the SQSBatchContainer and SerializationManager.
The use case is to view messages that failed to be processed. If this happens in our system, there is likely some remediation that needs to happen outside of just shovelling the messages back into the main queue. We could try to rely on our logs, but it's very possible that we won't get every piece of data that we need there. Additionally, if theres a poison pill message that continually fails to be processed, we need to view that message to determine why.
It seems impossible to replicate the deserialization code that Orleans would invoke on these messages. This appears to be where it all happens in Orleans 3.x: https://github.com/dotnet/orleans/blob/7dd8d299fedf2c21bda17b1c037c7bb3d817cafb/src/AWS/Orleans.Streaming.SQS/Streams/SQSBatchContainer.cs#L99
The issues that I've encountered are:
- SerializationManager is not accessible via DI in my application (which uses Microsoft.Extensions.DependencyInjection), and
- SQSBatchContainer is internal, so I have no reference type to deserialize the object into.
My suggestion is to:
- Make SerializationManager accessible via DI, and
- Make SQSBatchContainer public
SerializationManager should be accessible via DI. Are you trying to get inside a silo or an Orleans client?
I think this functionality needs to be implemented in all providers. We already have IStreamFailureHandler, I wonder if this would be the good place to add such functionality.
Currently we don't pass the event content to the failure handler. We need to be careful with that.
