googleapis
pubsub: With exactly once delivery and message ordering enabled, message order is not mantained
Client
PubSub
Expected behavior
Messages received are in the order the server delivers them
Actual behavior
Messages are reordered randomly when iterating over pendingMessages map
Additional context
On iterator.go, messageIterator.receive method, a pendingMessages map is created from the message list to remove any messages where modAck fails. After processing, a new message list is created iterating over the resulting pendingMessages object:
for _, m := range pendingMessages {
v = append(v, m)
}
return v, nil
The order in which items are appended is not necessarily the same of the original list of messages (https://go.dev/blog/maps). Message ordering feature is thus broken
I'm investigating this now. Probably requires keeping track of the messages in the order they came in a slice, and using that order to return them while keeping the pendingMessages map. I think the Python library might have a similar bug so following up there as well.
I was mistaken, the python client's behavior is different. When the receipt / lease modack happens (the first modack the client issues when a batch of messages are received), with exactly once delivery, the invalid ack_ids are kept track of in a separate array. When returning messages, the client iterates through the received messages and removes expired ack IDs.
I'll change the Go client to match this behavior.