BrighterCommand
Add better OTel information into Brighter
We created semantic conventions for Brighter: https://github.com/BrighterCommand/Brighter/blob/master/docs/adr/0010-brighter-semantic-conventions.md
This PR is about adding those conventions into Brighter.
We will not be able to get links on a publish or bulk clear until the ability to add links to an existing span is added to Activity. This is the relevant issue: https://github.com/dotnet/runtime/issues/97680 and matching PR: https://github.com/dotnet/runtime/pull/101381 so we are just waiting release of this.
@iancooper Talking about bulk I see a couple of Paths here, Generally I use implicit clearing of the outbox so in my application code I am only ever doing a Bulk outbox add (where necessary). The most important flow in my mind is the Bulk dispatch of messages, at the moment if we consider a batch of 100 messages that are to be dispatched:
Without Bulk Dispatch
- Call CommandProcessor.ClearOutbox
- Get Messages to Dispatch
- For Each Message Id
- Open SQL Connection
- Run Query
- Close SQL Connection
- For Each Message Id
- For each Message
- Publish Message
- Mark As Dispatched
- Open SQL Connection
- Run SQL Command
- Close Connection
In this instance we have
- 200 SQL Connections Opened and Closed
- 100 call to the Provider
This is very slow, in my experience dispatching from the outbox is slower than consuming (from memory it was around a message or two a second) so no active messages end up on the queue
The same example above with Bulk (as is currently functioning in v9 for MsSql and ASB)
- Call CommandProcessor.ClearOutbox
- Get Messages to Dispatch
- Open SQL Connection
- Run Query
- Close SQL Connection
- Publish Messages
- Mark As Dispatched
- Open SQL Connection
- Run SQL Command
- Close Connection
- Mark As Dispatched
In this instance we have
- 2 SQL Connections Opened and Closed
- 1 call to the Provider
Using this method I have successfully been able to dispatch a very large weight of messages (>100k) in well under a minute.
Without the ability to bulk dispatch my system will not work (we dispatch millions of messages per day)
@iancooper The most important flow in my mind is the Bulk dispatch of messages, at the moment if we consider a batch of 100 messages that are to be dispatched:
Not altered here. I'll pick up batches after. It would be good to support both batch writes to the outbox and batch read and write to producer.
There is a V10 change because we need the publication for the pipeline, so that we can use it for CloudEvents (and its there for generic mappers) so we do publication lookup when we run the transform/mapping pipeline.
That means we would need to do something a little different for deposit post as I moved the late binding earlier.
But the clear, I am still looking at
@iancooper The most important flow in my mind is the Bulk dispatch of messages, at the moment if we consider a batch of 100 messages that are to be dispatched:
Not altered here. I'll pick up batches after. It would be good to support both batch writes to the outbox and batch read and write to producer.
There is a V10 change because we need the publication for the pipeline, so that we can use it for CloudEvents (and its there for generic mappers) so we do publication lookup when we run the transform/mapping pipeline.
That means we would need to do something a little different for deposit post as I moved the late binding earlier.
But the clear, I am still looking at
Excellent, as long as in the Deposit Flow we can do a single Command for insert this will be excellent! let me know if there are any parts you want me to look at
@preardon As discussed going to merge. It is not complete, but it should work.
- Clear is still be bulk. I have renamed the 'bulk' method to ClearOutstandingFromOutbox for clarity as to purpose
- I'll look into the bulk deposit later
- No error handling in activities yet
- Only InMemory Producer and Inbox/Outbox currently instrumented; will need to pick up the concrete ones
Those will be in further PRs