Add transport metadata about logical/physical queues to pipeline

[ramonsmits]

I needed to create a behavior to snoop info about the message being fetched/processed. I wanted to use this to send metadata about the message to be used for telemetry.

I wanted to know transport metadata:

• from which logical queue the message was fetched
  • myendpointqueue
  • myendpointqueue-discriminator
• from which physical queue the message was fetched
  • myendpointqueue@zolder
  • myendpointqueue-discriminator
  • myendpointqueue-discriminator
• when the message was fetched from the queue (if possible, maybe not supported by all transports), which is different from when processing started (prefetching to processing latency)
• how long it took to fetch a message (if possible, maybe not supported by all transports)
• metadata about the connection/broker
  • which Azure subscription?
  • which SQL Server?

It would be nice if most of these static values would be set to one or more context properties:

• ITransportReceiveContext.LogicalQueue:Uri
• ITransportReceiveContext.PhysicalQueue:Uri
• ITransportReceiveContext.Message.FetchedAt:DateTime
• ITransportReceiveContext.Message.FetchCompletedAt:TimeSpan

The last can probably be global per endpoint instance and not need to be part of the receive context.

[dbelcham]

The @Particular/nservicebus-maintainers talked about this briefly and we had a couple of questions to help us narrow down how we might implement this and if that will require a breaking change or not.

• when, in the life of the pipeline, would you want to be able to access this metadata?
• what is the desired use of this data? (ie. perf counters)

[ramonsmits]

@dbelcham The data would indeed be used for metric collection and will be aggregated after the event took place. Properties can be added to a metric or event value to provide context.

As some data is static I would imagine that for a single receiver the global static value would be read once and then cached but that is an optimization.

[timbussmann]

so are you currently using a workaround to access those information or is some feature potentially blocked by this?

[ramonsmits]

@timbussmann I did not try to do more plumbing besides creating this issue. If it would have been present the NServiceBus.Metrics could be made much more interesting. If the meta data would have been available the monitoring samples currently made in docs for Azure Application Insights, Prometheus en maybe in the future for AWS Cloudfront could show much more meaningful data.

[timbussmann]

slightly adjusted the title to not limit any added metadata to a specific stage.

[mikeminutillo]

Had a support case with a customer where they wanted to retrieve the queue name the message came from within the handler.

The workaround I provided was to take a dependency on ReadOnlySettings and then call the LocalAddress() extension method on it. This works because the endpoint doing the receive is the one that creates the handler so it's container should be used and that has the right ReadOnlySettings instance in it.

Unfortunately this isn't a complete solution as any message that comes through a satellite or a callback queue will be wrong.