Adding tracing facility

[matthiashanel]

Adds msg_trace (trace rule name to trace) to operator and accounts. enables tracing for operator and accounts. Adds ping_probes option (array of probe names) to accounts, enabling header based tracing. Adds Nats-Ping header and ping_probes option (array of probe names) to accounts, enabling tracing by Nats-Ping header.

Nats-Ping header can be empty (default) or contain a list of probe names When set, the message will not be handed to subscriber/jetstream/other accounts. Optionally the message body can contain a list of server ids to filter responding servers on.

Still missing: handling responses, jwt integration & dynamic tracing (settable via system account API). Settle on what to use as trigger.

This basically works by having a single trace context per receive loop. With this context we call tracing functions for pub/sub/store. After handling of an incoming message is done we call complete to send traces.

[derekcollison]

Large ones like this might require a f2f walkthrough..

[matthiashanel]

Large ones like this might require a f2f walkthrough..

@derekcollison Absolutely, just pushed changes as I'm looking at something with Alberto

[derekcollison]

👍🏼

On Wed, Oct 27, 2021 at 2:50 PM Matthias Hanel @.***> wrote:

Large ones like this might require a f2f walkthrough..

@derekcollison https://github.com/derekcollison Absolutely, just pushed changes as I'm looking at something with Alberto

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/nats-io/nats-server/pull/2624#issuecomment-953337571, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAV74PI3RQ4SPLTHPGN7BDUJBX3HANCNFSM5GAV2H3Q .

[bwerthmann]

Fixes nats-io/nats-general#72

[bruth]

@derekcollison @aricart Not sure if this was discussed offline as desirable, but this PR has quite a number of conflicts, so closing now. We can revisit if the general concept should be revisited.