magistrala
magistrala copied to clipboard
absmach
Add message tracing over NATS using open tracing standard
FEATURE REQUEST
-
Is there an open issue addressing this request? If it does, please add a "+1" reaction to the existing issue, otherwise proceed to step 2. No.
-
Describe the feature you are requesting, as well as the possible use case(s) for it. After adding open tracing support to the existing gRPC and HTTP endpoints in
UsersandThingsservice, we should add support for tracing async messages that are sent over Mainflux. -
Indicate the importance of this feature to you (must-have, should-have, nice-to-have). This is a must-have.
After doing the previous tracing issue, I figured I would grab this one as well.
I'm going to take a little time to read up on NATS before starting but I see that there is a NATS publisher in a number of services. I could see the implementation of this being similar to the tracing middleware in users and things.
@anovakovic01 I took another look at this based on the not.go repo. I believe the tracing middleware I create was on the right track but I needed to transfer the span over a NATS message rather than a context.
The publish messages look like I can write the mainflux.RawMessage payload to a not.TraceMsg and save the result as the new payload. The span's context is then injected into this not.TraceMsg.
The subscriber side will be trickier due to messages coming in over a channel. I believe the tracing middleware should receive the message from this channel and extract the span context from this message. The remainder of the not.TraceMsg will be the payload that can be passed to the NATS broker on a separate channel.
Does this sound more like what you had in mind?
@nwneisen You're right, you can write context data and raw msg payload into the not.TraceMsg. Think about maybe embedding this info in the RawMessage, and avoid using TraceMsg, because, if we at some point change broker, we'll have to replace this not package for something else too. mainflux.RawMessage already contains metadata related to the message.
@anovakovic01 I started taking a look at this again. This draft PR adds tracing for publishing the way we described above. I'm still trying to wrap my head around how this can be done for subscribing.
@nwneisen thank you for the PR. It looks to me like it's the step in the right direction.
@nwneisen We recently created a broker package that wee use in all services to publish and subscribe. I guess that now it should be simpler to resolve this one. Would you like to take this one again?
