pixie icon indicating copy to clipboard operation
pixie copied to clipboard

AMQP support

Open tsipo opened this issue 3 years ago • 7 comments

Is your feature request related to a problem? Please describe. Reading Pixie's supported protocols I see there two message brokers / event streaming platforms: NATS and Apache Kafka. I miss the protocols supported by RabbitMQ, a popular battle-proof message broker in use by many organizations for more than a decade now.

Describe the solution you'd like RabbitMQ supports the following messaging protocols: AMQP 0.9.1, AMQP 1.0, MQTT 3.1.1, STOMP 1.0-1.2, Stream. AMQP 0.9.1 is AFAIK the most popular protocol, Stream was only added recently (3.9). AMQP 0.9.1 and AMQP 1.0 are different protocols BTW (not different versions of the same protocol). From RabbitMQ documentation:

All protocols supported by RabbitMQ are TCP-based and assume long-lived connections

I'd love to see support in Pixie for at least AMQP 0.9.1. Thank you!

Describe alternatives you've considered N/A

Additional context N/A

tsipo avatar Oct 15 '21 20:10 tsipo

mqtt would also be a very useful protocol to support. it is very common at the edge and k8s is finding it's home in many far edge devices these days.

mak3r avatar Oct 19 '21 15:10 mak3r

Just getting some thoughts/info down on this which others might find useful.

Outside of AMPQ 0.9.1 support, looking at AMPQ1.0 in the future could be handy as well. It's implementation is less prevailant, but it would give coverage for Azure Service Bus, Azure's Event Hubs as well as the standard qpid/Activemq/etc support.

On MQTT, would be super useful but figuring out which version(s) of the protocol to support might be key. Generally speaking I'd probably look at 3.1->5.0 support. Most of the implementations (mosquitto,mqtt-c,etc) only support from 3.1 and above. Although 3.1.1 -> 5.0 would likely cover everything ( I can't remember if there were major protocol changes between 3.1 and 3.1.1, but 3.1.1 has been the defacto since about 2014.

Links to the two standards for 3.1.1 and 5.0 http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.pdf https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.pdf

obowersa avatar Oct 22 '21 00:10 obowersa

@mak3r Can you please open a separate ticket for MQTT? Just to make the feature requests easier to track. :)

oazizi000 avatar Oct 22 '21 21:10 oazizi000

Also, any volunteers for implementing the protocol parsers? We'd love to get any contributions (and we'd help)!

oazizi000 avatar Oct 22 '21 21:10 oazizi000

@oazizi000 I'll shift this to the MQTT one, but I was talking about internally earlier today. I wouldn't be able to look into it until the new year, but if it hasn't been picked up by then I'd be interested in taking a crack.

obowersa avatar Oct 22 '21 23:10 obowersa

@mak3r Can you please open a separate ticket for MQTT? Just to make the feature requests easier to track. :)

An MQTT specific feature request is here https://github.com/pixie-io/pixie/issues/341

mak3r avatar Oct 25 '21 18:10 mak3r

Hi @tsipo, I'm working on adding AMQP support. I have an initial UI with the protocol parsed. What kind of fields/metrics/ graphs would you be interested in?

Attached an Image of the AMQP 0.9 showing up the pixie dashboard: inprogress_amqp_ui_showing_on_dashboard

vikranth22446 avatar Jun 30 '22 18:06 vikranth22446