keda icon indicating copy to clipboard operation
keda copied to clipboard

Published 20 hours ago • verified publisher icon kedacore

MQTT - based scaler

Open arschles opened this issue 4 years ago • 20 comments

I'd like to have a scaler listen to an MQTT source and scale up/down based on incoming messages. This would be useful for Kubernetes clusters that run on the edge (for example, in IoT scenarios and in scenarios where k8s is running on underpowered machines)

  • Scaler Source: A scaler to support scaling applications based on incoming MQTT messages. Not intended to support bidirectional communications
  • How do you want to scale: Based on volume of messages on a particular topic
  • Authentication: Two options are username/password auth and advanced (x.509) auth. username/password auth should be sufficient for now, given that you can already place credentials in a secret and pass them into a ScaledObject

arschles avatar Oct 22 '20 18:10 arschles

I would be happy to get started on this.

arschles avatar Oct 22 '20 18:10 arschles

I'd be interested in helping out as well!

andschneider avatar Oct 22 '20 18:10 andschneider

I like the idea. It's up to you guys, who wants to work on this! :) Thanks!

zroubalik avatar Oct 23 '20 09:10 zroubalik

@zroubalik what does the "needs discussion" label mean? It's ok to start implementing right?

iennae avatar Oct 26 '20 19:10 iennae

I started looking into making the scaler and was trying to figure out what to use as the metric value.

Since MQTT brokers don't have the concept of a queue, using queue length or similar wouldn't work. I was then thinking of using number of messages received over a certain period of time. However, I'm not sure if time based scalers are possible. @arschles Is this what you meant with "volume of messages"?

andschneider avatar Oct 26 '20 19:10 andschneider

I didn't have anything specific in mind regarding "volume of messages" - just an abstract idea. I think that number of messages received over a period of time could work. another idea is to maintain a rolling average of messages received over a window. according to you, @andschneider , you'd need to have KEDA be a subscriber to the queue - not sure if that's ok to do. @zroubalik thoughts on that part?

arschles avatar Oct 26 '20 19:10 arschles

@zroubalik what does the "needs discussion" label mean? It's ok to start implementing right?

that means that this issue needs some clarification/specification/proposal on how it will be implemented, basically the stuff that you can see is happening.

zroubalik avatar Oct 27 '20 08:10 zroubalik

@arschles so your idea is, that KEDA will subscribe to the topic and will do the calculation (messages received over a window etc)? I am not an expert on MQTT, I wonder, are there any metrics collected by the MQTT Broker itself?

zroubalik avatar Oct 27 '20 08:10 zroubalik

I'm by no means an MQTT expert either so take this all with a grain of salt, but I did a little more research and it seems that some brokers do expose metrics. As it's not part of the MQTT spec it's up to the broker's implementation for what they are though. For example, mosquitto and HiveMQ seem to be popular choices and they expose different metrics.

To keep the scaler as general as possible my thought was to stick to something in the spec. There's a thing called retained messages which is flag set on a published message that basically tells the broker to always store one (and only one) message for a topic, until it gets deleted. Essentially it could be used like a status message and if present the scaler would spin something up.

Obviously a little different than using some sort of messages in a time span/window approach - but it gets around having the scaler subscribed to a topic. @arschles @zroubalik what do you guys think?

andschneider avatar Oct 30 '20 05:10 andschneider

@andschneider would you be up for getting this started? A lot of my time is taken getting the http-add-on started

arschles avatar Dec 04 '20 19:12 arschles

@arschles Yep, if everyone is cool with the direction I'll have some time towards the end of this week to get it going.

andschneider avatar Dec 07 '20 04:12 andschneider

@arschles Apologies for the delay - I finally started working on it this weekend.

Here's a repo with an example setup for running a MQTT broker and pub/sub components. The basic scaler seems to be working (its on the mqtt-scaler branch of my Keda fork). Main things left to do are auth, higher QoS messages, and general robustness.

andschneider avatar Dec 21 '20 05:12 andschneider

Great to hear, thanks!

tomkerkhove avatar Dec 23 '20 10:12 tomkerkhove

Quick update - with holidays and whatnot I haven't been able to spend much time on this. Next week is finally looking all clear though and I should be able to make some progress!

andschneider avatar Jan 05 '21 18:01 andschneider

That's great, thanks!

tomkerkhove avatar Jan 05 '21 18:01 tomkerkhove

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Nov 27 '21 08:11 stale[bot]

@zroubalik it doesn't seem like there's much support behind the MQTT-based scaler. should we close this or let the stale bot do it in due time?

arschles avatar Dec 02 '21 23:12 arschles

@arschles let's keep this open as it is a valid ask, maybe someone else will pick it up in the future 🤞

zroubalik avatar Dec 03 '21 08:12 zroubalik

Sounds good!

arschles avatar Dec 03 '21 17:12 arschles

MQTT scaler would be definitely good for AWS iot core processing https://docs.aws.amazon.com/iot/latest/developerguide/mqtt.html

vainkop avatar Feb 07 '24 07:02 vainkop