Payload helpers

[bertmelis]

A lot of users seem to have problems to handle MQTT payloads. I was thinking to create a separate file with some helper classes. I'm reluctant to implement this behaviour by default because my feeling says it is not needed in 95% of the cases.

pseudo-c++-code

#include <AsyncMqttHelpers.h>
PayloadHandler payloadHandler;

void onMessage(properties, data, totalLength) {
// will be called when the message fits in emory
}

void onChunkedMessage(properties, data, index, size, totalSize) {
// will be called when the message doesn't fit in memory
}

payloadHandler.onMessage(onMessage);
payloadHandler.onChunkedMessage(onChunkedMessage);
mqttClient.onMessage(/*bind payloadHandler callback*/);

Converting the binary payload to a std::string or c-string is trivial...

what's your opinion?

[mcspr]

3rd option would be to allow a small scratch buffer, storing the message and simply skipping the onMessage callback until it is complete? In case where the total length is less than the provided buffer size (e.g. mqttClient.setMessageBuffer(1024) or give it an unique_ptr). (ref. the issue I mentioned before, again, where 1024byte array is used, but large payloads are simply skipped)

I guess, this also applies to the helper class, but I don't really get how it will handle the situation where both callbacks are registered.

[bertmelis]

I still have to find a use case for multiple callbacks. But that's a breaking change in the API.

Anyway, I'll work it out a bit more so you'll have a more clear idea of what I'm thinking about.

[Pablo2048]

I'm with @bertmelis for PayloadHandler class. String, std::string result can be handled same way as AruduinoJSON does - by using templates and syntax like

PayloadHandler payloadHandler;

...

String result = payloadHandler.as<String>();

[proddy]

open question: how about natively supporting ArduinoJson as a payload type? Most of my publish payloads are JSON strings. They get serialized to a char * buffer and sent to MQTT's publish(). It'll be great if asyncmqttclient would do the serialzing for you. It would add an additional dependency mind you. It's a bit off topic here, sorry, as this is about outbound transmissions, not incoming.

[bertmelis]

Didn't think about outgoing. It's not that hard to add boilerplate code for various types of payloads.

I'd still do the implementation in the "Helpers" and not in the client itself. Something like this?

uint16_t AsyncMqttclient::publish(char* topic, AsyncMqttPayload payload /* and rest of args */);
char* AsyncMqttPayload::toChar();
size_t AsyncMqttPayload::size();
// and various constructors of AsyncMqttPayload

If it's not being used, the compiler will leave it out.

[kleini]

open question: how about natively supporting ArduinoJson as a payload type? Most of my publish payloads are JSON strings. They get serialized to a char * buffer and sent to MQTT's publish(). It'll be great if asyncmqttclient would do the serialzing for you. It would add an additional dependency mind you. It's a bit off topic here, sorry, as this is about outbound transmissions, not incoming.

That is your case, that you have JSON in MQTT payload. This may be very different for other users of this library. So having a binary dependency to something, that is not required for me can result in serious flash size issues.

[bertmelis]

I'm having second thoughts. By using an intermediate buffer, you'd be copying three times: the original ArduinoJSON, the temporary payload buffer and the mqtt packet. That's not good.

It's still possible though.

[proddy]

yeah, you're right. probably not a good idea. In the code do you check for available heap before allowing it be stored in the temp buffer? I've seen someone implement it before in asyncmqttclient and pangolin copied it. It's a nice idea too.

[bertmelis]

There is indeed a check: https://github.com/marvinroger/async-mqtt-client/blob/2991968a97193aaa6402d146490b93ea671c7e02/src/AsyncMqttClient.hpp#L8 https://github.com/marvinroger/async-mqtt-client/blob/2991968a97193aaa6402d146490b93ea671c7e02/src/AsyncMqttClient.cpp#L739

It is only applied on publish messages. Also no check on incoming messages as normally the payload is passed by pointer. So it could fail with the helper-buffer as in the PR.

[proddy]

ok, should have checked the code myself. nice work!