Mosquitto client implements QoS 1 a zero or more times and QoS 2 as at most once

[fhars]

In both cases, an adversary scheduler will run the code right up to the if(mosq->on_message){ and then reboot the system. When the clients reconnects with clean session equal to 0, the server will regard the message as completely acknowledged, while the application will never have seen the message at all.

This is related to #118, and there are several bad choices here. It may actually be impossible to implement QoS 2 correctly without writing the message to persistent storage before sending the PUBREC message.

[ralight]

I'm not familiar with adversary schedulers, but the point stands - if something happens to stop the client before that call then the broker will not retry.

I'd be happy to receive a pull request for (optional) persistence support.