paho.mqtt.java Incorrect UTF-8 surrogates check in MQTTv3 client

When topic name contains a pair of high surrogate + low surrogate UTF-8 check is performed incorrectly and following error. Similar problem was fixed for MQTT5 client here - https://github.com/eclipse/paho.mqtt.java/issues/533.

Please fill out the form below before submitting, thank you!

[x] Bug exists Release Version 1.2.5 ( Master Branch)
[x] Bug exists in MQTTv3 Client on Snapshot Version 1.2.6-SNAPSHOT (Develop Branch)
[ ] Bug exists in MQTTv5 Client on Snapshot Version 1.2.6-SNAPSHOT (Develop Branch)

Exception in thread "MQTT Rec: service-f7bee955-6130-4add-b0f7-65312619d43b" java.lang.IllegalArgumentException: Invalid UTF-8 char: [db40]
at org.eclipse.paho.client.mqttv3.internal.wire.MqttWireMessage.validateUTF8String(MqttWireMessage.java:407)
at org.eclipse.paho.client.mqttv3.internal.wire.MqttWireMessage.decodeUTF8(MqttWireMessage.java:365)
at org.eclipse.paho.client.mqttv3.internal.wire.MqttPublish.<init>(MqttPublish.java:64)
at org.eclipse.paho.client.mqttv3.internal.wire.MqttWireMessage.createWireMessage(MqttWireMessage.java:207)
at org.eclipse.paho.client.mqttv3.internal.wire.MqttWireMessage.createWireMessage(MqttWireMessage.java:182)
at org.eclipse.paho.client.mqttv3.internal.wire.MqttInputStream.readMqttWireMessage(MqttInputStream.java:119)
at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(CommsReceiver.java:137)
at java.base/java.lang.Thread.run(Thread.java:834)

If this is a bug regarding the Android Service, please raise the bug here instead: https://github.com/eclipse/paho.mqtt.android/issues/new

Aug 26 '20 15:08 silh

Just a comment on this patch and the original: the function name is misleading. Although UTF-8 is used for MQTT, Java's internal encoding of String's characters is UTF-16. charAt() is used to address the high and low surrogates. As a result, the code is checking for valid combinations of UTF-16 surrogate pairs.

Sep 06 '20 08:09 sp193

It's only part of the check. Besides, there is a similar function in guava which is located in UTF-8 class- https://github.com/google/guava/blob/bafab6242d7fb05560b6d3ebe0f7cfc7f30803f2/android/guava/src/com/google/common/base/Utf8.java#L91 Also there is a similar check in HiveMQ MQTT client MqttUtf8StringImpl - https://github.com/hivemq/hivemq-mqtt-client/blob/95fe44eaa3154353d0dffb6d4282bd1b1d233576/src/main/java/com/hivemq/client/internal/mqtt/datatypes/MqttUtf8StringImpl.java#L164

Sep 07 '20 08:09 silh

Having same issue in version 1.2.5,Here is the link

https://github.com/eclipse/paho.mqtt.java/issues/836

Sep 30 '20 07:09 kavidriod

Is someone going to check the suggested Pull Request?

Dec 10 '20 08:12 silh

Why was this check added in the first place? This wasn't an issue in 1.2.0, so it definetly also won't be one now. This "patch" broke nearly all systems that use encryption and, thus, may need to send bytes that aren't "normal" characters. Is this project still maintained or is there no proper fix to be expected?