Adding support for version 5.0 of the MQTT protocol.

[luisgar1990]

Currently the MQTT module of Scapy (contrib/mqtt.py) only supports version 3.1.1 of the MQTT protocol. Thus, a new module was developed (contrib/mqtt5.py) that supports version 5.0. Version 5.0 contains many changes compared to Version 3.1.1, and thus we had to develop a library specifically for MQTT 5.

It is worth noting that we worked on top of the MQTT 3.1.1 module (contrib/mqtt.py), thus many fields, classes or dictionaries are still roughly the same in this new version. Main differences compared to the MQTT 3.1.1 (contrib/mqtt.py) are :

1. Added Class MQTTProperty : New to version 5.0, most MQTT packets allow users to define properties. Depending on the property, the fields may be either ShortField, IntField, or StrLenField. This is the reason why we defined a MultipleTypeField in the class.

2. Added a dictionary called Property that contains the values and names of the MQTT properties.

3. Class UTF8EncodedString and UTF8String Pair were defined since some properties may have more than 1 field.

4. Each MQTT Packet has fields for users to define these MQTT Properties. We've highlighted where these fields are located (example: "#CONNECT Properties"). This is one of the reasons why we had to develop a module specifically for MQTT 5, since we had to modify fields_desc for each packet.

5. Added reason/return/retain codes for the following packets : Disconnect, Connack, Puback, Pubrec, Pubrel, Pubcomp, Subscribe, Suback, and Unsuback (feature specific to MQTT 5.0).

6. Added Bitfields (reserved, retainhandle, RAP, NL, QoS) for topics in MQTT subscribe packets (feature specific to MQTT 5.0). Version 3.1.1 of the MQTT protocol only had 1 field (QoS).

7. Added class MQTTAuth, to support the new MQTT Auth Packet in MQTT 5.0.

8. Added authenticate reason codes for MQTT Auth packets.

9. Added unit tests. The unit tests evaluate when packets have or do not have properties.

[codecov[bot]]

Codecov Report

Merging #3292 (efbdd1f) into master (66b611c) will increase coverage by 0.01%. Report is 587 commits behind head on master. The diff coverage is 97.74%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3292      +/-   ##
==========================================
+ Coverage   85.84%   85.85%   +0.01%     
==========================================
  Files         271      272       +1     
  Lines       55227    55360     +133     
==========================================
+ Hits        47407    47532     +125     
- Misses       7820     7828       +8     

Files	Coverage Δ
scapy/contrib/mqtt5.py	97.74% <97.74%> (ø)

... and 2 files with indirect coverage changes

[luisgar1990]

@gpotter2 Hello. I've tried importing some dictionaries/classes that are the same in the older MQTT, however I'm having an issue where the packets are not dissected correctly. For example, there is a dictionary called CONTROL_PACKET_TYPE in the older MQTT. I've tried importing it by writing "from scapy.contrib.mqtt import CONTROL_PACKET_TYPE" in mqtt5.py . However, when I import this dictionary, it seems that the classes in the new file (mqtt5.py) are overwritten with the classes from the older MQTT and thus Scapy dissects the packets as if they were from the older version (contrib/mqtt). Interestingly, if I import a class from contrib/mqtt5.py to the older MQTT file, it seems to overwrite the older classes with the newer versions as well.

Moreover if I comment out the "bind_layer" functions from the older MQTT, then importing classes/dictionaries from the older MQTT seems to work correctly. The packets are dissected correctly.

Do you have any suggestions on how can I fix this issue, or import successfully without overwriting the modified classes? I find it strange not being able to import classes or dictionaries from files in the contrib directory correctly.

Thank you very much for your help and time.