Hello to all lovers of IoT and hardware!
In this article, I will talk about perhaps the most popular data transfer protocol used in the Internet of Things – MQTT. And more specifically, about MQTT Version 5.0 (version published on March 7, 2019). And more specifically, about the pleasant innovations of version 5.0 compared to version 3.1.1.
By the way, why v5.0? Where did v4.0 go?
The root cause lies in the header of the CONNECT packet. The protocol version, or more specifically the protocol version level, is set in byte 7.
Protocol level v3.1 is defined as 3 and v3.1.1 is defined as 4. Therefore, the next protocol level is 5. To avoid inconsistency, the specification group agreed to name the next version MQTTv5.0:
“The value of the Protocol Version field for version 5.0 of the protocol is 5 (0x05).”
According to Lee Stacey, product evangelist at Thingstream, soon MQTT will be the main communication protocol between sensors.
“Currently, IoT devices use different messaging protocols, for example: XMPP, AMQP, DDS. This fragmentation has been an obstacle to progress in the IoT from the very beginning.
Messaging protocols are one area where standardization is needed.
MQTT will probably take the lead. It’s been around for over 25 years, it’s simple, it’s low power, and it’s other benefits make it the ideal IoT standard. Additional features added to MQTT to relay data from sensor networks allow IoT devices to use even less power to send messages.”
And since this is the case, I was given the task of implementing support for the fifth version of MQTT in one of the services of the IoT platform architecture. I had to study quite a few pages of the protocol, so I hasten to share my best practices and understanding of what is happening.
Note. – Further in the text I will call topic – topic, and payload – payload.
Most importantly, MQTTv5.0 is not backward compatible (like v3.1.1). Obviously, too many new things have been introduced, so existing implementations need to be reviewed.
According to the spec, MQTT v5.0 adds a significant number of new features to MQTT while keeping most of the core in place.
Main functional goals of MQTTv5.0:
• improvement for large scale systems in relation to establishing communication with thousands and millions of devices;
• improved error reporting (reason code and reason string);
• formalization of common patterns of interaction, including the definition of the capabilities of interacting devices and responses to requests;
• adding extensibility mechanisms including custom properties, payload format and content type;
• increased performance and improved support for smaller clients.
The main new functions can be divided, for clarity, into the following groups:
1.Features of connection
• Enhanced authentication
• Session expiration
• Client and Server restrictions/limitations
• Will Message Delay Interval
• Server Reference or Server Redirect
2.Features of publishing messages
• Message Expiry Interval
• Payload format indicator & Content type
• Topic Aliases
• Response to the request (Response Topic)
3.Features of the subscription
• Lack of local publications (Non local publishing)
• Retained message control
• Subscription identifier
• Shared subscriptions
4.General Features
• Reason codes and strings for all ACK messages
• Server disconnect
• User properties
• Connection Features
• Extended Authentication (see clause 4.12 in the specification)
• In MQTTv5.0, additional authentication methods are now available, in addition to the username and password as in MQTTv3.1.1 (see the Authentication Method and Authentication Data properties).
Session expiration
In MQTTv3.1.1 and earlier, the client can control how the server handles the client’s session (session means the client’s subscription and any queued messages) using the “clean session” flag. If set to 1, the server will delete any existing session for this client and will not retain the session after disconnection. If set to 0, the server will restore any existing session for the client when reconnecting, and retain the session when the client disconnects.
In MQTTv5.0, this mechanism is mostly preserved, except that now there is a session expiration timer (see the Session Expiry Interval property). If the flag is set to false, you must specify a session end value. If this is not done, it will be set to 0, which makes the connection the same as if the pure session had been set to true.
For example, you can set a session expiration time of 300 seconds. This means that if a client disconnects and reconnects within 5 minutes with a net session value of false, QOS > 1, then session state data (eg, topic subscriptions, queued messages) is preserved. However, if the client disconnects and reconnects after 5 minutes with the same settings, session state data is deleted and the client must resubscribe to topics, and messages sent while the client was disconnected are lost.
Client restrictions
The client can now specify the following restrictions to the server:
by the size of the messages to be sent (see the Maximum Packet Size property),
by the number of QOS 1 and 2 messages (see the Receive Maximum property).
Message size limit
The client tells the server the maximum message size it is willing to receive, and any messages that exceed this value are discarded by the server. Used by low-power clients that do not have enough memory to store messages.
Post limit
The client tells the server to limit the number of QOS 1 and 2 messages that can be sent at the same time. Also used by low power clients because QOS messages 1 and 2 mean that the client must send additional acknowledgment messages.
Server limits
Providing certain features may put a strain on server resources, so the server may not support them. The server can now specify this using the properties field.
For example, in the CONNACK message, the server MAY specify the maximum value of the topic alias (see the Topic Alias Maximum property) and the session expiration time (see the Session Expiry Interval property).
Will Message Delay Interval
Used to specify a time delay (see the Will Delay Interval property) when sending a Will Message, preventing the message from being sent if the connection is interrupted for a short time.
Server redirection (see clause 4.11 in the specification)
This is a very interesting feature because it allows the server to redirect the client to another broker/server (see the Server Reference property).
Features of publishing messages
Message expiration interval
When a message is published with QOS 1 or 2 and the receiving client has:
• clean session false,
• subscription with QOS 1 or 2,
• the broker will store published messages for the client if it is currently disabled.
However, you can set the period for which the server will keep this message. For example, set the value of the Message Expiry Interval property to 20. With this setting, if the receiving client does not connect within 20 seconds, the message will be deleted.