MQTT Client: MQTT Client Library - App Notes

Overview

This narrative outlines the usage / working apects and operational modes that are available to the users to make use of the services of the MQTT Client Library to build applications that are just apt for the targeted use-case.

QoS Messages

In MQTT, a QoS0 message does not warrant the remote entity to acknowledge the receipt of the message and the protocol does not guarantee to the sender the delivery of the message to the recepient. In other words, the message from the sender may or may not get delivered to the receipient and therefore, the QoS0 messages get associated with the "at most once" delivery concept. However, with QoS1 (at least once, delivery) and QoS2 (exactly once), the messages need to be acknwoledged (in a differing manner). This section enumerates the client LIB specific implementation aspects of QoS1 and QoS2 messages.

The non QoS0 messages in MQTT protocol need some additional processing. Specifically,

• Ack QoS1 and QoS2 packets (received from the server)
  The client LIB 'context' relieves the application from sending the ACK messages to the server by implementing the required support to send the corresponding ACK as and when required.

• Track QoS1 and QoS2 message sequences
  The client LIB 'context' manages the non-QoS0 packets to track and ascertain the sequence of the message ID(s) from the server. In addition, the 'context' resends the unacknowledged QoS1 / QoS2 packets to the MQTT server at the time of next MQTT connection. The packets are resent to the MQTT server, only if, the on-going session (connection) has not been configured for the 'clean session', the ACK(s) for these packets have not been received and the next iternation of the MQTT session (connection) also does not assert the 'clean session' parameter.
  
  In case, the on-going session for a 'context' has been configured with for a 'clean session', the references to un-acknowledged packets are removed from the 'context' at the time of termination of the network connection to the server. In other case, where the on-going session in the 'context' has not been configured for a 'clean session', however, in the next iteration of the MQTT connection, the 'clean session' parameter gets asserted, the client LIB 'context' will drop all the references to the un-acknowledged packets at the time of the establishment of the second MQTT connection.
  
  When required to resend the unacknowledged packets to the server, the client LIB 'context', when configured to operate in the MQTT 3.1.1 mode, resends only the PUBLISH packets to the server. Whereas, in the MQTT 3.1 mode, the client LIB 'context', additionally, resends to the server,any unacknowledged SUBSCRIBE and UNSUBSCRIBE messages.

Context Overview

The MQTT Client LIB can support simultaneous MQTT connections to the servers. And the operational configuration and other related aspects of such a connection, in this software design, is referred as a 'context'. Therefore, the implied and intuitive sequence for the usage of MQTT Client library evolves to the following:

• Initialize LIB (singleton operation)
• Create one or more 'context(s)'
• Bidirectional transfer of MQTT packets
• Delete 'contexts'
• Exit LIB (singleton operation)

The Client LIB must be initialized only once in the system and the 'contexts' can be created and / or deleted by one or more applications (tasks). In a system, if more than one application is going utilize the services of the Client LIB, then the user must provision the platform specific support for Mutex.

Given, the richness of the features offered by the Client LIB, the services available to manage the contexts and transfer of MQTT packets need some elaboration. Specifically, the following aspects of the 'context' makes the Client LIB flexible and scalable across the deployments.

• 'Context' Management: Either in a Group or as an individual
• 'Context' Operation: Either callback (CB) or sync-wait (WT)

The afore mentioned parameters can be configured for a 'context' at the time of its creation but the restrictions outlined below must be adhered.

• There can be only one group of 'contexts' in the system. However, there can be as many as possible, the individual 'contexts' in the system along-with or without a single group-text.
  
• All the 'contexts' in the group will use the same mode of operation i.e. callback mode or otherwise. In other words, it is not possible to operate the constituent 'contexts' of the group in different modes. And the table below further enumerates the permissible combination of 'contexts' in the system.

Permissible 'Context' combination

The concept of 'group of contexts' is aimed at enabling a single application to manage multiple 'contexts' or connections to the same or different servers. Conversely, a single 'individual context' can be managed by an independent application. The Client LIB, for managing the 'group contexts' allocates, additional resources - primarily, the Client LIB is required to incorporate the 'loopback' (UDP) port in the system and the user should carefully choose and configure, without creating any conflict, the 'loopback' port in the system.

It is anticipated that an application which utilizes the services of the Client LIB can be a multi-task composition and the Client LIB will be required to maintain integrity of the operation and resources. In such a scenario, the application must indicate to the Client LIB that a dedicated RX task is being used to handle the packets from the network for a given 'context'. Such a configuration for the 'context' will enable the Client LIB to enforce steps to maintain integrity of the network resources.

Receive Operation

While the services for transmission of data to the network remains same across the various modes of 'Context', the receive operation has several flavors and are enumerated in the table below. Depending on the needs of the use-case, the application can choose the appropriate options.

RX 'Context' operation - options

Buffer Pool

The client 'context' allocates buffer internally to transact the network packets. Specifically, the 'context' needs to allocate a buffer for sending a message to the network. Similarly, the 'context', whilst operating in the call-back mode, must provision a buffer to receive a message from the network and hand it to the application. In the MQTT client LIB, the allocation of a buffer is made from a pre-configured pool and the user of the client LIB, as part of the initialization sequence, must provision adequate resources in the pool.

In MQTT, the QoS1 & QoS2 messages / packets need to be acknowleged by the remote entity. For transmission, the number of in-flight messages relates to the maximum number of packets that can remain un-acknowledged at any given point of time. For example, the value of single in-flight message would imply that there can be only one un-acknowledged message in the client 'context' and the subsequent packet can be scheduled for transmission, only if, the previous non-QoS0 packet has been acknowledged. After sending the packet to the network and if required, getting the ACK from the peer, the client LIB returns the packet to the buffer pool.

On the receiving side, if required, the appliaction can take over the ownership of a packet buffer delivered by the client LIB 'context' for further processing. Such an arrangement promotes 'zero copy' philosophy in a low power system. The mechanism of 'zero-copy' is suitable for scenarios, in which, the application is required to store or queue the packet for a differed or later processing. After the compeltion of the processing, the application has to 'free' the packet that it had taken from the client LIB 'context'. When freed, the packet is returned to the buffer pool of the client LIB.

The number of in-flight TX messages in the network and the number of RX packets including the ones that can be handed over to the application are dependent on the resources that have been made available in the buffer pool. In other words, the user of the client LIB must provision an adequate number of packet-buffers to commensurate with the intended run time configuration of transmit and receive buffers across all the 'contexts'. The routine / interface mqtt_client_buffers_register can be used to set-up the buffer pool.

Formula 1: 'Context' buffer pool = 'Context' in-flight TX buffers + 'Context' RX buffers (callback mode only)

Formula 2: Client 'LIB' buffer pool = Sum of all 'Contexts' buffer pool

Note:

: The RX buffers include the buffers that can be potentially handed over to the application. It is strongly recommended that the application ensures that the client 'context' always has ownership of one RX buffer at any time to support the incoming message from the network.

Formula 3: 'Context' buffer pool (with clean session as false) = 'Context' buffer pool (with clean session as true) + 1 (for CONNECT message)

The additional buffer for the 'clean session' = false configuration is required to support the dispatch of the CONNECT message to the network to establish the MQTT connection with the broker. The need of the additional buffer arises for the cases, in which, all the previous non-QOS0 message(s) (PUB / SUB / UNSUB) has / have not been acknowledged and the user application is holding all the RX buffers.

For example, for a minimalistic configuration of one in-flight message and one RX buffer with no handover of the packet to the user application, the size of the buffer pool for the configuration having 'clean session' as true will be 2. Where as, the size of buffer pool for the configuration 'clean session' as false will be (2 + 1) = 3.

Note:

: The client LIB does not support segregation of the buffers for sending (in-flight) and receiving. This responsibility is left to the application that uses the client 'context'.

Summary:

• The buffer pool must be provisioned with adequate resources to handle both the TX and RX operations.
• The size of the buffer pool must account for all the 'contexts' that will be used or has been configured.
• The number of buffers for a 'context' is established by adding the count of intended 'in-flight' messages to the network and when operating the 'context' in the callback mode, the number of overall RX packets including the ones that can be handed over to the application.
• The 'context' with 'clean session' = false needs an additional buffer to ensure that it is able to send the CONNECT message.
• The 'context' that is operating in the sync-wait mode does not need to include the RX packets in the configuration of the buffer pool.
• All the buffers in the pool has same size, therefore, the pool must be created for buffer that has adequeate length.