NOTE: The project and documentation presented on this page is a work in progress so provided as a separate zip file download. It will be released in the main FreeRTOS download following full review and completion of this documentation.

IoT MQTT
Library and Demo Application


On this page:

  1. The IoT MQTT library
  2. The IoT MQTT Demo Application

 

Introduction

The IoT MQTT (Message Queue Telemetry Transport) library provides a lightweight publish/subscribe (or PubSub) messaging protocol that runs on top of TCP/IP and is often used in Machine to Machine (M2M) and Internet of Things (IoT) use cases.  The IoT MQTT library implements a subset of the MQTT 3.1.1 protocol standard and uses the IoT Task Pool library for threading. 

The FreeRTOS IoT libraries can be used individually or collectively to create locally connected or internet connected MCU applications.  Each library can be freely used and is distributed under the MIT open source license.

 

MQTT Terminology

Broker/Server

The MQTT broker (or server) is the central server to which all MQTT clients connect. The MQTT broker manages message topics. When a client updates a topic, the message broker sends the new messages to all of the clients that are subscribed to the topic.

Topics

MQTT Messages are arranged in a hierarchy of topics that are addressed as UTF-8 strings. Levels in the topic hierarchy are separated by forward slash characters (‘/’) similar to how files are arranged on your local disk. Topic strings (paths) are used to access MQTT messages in the same way file paths are used to access files. 

Valid examples include:

“my_house/family_room/temperature” and “my_house/kitchen/temperature”.

MQTT Payloads

MQTT payloads are the body of each MQTT message – the application data carried by the MQTT packet.  The payload is data format agnostic and can support anything including text, images, binary numbers, etc.

Publish

When a client publishes to a topic on the MQTT broker, it is updating the data associated with that topic on the broker, and publishing new messages for the topic’s subscribers (if any). There is no limit to the number of clients that can publish to a topic.  Clients can only publish to one topic at a time. 

Subscribe

Once a client subscribes to a topic on the MQTT broker, it will receive all of the subsequent messages that have been published to the topic.    There is no limit to the number of clients that can subscribe to a topic.  Clients can subscribe to either a specific topic such as “my_house/kitchen/temperature” or to a range of topics (Topic Wildcards). 

Topic Filters

A topic filter is used by the client to define its subscription to topic(s).  A subscription’s topic filter can contain wild cards to subscribe to multiple topics. 

Topic Wildcards

Topic Wildcards enable a single subscription to receive updates from multiple topics. 

The multi-level (‘#’) wildcard is used to specify all remaining levels of hierarchies and must be the last character in the topic subscription string. 

  • If the client subscribes to “my_house/#”, then the client will receive the messages published to “my_house/family_room” and “my_house/kitchen”. 
  • If the client subscribes to “my_house/family_room/#”, then the client will receive the messages published to “my_house/family_room/temperature” and “my_house/family_room/humidity”.  However, it would not receive any messages published to “my_house/kitchen”. 

The single level (‘+’) wildcard is used to match one topic level of hierarchy.  The single (‘+’) wildcard can be used more than once in the topic filter and in conjunction with the multi-level wildcard. 

  • If the client subscribes to “my_house/+/temperature”, then the client will receive the messages published to both “my_house/family_room/temperature” and my_house/kitchen/temperature”.
  • If the client subscribes to “my_house/+/temperature/#, then the client will receive the messages published to both “my_house/family_room/temperature/sensor_1” and “my_house/kitchen/temperature/sensor_2” 

Quality of Service (QoS)

Quality of Service (QoS) refers to the reliability of message delivery between the publisher and subscriber. MQTT defines three quality of service levels.

  • QoS0 is the lowest reliability level and is defined as “at most once” delivery. QoS0 messages are the fastest to send but do not guarantee delivery.  This is analogous to an UDP packet on an IP network – the UDP/IP protocol does not guarantee that UDP packets will reach their intended destination.
  • QoS1 is defined as “at least once” delivery. QoS1 messages are guaranteed to be delivered (if delivery is feasible), but may get delivered multiple times (duplicated).
  • QoS2 is the highest reliability level and is defined as “exactly once” delivery. QoS2 messages are guaranteed to be delivered (if delivery is feasible) and guaranteed not to be duplicated.  QoS2 is the safest but the slowest quality of service level. The IoT MQTT library does not support QoS2. QoS2 like behavior can be achieved using QoS1 and filtering duplicate messages in the application.

Clean Session & Persistent Sessions

A client can specify a connection to an MQTT broker as a ‘clean session’ or a ‘persistent’ session.

If a client disconnects from a clean session, the broker will delete the client’s subscriptions, and the client must re-create the subscriptions the next time it connects. The client will not receive any messages that were published while disconnected.

If a client disconnects from a persistent session, the broker will queue (remember) the client’s subscriptions, and the broker will store all non QoS0 messages that were published while the client was disconnected.  When the client re-connects, its subscriptions are automatically re-established, and it will receive all of the messages that were published while disconnected.

Last Will and Testament 

The Last Will and Testament (LWT) message is an MQTT message used to notify other clients about an ungracefully disconnected client. Clients can get ungracefully disconnected if they are disconnected unexpectedly while an MQTT connection is active.

A Last Will and Testament message is a normal MQTT message that contains a topic, a QoS level and a payload.  Each client can optionally specify its own LWT message when it connects to a broker.  The broker stores this message so that if the client disconnects ungracefully, the broker will send the disconnected client’s LWT message to all the other clients that are subscribed to that last will message topic. 

 

 

Implementation

The IoT MQTT Library follows the MQTT v3.1.1 spec with the exception of the following:

  • QoS2 messages are not supported. It is recommended to utilize QoS1 messages then filter duplicate messages manually. 

 

 

IoT MQTT API Reference

Doxygen API documentation.

 

 

IoT MQTT Demo Project

Introduction

The basic MQTT demo project uses the FreeRTOS Windows port, enabling it to be built and evaluated with the free Community version of Visual Studios on Windows, so without the need for any particular MCU hardware.

The basic MQTT demo is intended to showcase only the basic MQTT use cases of connecting to a MQTT broker, publishing to a topic, and subscribing to a topic.  It uses the simpler asynchronous MQTT library API and does not create a secure connection. The basic MQTT demo can connect to any non-secure MQTT broker. The instructions provided below will demonstrate how to connect to either Mosquitto’s test broker hosted on the internet or on a server running locally on the host.

Note: Mosquitto is an open source MQTT message broker that supports MQTT versions 5.0, 3.1.1, and 3.1.  It is part of the Eclipse foundation and is an project.

Source Code Organization


Click to enlarge

The Visual Studio solution for the basic MQTT demo is called mqtt_demo.sln and is located in the /FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/mqtt directory of the main FreeRTOS download. Note:The project is not in the main FreeRTOS download yet so for now is provided as a separate zip file download.

The IoT MQTT library utilizes the IoT Task Pool library, and the MQTT demo project was created by adding the MQTT library source files shown in the image below into task pool demo project.

 

Building the Demo Project

The demo project uses the free community edition of Visual Studio

To build the demo:

  1. Open the /FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/mqtt/mqtt_demo.sln Visual Studio solution file from within the Visual Studio IDE
  2. Select ‘build solution’ from the IDE’s ‘build’ menu (or press F7)

 

Configuring the Demo Project

The demo uses the FreeRTOS+TCP TCP/IP stack, so follow the instructions provided for the TCP/IP starter project to ensure you:

  1. Have the pre-requisite components installed (such as WinPCap).
  2. Optionally set a static or dynamic IP address, gateway address and netmask.
  3. Optionally set a MAC address.
  4. Select an Ethernet network interface on your host machine.
  5. …and importantly test your network connection before attempting to run the MQTT demo.

All these setting should be performed in the MQTT demo project, not the TCP/IP starter project referenced from the same page! As delivered the TCP/IP stack is configured to use a dynamic IP address.

Configuring the MQTT Broker Connection

Mosquitto Public Message Broker (Web Hosted)

The demo project is pre-configured to communicate with Mosquitto’s publicly hosted message broker at “test.mosqitto.org” – so the network to which the demo is connected must have a DHCP service and internet access.  Note public MQTT brokers can be slow.

If you would like to connect to a different public broker then:

  1. Open FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/mqtt/DemoTasks/SimpleMQTTExamples.c.
  2. Edit the following lines to be correct for your chosen broker:

    #define mqttexampleMQTT_BROKER_ENDPOINT "test.mosquitto.org"
    #define mqttexampleMQTT_BROKER_PORT 1883

Mosquitto Local Message Broker (Host Machine)

It is also possible to run the Mosquitto broker locally, either on your host machine (the machine used to build the demo application), or another machine on your local network.  To do this:

  1. Follow the instructions on https://mosquitto.org/download/ to download and install Mosquitto locally.
  2. Find the IP address of your host machine (run the ipconfig command on Windows, or ifconfig on Linux or MAC OS).
  3. Open FreeRTOS-Plus/Demo/FreeRTOS_IoT_Libraries/mqtt/DemoTasks/SimpleMQTTExamples.c.
  4. Edit the following lines so mqttexampleMQTT_BROKER_ENDPOINT is set to the IP address of the machine on which Mosquitto is running. #define mqttexampleMQTT_BROKER_ENDPOINT "test.mosquitto.org"
    #define mqttexampleMQTT_BROKER_PORT 1883

Note: Port number 1883 is the default port number for unencrypted MQTT.  If you cannot use that port (for example if it is blocked by your IT security policy) then change the port used by Mosquitto to a high port number (for example something in the 50000 to 55000 range), and set mqttexampleMQTT_BROKER_PORT accordingly.

 

Functionality

The demo creates a single application task that loops through a set of examples that demonstrate how to connect to the broker, subscribe to a topic on the broker, publish to a topic on the broker, and disconnect from the broker again. The demo application both subscribes to and publishes to the same topic, as a result of which each time the demo publishes a message to the MQTT broker the broker sends the same message back to the demo application. The structure of the demo is shown below:

static void prvMQTTDemoTask( void *pvParameters )
{
IotMqttError_t xResult;
uint32_t ulNotificationValue = 0, ulPublishCount;
const uint32_t ulMaxPublishCount = 5UL;
const TickType_t xNoDelay = ( TickType_t ) 0;

  /* The MQTT library needs a task pool, so create the system task pool. */
  IotTaskPool_CreateSystemTaskPool( &( xTaskPoolParameters ) );

  /* MQTT library must be initialized before it can be used. This is just one
  time initialization. */
  IotMqtt_Init();

  for( ;; )
  {
    /****************************** Connect. ******************************/

    /* Establish a connection to the MQTT broker. This example connects to
     * the MQTT broker as specified in mqttexampleMQTT_BROKER_ENDPOINT and
     * mqttexampleMQTT_BROKER_PORT at the top of this file. Please change
     * it to the MQTT broker you want to connect to. Note that this example
     * does not use TLS and therefore will not work with AWS IoT. */
    prvMQTTConnect();


    /**************************** Subscribe. ******************************/

    /* The client is now connected to the broker. Subscribe to the topic
     * as specified in mqttexampleTOPIC at the top of this file.  This
     * client will then publish to the same topic it subscribed to, so will
     * expect all the messages it sends to the broker to be sent back to it
     * from the broker. */
    prvMQTTSubscribe();


    /*********************** Publish 5 messages. **************************/

    /* Publish a few messages while connected. */
    for( ulPublishCount = 0; ulPublishCount < ulMaxPublishCount; ulPublishCount++ )
    {
      /* Publish a message on the mqttexampleTOPIC topic as specified at
      the top of this file. */
      prvMQTTPublish();

      /* Since we are subscribed to the same topic as we published on, we
      will get the same message back from the MQTT broker. Wait for the
      message to be received which is informed to us by the publish
      callback (prvExample_OnMessageReceived) by setting the
      mqttexampleMESSAGE_RECEIVED_BIT in this task's notification
      value. Note that the bit is cleared in the task's notification
      value to ensure that it is ready for next message. */
      xTaskNotifyWait( 0UL, /* Don't clear any bits on entry. */
                       mqttexampleMESSAGE_RECEIVED_BIT, /* Clear bit on exit. */
                       &( ulNotificationValue ),/* Obtain the notification value. */
                       pdMS_TO_TICKS( mqttexampleMQTT_TIMEOUT_MS ) );
      configASSERT( ( ulNotificationValue & mqttexampleMESSAGE_RECEIVED_BIT ) == 
                                                mqttexampleMESSAGE_RECEIVED_BIT );
    }


    /******************* Unsubscribe and Disconnect. **********************/

    /* Unsubscribe from the topic mqttexampleTOPIC and disconnect gracefully. */
    prvMQTTUnsubscribe();
    prvMQTTDisconnect();

    /* Wait for the disconnect operation to complete which is informed to us
     * by the disconnect callback (prvExample_OnDisconnect)by setting
     * the mqttexampleDISCONNECTED_BIT in this task's notification value.
     * Note that the bit is cleared in the task's notification value to
     * ensure that it is ready for the next run. */
    xTaskNotifyWait( 0UL, /* Don't clear any bits on entry. */
                     mqttexampleDISCONNECTED_BIT, /* Clear bit on exit. */
                     &( ulNotificationValue ), /* Obtain the notification value. */
                     pdMS_TO_TICKS( mqttexampleMQTT_TIMEOUT_MS ) );
    configASSERT( ( ulNotificationValue & mqttexampleDISCONNECTED_BIT ) == 
                                                      mqttexampleDISCONNECTED_BIT );

    /* Wait for some time between two iterations to ensure that we do not
    bombard the public test mosquitto broker. */

    vTaskDelay( pdMS_TO_TICKS( 5000 ) );
  }
}

The screenshot below shows the expected output when the demo is executing correctly. 


Click to enlarge

Connecting to the MQTT Broker

The function prvMQTTConnect() demonstrates how to establish an unencrypted connection to a MQTT broker with clean session. It uses the FreeRTOS+TCP network interface which is implemented in the file FreeRTOS-Plus\Source\FreeRTOS-IoT-Libraries\abstractions\platform\freertos\iot_network_freertos.c. The definition of the function is shown below:

 

static void prvMQTTConnect( void ) 
{ 
  IotMqttError_t xResult;
  IotNetworkServerInfo_t xMQTTBrokerInfo;
  IotMqttNetworkInfo_t xNetworkInfo = IOT_MQTT_NETWORK_INFO_INITIALIZER;
  IotMqttConnectInfo_t xConnectInfo = IOT_MQTT_CONNECT_INFO_INITIALIZER;

  /******************* Broker information setup. **********************/ 
  xMQTTBrokerInfo.pHostName = mqttexampleMQTT_BROKER_ENDPOINT;
  xMQTTBrokerInfo.port = mqttexampleMQTT_BROKER_PORT;

  /******************* Network information setup. **********************/ 
  /* No connection to the MQTT broker has been established yet and we want to
  establish a new connection. */ 
  xNetworkInfo.createNetworkConnection = true;
  xNetworkInfo.u.setup.pNetworkServerInfo = &( xMQTTBrokerInfo );

  /* This example does not use TLS and therefore pNetworkCredentialInfo must
  be set to NULL. */ 
  xNetworkInfo.u.setup.pNetworkCredentialInfo = NULL;

  /* Use FreeRTOS+TCP network. */
  xNetworkInfo.pNetworkInterface = IOT_NETWORK_INTERFACE_FREERTOS;

  /* Setup the callback which is called when the MQTT connection is 
  disconnected.  The task handle is passed as the callback's context so the
  callback can send a direct to task notification to the calling task. */
  xNetworkInfo.disconnectCallback.pCallbackContext = xTaskGetCurrentTaskHandle();
  xNetworkInfo.disconnectCallback.function = prvExample_DisconnectCallback;

  /****************** MQTT Connection information setup. ********************/
  /* This example does not use TLS and therefore won't work with AWS IoT. */
  xConnectInfo.awsIotMqttMode = false;

  /* Since we are starting with a clean session, there are no previous
  subscriptions to be restored. */
  xConnectInfo.cleanSession = true;

  /* Start with a clean session i.e. direct the MQTT broker to discard any
  previous session data. Also, establishing a connection with clean session
  will ensure that the broker does not store any data when this client
  gets disconnected. */
  xConnectInfo.pPreviousSubscriptions = NULL;
  xConnectInfo.previousSubscriptionCount = 0;

  /* We do not want to publish Last Will and Testament (LWT) message if the
  client gets disconnected. */
  xConnectInfo.pWillInfo = NULL;

  /* Send an MQTT PING request every minute. */
  xConnectInfo.keepAliveSeconds = mqttexampleKEEP_ALIVE_SECONDS;

  /* The client identifier is used to uniquely identify this MQTT client to
  the MQTT broker. */
  xConnectInfo.pClientIdentifier = mqttexampleCLIENT_IDENTIFIER;
  xConnectInfo.clientIdentifierLength = strlen( mqttexampleCLIENT_IDENTIFIER );

  /* This example does not use any authentication and therefore username and
  password fields are not used. */
  xConnectInfo.pUserName = NULL;
  xConnectInfo.userNameLength = 0;
  xConnectInfo.pPassword = NULL;
  xConnectInfo.passwordLength = 0;

  /* Establish the connection to the MQTT broker - It is a blocking call and
  will return only when connection is complete. */
  xResult = IotMqtt_Connect( &( xNetworkInfo ),
                             &( xConnectInfo ),
                             mqttexampleMQTT_TIMEOUT_MS,
                             &( xMQTTConnection ) );
  configASSERT( xResult == IOT_MQTT_SUCCESS );
}

Subscribing to a MQTT Topic

The function prvMQTTSubscribe() demonstrates how to subscribe to a topic filter on the MQTT broker. The example demonstrates how to subscribe to one topic filter but it possible to pass a list of topic filters in the same subscribe API call to subscribe to more than one topic filters. The example code uses the synchronous version of the subscribe API which returns only when the subscribe operation is complete or a timeout occurs. The definition of the function is shown below:

 

static void prvMQTTSubscribe( void )
{
IotMqttError_t xResult;
IotMqttSubscription_t xMQTTSubscription;

  /* Subscribe to the mqttexampleTOPIC topic filter. */
  xMQTTSubscription.qos = IOT_MQTT_QOS_1;
  xMQTTSubscription.pTopicFilter = mqttexampleTOPIC;
  xMQTTSubscription.topicFilterLength = strlen( mqttexampleTOPIC );
  xMQTTSubscription.callback.pCallbackContext = xTaskGetCurrentTaskHandle();
  xMQTTSubscription.callback.function = prvExample_PublishCallback;

  /* Use the synchronous API to subscribe - It is a blocking call and only
  returns when the subscribe operation is complete. *//

  xResult = IotMqtt_TimedSubscribe( xMQTTConnection,
                                    &( xMQTTSubscription ),
                                    1, /* Subscribing to one topic filter. */
                                    0, /* flags - currently ignored. */ 
                                    mqttexampleMQTT_TIMEOUT_MS );
  configASSERT( xResult == IOT_MQTT_SUCCESS );
}


Publishing to a Topic 

The function prvMQTTPublish demonstrates how to publish on a topic on the MQTT broker. It uses the synchronous version of the publish API which returns only when the publish operation is complete or a timeout occurs. The definition of the function is shown below:

 

static void prvMQTTPublish( void )
{
IotMqttError_t xResult;
IotMqttPublishInfo_t xMQTTPublishInfo;

  /* Publish a message with QoS1 on the mqttexampleTOPIC topic. Since we are
   * subscribed to the same topic, the MQTT broker will send the same message
   * back to us. It is verified in the publish callback. */ 
  xMQTTPublishInfo.qos = IOT_MQTT_QOS_1;
  xMQTTPublishInfo.retain = false;
  xMQTTPublishInfo.pTopicName = mqttexampleTOPIC;
  xMQTTPublishInfo.topicNameLength = ( uint16_t ) strlen( mqttexampleTOPIC );
  xMQTTPublishInfo.pPayload = mqttexampleMESSAGE;
  xMQTTPublishInfo.payloadLength = strlen( mqttexampleMESSAGE );
  xMQTTPublishInfo.retryMs = mqttexamplePUBLISH_RETRY_MS;
  xMQTTPublishInfo.retryLimit = mqttexamplePUBLISH_RETRY_LIMIT;

  /* Use the synchronous API to publish - It is a blocking call and only
   * returns when the publish operation is complete. */ 
  xResult = IotMqtt_TimedPublish ( xMQTTConnection,
                                  &( xMQTTPublishInfo ),
                                  0,  /* flags - currently ignored. */
                                  mqttexampleMQTT_TIMEOUT_MS );  
  configASSERT( xResult == IOT_MQTT_SUCCESS );
}

Unsubscribing from a Topic

The function prvMQTTUnsubscribe demonstrates how to unsubscribe from a topic on the MQTT broker. It uses the synchronous version of the unsubscribe API which returns only when the unsubscribe operation is complete or a timeout occurs. The definition of the function is shown below:

 

static void prvMQTTUnsubscribe( void )
{
IotMqttError_t xResult;
IotMqttSubscription_t xMQTTSubscription;

  /* Unsubscribe from the mqttexampleTOPIC topic filter. */ 
  xMQTTSubscription.pTopicFilter = mqttexampleTOPIC;
  xMQTTSubscription.topicFilterLength = ( uint16_t ) strlen( mqttexampleTOPIC );
   
  /* The following members of the IotMqttSubscription_t are ignored by the
  unsubscribe operation. Just initialize them to avoid "use of uninitialized
  variable" warnings. */ 
  xMQTTSubscription.qos = IOT_MQTT_QOS_1;
  xMQTTSubscription.callback.pCallbackContext = NULL;
  xMQTTSubscription.callback.function = NULL;

  /* Use the synchronous API to unsubscribe - It is a blocking call and only
  returns when the unsubscribe operation is complete. */
  xResult = IotMqtt_TimedUnsubscribe( xMQTTConnection,
                                      &( xMQTTSubscription ),
                                      1, /* Unsubscribing from 1 topic filter. */
                                      0, /* flags - currently ignored. */
                                      mqttexampleMQTT_TIMEOUT_MS );
  configASSERT( xResult == IOT_MQTT_SUCCESS );
}

Disconnecting from the MQTT Message Broker

The function prvMQTTDisconnect() demonstrates how to gracefully disconnect from the MQTT broker. The definition of the function is shown below:

 

 static void prvMQTTDisconnect( void )
{
   /* Send a MQTT DISCONNECT packet to the MQTT broker to do a graceful
   disconnect. */ 
   IotMqtt_Disconnect( xMQTTConnection,
                       0 /* 0 means a graceful disconnect */ );
}