Introduction

The MQTT protocol, standing for Message Queuing Telemetry Transport, was conceived in 1999 by Dr. Andy Stanford-Clark of IBM and Arlen Nipper of Arcom, which is now known as Eurotech. The invention of MQTT was driven by the need for a lightweight communication method that could operate over constrained networks with limited bandwidth, such as those used in the oil and gas industry. Specifically, engineers required a protocol that would minimize bandwidth usage and battery loss to effectively monitor oil pipelines via satellite. 

The MQTT protocol consists of two main components: clients and a broker (also known as an MQTT server). 

Clients

An MQTT client is any device that runs an MQTT library and connects to an MQTT broker over a network. The clients can be very diverse ranging from a small IoT sensor, a mobile phone app, to a large server. They interact with the broker using a publish/subscribe messaging pattern. 

  • Publishers:  A client becomes a publisher when it sends messages to the broker. It assigns a topic to each message, which is used by the broker to route the message to interested subscribers. The data payload can contain anything from telemetry data like temperature readings to simple text messages. 
  • Subscribers:  A client is a subscriber when it receives messages from the broker. It informs the broker of its interest in particular topics (or topic patterns) and the broker sends all messages with matching topics to the client. 
  • Connections:  Prior to any communication, a client must establish a connection with the broker using TCP/IP. During this process, it can specify certain connect options such as: 
  1. QoS Level for the Connection
  2. A unique client ID
  3. Will messages – in case the client disconnects unexpectedly
  4. Keep Alive period in seconds to permit the broker to keep the connection alive in periods of inactivity. 

Broker

The MQTT broker, sometimes referred to as an MQTT server, is the central hub responsible for receiving all messages, filtering them, determining who is interested in them, and publishing those messages to the subscribed clients. The Broker handles various tasks: 

  • Session Management: Maintains client sessions, including subscription information and missed messages if persistent sessions are configured and the client subscribes with QoS level 1 or 2. 
  • Message Distribution: Forwards messages to interested subscribers based on topic matching. 
  • QoS Level Handling: Manages the different Quality of Service levels for message delivery from publishers to subscribers. 
  • Security: Enforces security measures such as authentication and authorization to ensure only permitted clients can connect and subscribe to specific topics. 
  • Retained Messages: Can store the last known good value for a topic path so new subscribers immediately receive current data upon subscription. 
  • Client Status: Tracks whether clients are connected or not and sends messages with the 'will' option to specified topics if a client disconnects unexpectedly. 

In industrial communication and automation, MQTT facilitates the exchange of real-time data between machines and systems with minimal memory or computing power. It supports bridging, allowing for data integration across different plants or locations. MQTT's lightweight nature also makes it scalable and secure when used over SSL/TLS, which encrypts the data for safe transmission. 

When compared to other protocols like AMQP and CoAP, MQTT stands out for its simplicity and efficiency. AMQP (Advanced Message Queuing Protocol), on the other hand, is an open standard application layer protocol known for its message orientation, routing, queuing, security, and reliability. It provides more capabilities for general-purpose message queuing, including flexible message routing and store-and-forward queuing, albeit with some efficiency trade-offs and additional complexity compared to MQTT. CoAP, on the other hand, is designed for constrained devices and enables them to communicate with the wider Internet using similar protocols. It is primarily a one-to-one protocol for transferring state information between client and server, with built-in support for content negotiation and discovery, which allows devices to probe each other to find ways of exchanging data. CoAP operates over UDP, which requires minimal overheads and empowers prompt wake-up times and longer sleepy states, thus achieving longer-lasting battery life for IoT and M2M communication. When comparing MQTT, AMQP and CoAP, it is evident that while MQTT excels in scenarios requiring reliable and fast message delivery with support for many devices. 

Applications of MQTT in Industry

In the oil and gas industry, MQTT plays a transformative role by consolidating data from various sources onto a single platform, thereby eliminating data silos. This consolidation enhances operational efficiency by providing real-time remote monitoring, data-driven analytics, and proactive maintenance. The protocol's ability to monitor environmental parameters such as air quality, gas leaks, or radiation levels and send immediate alerts if safety thresholds are crossed further ensures worker and environmental safety. Its support for persistent sessions reduces the time needed to reconnect clients to brokers, which is beneficial for IoT devices connected via unreliable cellular networks. Additionally, MQTT's compatibility with modern security practices, such as TLS encryption and authentication protocols like OAuth, helps secure the transmission of sensitive data. MQTT's publish-subscribe architecture enhances operational efficiency, minimizes downtime, and boosts safety through real-time remote monitoring, data-driven analytics, and proactive maintenance. The protocol's ability to support asynchronous industrial communication further decouples message producers and consumers, allowing for efficient data acquisition, remote control, and device management in various industrial automation scenarios.  

Additionally, MQTT's integration with existing industrial communication protocols like OPC UA and Modbus is also noteworthy. While OPC UA is known for its robust messaging protocol capable of handling larger data volumes and is better suited for high-speed and secure networks, MQTT utilizes a lightweight messaging protocol, making it a perfect fit for low-bandwidth or unreliable networks. The integration of OPC UA over MQTT brings together the strengths of both protocols. OPC UA provides a standardized data modelling framework and security features, while MQTT offers an efficient communication model with its lightweight publish-subscribe mechanism. This combination allows for secure and reliable communication with real-time data exchange and complex information sharing, which is crucial for Information Technology (IT) and Operational Technology (OT) convergence in industrial systems. The convergence of IT and OT through OPC UA and MQTT enables organizations to optimize their operations. It allows for improved maintenance procedures, reduced downtime, and the ability to compare performance across different plants and production lines. This leads to more cost-effective and resource-efficient manufacturing operations, driven by data from sales and inventory to optimize equipment use, minimize maintenance, and reduce unsold inventory. Meanwhile, the combination of MQTT and Modbus result in system leveraging MQTT's lightweight and efficient data transmission capabilities complementing Modbus robustness and reliability in industrial settings. By converting Modbus RTU or TCP data to MQTT messages, industries can enhance operational efficiency and enable real-time remote monitoring and control. This conversion facilitates the seamless flow of data from the factory floor to cloud-based platforms, where advanced analytics and proactive maintenance strategies can be applied. 

In conclusion, the bridging of Modbus to MQTT involves the use of edge devices that convert Modbus protocol into MQTT, transmitting the data to an MQTT broker, which then handles the message forwarding. This architecture not only improves security by supporting SSL/TLS encryption but also ensures reliable message transmission, even in cases of network interruptions, through reconnection and communication restoration mechanisms. 

Advantages of MQTT

The primary advantages of MQTT are its security, reliability, and efficiency pertaining to industrial communication.  

Security

One of the key advancements is the native support for SSL/TLS encryption. This encryption ensures that data transmitted between devices and brokers is protected from eavesdropping, safeguarding sensitive information. 

The implementation of SSL/TLS in MQTT involves a handshake process where the client and server establish a secure connection by exchanging messages to agree on encryption algorithms, session keys, and authentication methods. The use of SSL certificates and/or passwords facilitates identity verification, authentication, and authorization between clients and the broker. These certificates are digital documents that contain the public key of the server or client and are signed by a trusted Certificate Authority (CA), establishing the identity of the communicating parties. 

MQTT's security model also supports various authentication methods, including password-based authentication, where the broker verifies the client's credentials, such as username, client ID, and password. Advanced mechanisms like mutual TLS (mTLS) authentication provide an even higher level of security by requiring both the client and the server to authenticate each other using digital certificates. 

Furthermore, MQTT's security framework is designed to protect against common cyber threats. It guards against man-in-the-middle attacks, where an attacker intercepts and potentially manipulates messages exchanged between clients and the broker. By encrypting the content of messages and digitally signing them, MQTT ensures data integrity, preventing unauthorized tampering or modification during transmission. 

Reliability

MQTT's Quality of Service (QoS) levels are one of its standout features regarding reliability. The protocol offers three QoS levels to suit different messaging needs: 

  1. QoS 0 – At most once: This level delivers messages without requiring an acknowledgement from the receiving client. It is the fastest and requires the least overhead but does not guarantee message delivery. It is suitable when timely delivery is more critical than accurate delivery, such as with environmental sensor data that updates regularly.                                                                                                                                                                                              
  2. QoS 1 – At least once: At this level, messages are delivered at least once to the receiver by sending a PUBLISH packet to the MQTT broker with the QoS 1 flag set. The sender stores the message until it receives an acknowledgement from the receiver with a PUBACK packet as an acknowledgment that it has received the message and taken responsibility for delivering it to the appropriate subscribing clients. If an acknowledgement is not received it assumes that the PUBLISH packet was lost in transit and therefore resends the PUBLISH packet. This could result in the broker receiving multiple copies of the same message if there are issues with the network, leading to potential duplicates on the subscribing clients' side.
  3. QoS 2 – This is the highest QoS level, providing guaranteed delivery of messages exactly once. This is achieved through a four-step handshake process between the sender and the receiver. The process begins with the sender publishing a message, which is followed by a PUBREC packet from the receiver acknowledging the receipt of the PUBLISH packet. The sender then responds with a PUBREL packet, indicating that it has received the PUBREC and is releasing the message. Finally, the receiver sends a PUBCOMP packet back to the sender to confirm the completion of the exchange. This sequence ensures that messages are not lost or duplicated during transmission. It provides the most assurance but also introduces additional overhead, making it suitable for operations where message duplication or loss cannot be tolerated, like financial transactions within industrial processes. 

Efficiency

Unlike traditional client-server models, MQTT utilizes publish-subscribe model allows for efficient message distribution and decouples the producer and consumer of the message, resulting highly efficient, scalable, and flexible systems. Here's why this model is beneficial: 

Decoupling of Devices

In the publish-subscribe model, the producers of messages (publishers) are decoupled from the consumers of messages (subscribers). This means that devices do not need to have specific knowledge about each other, such as subscribers' identities, locations, and IP addresses—an advantage in dynamic industrial environments where devices may often change or move. Consider a manufacturing plant where various field components such as sensors, actuators, and PLCs need to communicate effectively. In traditional systems, these devices would be tightly coupled, requiring direct and often complex interactions. However, with the publish-subscribe model, each device operates independently. For instance, a temperature sensor in the plant can publish its readings to a central broker without needing to know which PLCs or other devices will use this data. 

The central broker, which acts as the nerve center of this communication paradigm, manages all data subscriptions. PLCs or other interested parties can subscribe to the temperature readings and receive updates automatically from the broker. This setup eliminates the need for constant polling of devices for new messages, thereby reducing network traffic and increasing scalability. 

Scalability

The broker-based architecture allows for effortless scalability. Many devices can publish their data to a broker, and an equally large number can subscribe to topics of interest, without requiring point-to-point connections between devices. Furthermore, the broker can manage a vast number of concurrent connections. As demands grow and more devices connect to the network, the broker merely needs to handle additional publish requests and manage new subscriptions, all without the need for individual clients to manage multiple connections themselves. For instance, in a manufacturing setting with a multitude of sensors and actuators spread across the facility, each sensor can publish its readings—such as temperature, pressure, or speed—to a specific topic managed by the broker. PLCs, which are programmed to respond to certain thresholds or changes in the environment, can subscribe to these topics. When a sensor's reading crosses a predefined limit, the specific PLC receives this information through the broker and can take appropriate action, such as adjusting a control valve or shutting down a conveyor belt. 

Efficient Bandwidth Usage

Since clients only receive messages they are subscribed to, bandwidth is not wasted transmitting all data to all devices. Control messages can be as small as two bytes, and message headers are also kept to a minimum size to ensure that network bandwidth is used as efficiently as possible. Each device gets only the information it needs, which conserves network bandwidth and reduces unnecessary data traffic. For instance, in a factory where predictive maintenance and flexible manufacturing are critical, the use of Pub/Sub can support the production of highly configured goods. Devices like sensors and actuators no longer need to send their data to a master PLC for processing. Instead, they publish their data, such as temperature readings or cycle times, to a message broker. The broker then distributes this information to all subscribed entities, which could include other PLCs, monitoring systems, or even cloud-based analytics platforms. 

Flexibility in Data Routing

Topics in MQTT facilitate sophisticated data routing. Publishers send messages to topics that subscribers listen to, and the same message can be delivered to multiple subscribers, offering great flexibility and the capability to implement complex communication patterns without additional programming on the devices themselves. The model also allows for shared subscriptions, where messages can be load-balanced among multiple subscribers within a subscription group. This feature further enhances the flexibility of the Pub/Sub model by allowing efficient message distribution even when there are numerous subscribers. In the realm of smart homes, for instance, the pub-sub model is instrumental in enabling sensors to communicate with actuators, such as turning on lights when someone enters a room. This example underscores the model's capability to streamline interactions within an Internet of Things (IoT) ecosystem. 

Two-Way Communication

Through subscriptions, devices can not only publish data but also to listen for commands and updates. This bidirectional communication is achieved through subscriptions, where devices can subscribe to specific topics and receive relevant messages from the control system or other devices. For instance, consider a scenario where a temperature sensor in a smart home setup is designed to monitor the ambient conditions of a room. This sensor acts as a publisher, periodically sending temperature readings to a designated topic within the pub-sub system. An air conditioning system, equipped with the capability to subscribe to specific topics, listens for updates on the temperature topic. When the temperature sensor publishes a new reading indicating that the room's temperature has risen above a certain threshold, the air conditioning system, as a subscriber, receives this message. It then processes the information and responds by adjusting its settings to cool the room down to the desired temperature. 

Real-time Communication

The broker acts as a central hub in the publish-subscribe model. When a publisher has a new message to send, it publishes this message to a topic on the broker. The broker then handles the delivery of that message to all subscribers who have previously subscribed to that topic. Since the broker maintains persistent connections with both publishers and subscribers, messages are pushed out as they arrive without waiting for subscribers to request them, in contrast to polling mechanisms. This immediate forwarding of messages reduces latency and ensures near real-time communication. Subscribers can express interest in specific topics, and the broker will ensure that they only receive messages published under those topics. For instance, in an industrial setting, sensors on a conveyor belt might publish readings to the topic "conveyor/speed," and only the necessary control systems, which are subscribed to that topic, would receive those messages. 

Example - Industrial Sensor Monitoring

Consider a manufacturing plant with various sensors that monitor equipment status and environmental conditions. These sensors act as publishers and send data continuously to topics such as "temperature/sensor-1" or "equipment/machine-3/status". 

An MQTT broker manages these topics. If the temperature regulator system subscribes to "temperature/sensor 1", it immediately receives updates whenever the sensor publishes new temperature readings. In case a threshold is exceeded, the system can instantly react by activating a cooling system; similarly, if a machine reports a fault condition under "equipment/machine 3/status," the maintenance team’s monitoring system, subscribed to that topic, gets alerted so they can quickly address the issue. 

Adaptable to Various Network Conditions

The model fits well with harsh network conditions common in remote industrial sites, due to its ability to support event-driven architectures and asynchronous parallel processing, which are essential for improving performance, reliability, and scalability. This is particularly beneficial in remote industrial environments where network conditions can be challenging and unpredictable. The Pub/Sub model simplifies communication by removing the need for point-to-point connections, instead relying on a single connection to a message topic that manages subscriptions and message delivery. In scenarios where network architecture may change, such as in remote industrial sites, Pub/Sub supports asynchronous message delivery, ensuring reliable message delivery even when the message broker's topic architecture changes. This builds elasticity into the system, as the business logic does not depend on the number of active publishers and subscribers at any given instance. 

The pub/sub model plays a crucial role in both predictive maintenance and flexible manufacturing by facilitating real-time data exchange and increased flexibility. This model allows for asynchronous communication between services producing messages and those processing them, with latencies on the order of 100 milliseconds. In industrial networks, the pub/sub paradigm is gaining traction through emerging standards like Message Queueing Telemetry Transport (MQTT) and OPC Unified Architecture (OPC UA) with extensions for time-sensitive networking (TSN). 

Current and Emerging trends in MQTT and Hilscher portfolio

The MQTT (Message Queuing Telemetry Transport) protocol is experiencing a surge in popularity within Industry 4.0, as it becomes increasingly integral to smart manufacturing and the broader scope of the Industrial Internet of Things (IIoT). As an ISO 20922 standard, MQTT facilitates the connection of IoT devices, offering a lightweight and efficient messaging system that is particularly well-suited for environments with limited bandwidth. 

One of the most significant trends in MQTT's application is its role in enabling seamless data exchange between Manufacturing Execution Systems (MES) and Supervisory Control and Data Acquisition (SCADA) systems within the manufacturing sector. This interoperability extends beyond level 3 automation, allowing for integration from the factory floor to cloud-based services. MQTT protocols publish/subscribe model decouples communication between clients, which means that various systems and vendors can implement MQTT independently, fostering interoperability across different platforms. 

A promising application for this is Hilscher’s netFIELD App MQTT Broker, a ready-to-use container that can be deployed on any netFIELD Edge device or its virtual counterpart. The MQTT Broker plays a pivotal role in the IIoT ecosystem by enabling the conversion of process data from fieldbus and RTE networks into the MQTT protocol, thus adding value through passive network data tapping. The netFIELD application container reference architecture, which includes the MQTT broker, also offers interfaces for inter-container communication using MQTT. This design allows for seamless integration and communication within the netFIELD platform, enhancing the overall functionality and flexibility of IoT solutions. 

Hilscher is a pioneer in industrial communication technologies and recognizes the importance of MQTT in the evolving landscape of the Industrial Internet of Things (IIoT). The company's netFIELD technology provides a holistic IIoT platform that simplifies centralized edge management and integrates edge gateways with container management. The netFIELD platform provides a reliable data infrastructure solution that supports decentralized IoT projects and ensures efficient and secure data processing and management. MQTT is also integrated into Hilscher's IoT-enabled netX firmwares to allow companies to run conventional industrial communication protocols and MQTT communication in parallel. 

Our products

PCカードM.2 2230 Key A+E - リアルタイム・イーサネット

PCベースのオートメーション用通信

netRAPID 51/52ローダブルファームウェア- EtherNet/IPアダプタ、MQTTクライアント、OPC UAサーバ

NRP 51-REのフルバージョン、6時間のサポート付き

PCカードMini PCI Expressハーフサイズ - リアルタイム・イーサネット

PCベースのオートメーション用通信

開発ボード

 

メモリ増設netX 90チップキャリア - プログラミングプラットフォーム

超小型netX 90デザイン - 十分にテスト済み

最小サイズのマルチプロトコルSoC