Skip to content

Concepts

Introduction

Kamea supports MQTT as a communication interface with the devices. The MQTT interface supports sending telemetries, configurations, commands and connectivity state.

Kamea uses RabbitMQ as MQTT broker and supports the MQTT version 3.1.

This documentation will present how to use this interface with your devices.

Prerequisites

To fully understand this documentation, you have to know how to provision a device and how the ingestion cycle works.

Architecture

The global MQTT architecture can be summarized by the following schema:

The API can communicate with the devices over MQTT. But in most of the cases the messages sent over MQTT will not be sent directly to the API, instead they will be pushed into AMQP queues. Note that because RabbitMQ is used as MQTT and AMQP broker this operation will be done seamlessly by RabbitMQ internal routing. These AMQP queues allow us to have different workers, called AMQP consumers, to process all the messages concurrently. Delegating the processing of this messages to a dedicated service prevent us from overloading the API and allow us to scale easily when needed. Once the messages are processed, the extracted data is pushed in the bus to continue the ingestion cycle. In some specific cases (for example command response) the devices will answer directly to the API in MQTT.

Use cases

The previous schema showed that the communications handled by the broker include different use cases.

Cloud to device - Configurations and command requests

The first case is when the API initiates the communication with a device. This requires the device to previously have opened a MQTT connection to the MQTT broker. This occurs when the API sends a desired configuration or a command to a device. In this case, the message is sent directly from the API to the MQTT broker, and is then received by the device.

Device to cloud - Telemetries, states and command responses

The second case is when the device sends a telemetry. These messages are sent autonomously by the device and there is no constraint on the frequency of those messages. Consequently, the amount of telemetry data to process can be really high even with only a few devices. To avoid overloading the API, these messages are processed asynchronously by the ingestion chain. RabbitMQ is configured to internally forward those messages to an AMQP queue. Then, the AMQP consumers read the messages, and forward them to the rest of the ingestion chain.

The third case is when the device reports a configuration to Kamea. These messages can be sent autonomously by the device or by a reaction to an instruction sent by Kamea, for example after receiving a "desired configuration" the device can send a "reported configuration" to confirm the change has been made. But there are no time constraint to send these messages so they can be sent at any time. Like the telemetries, these messages will be routed to the AMQP queues to be processed asynchronously by the AMQP Consumers and the "ingestion cycle".

The fourth case is when a device answers a command. In this specific case, the device has to answer in the given time otherwise the command will be considered failed. It means that the command response has to be processed as soon as possible to answer the request before the allocated time runs out. So, contrary to telemetries and reported configurations, the command response is directly processed by the API instead of the AMQP consumers.

Broker configuration

The last communication type is done between Kamea API and RabbitMQ management API.

The typical use case is provisioning devices. If a device is provisioned through Kamea, the API sends an HTTP request to RabbitMQ to create the necessary user and rights. Or in the other way, during "auto-provisioning", the device connects to RabbitMQ before being created in Kamea. In that case, RabbitMQ sends a request to the Kamea API to create the device in the system.

The other use case of these HTTP communications is to retrieve the connectivity state of the devices. The Kamea API polls RabbitMQ to get the list of the currently connected devices.

TLS

All protocols used by RabbitMQ are secured with TLS by default, as presented on the following schema:

By default, RabbitMQ supports login/password and certificate based authentication. When using a certificate instead of a login/password, the certificate will be verified, using RabbitMQ trust store, to enforce mTLS.

Provisioning

Manual provisioning

There are two ways to provision a device in MQTT: with login/password or with a TLS certificate. Both are available through the Kamea API or the Device Management UI. First, in both cases, the device must be created through the API.

  • When provisioning the device with login/password, Kamea generates a login/password pair that needs to be provided to the device to allow it to connect to the MQTT broker.
  • When provisioning with a CA-signed certificate, the certificate Common Name must be provided. Note that the CA that emitted this certificate has to be whitelisted in the MQTT broker. The device then authenticates itself through mTLS with its certificate.
  • When provisioning with a self-signed certificate, the certificate must be uploaded to the API. It will the be store in RabbitMQ's trust store.

Once the device is provisioned, the list of its topics used for the different features will be accessible from the Kamea API or the Device Management UI.

Auto-provisioning

Auto-provisioning refers to devices declaring themselves to the platform without any manual intervention. This feature requires using CA-signed certificates.

If the Root CA that signed the device certificate is whitelisted in the broker, the device will be able to connect to the broker and the device will be created automatically in Kamea afterwards. The Common Name (CN) of the certificate will be used as the device name in Kamea.

Manual provisioning vs Auto-provisioning

When provisioning a device with TLS certificates, auto-provisioning appears to be simpler than certificate provisioning through Kamea. Indeed, in both cases the Root CA has to be white-listed but for certificate provisioning you need to add a manual step by creating the device on Kamea, while it will be created automatically with auto-provisioning. Manual provisioning is in fact used for one specific case: to pre-configure a device. In some cases you will want to preset some desired configurations, for example if the device will start in an unusual environment, for the device to receive on its first connection.

Interface Name

Once a device is provisioned, Kamea assigns it an interface name. Depending on the way it has been provisioned, this name can be different. If it has been provisioned with a login/pwd, its interface name will be a randomly generated UUID. When provisioned with a certificate, it will be the Common Name (CN) of the certificate.

The interface name is used by the API to identify the devices on RabbitMQ since they will not necessarily be created with their Kamea ID as an identifier. It is stored as a system metadata.

Topics

When a device is provisioned on Kamea, a set of topics and rights will be created for it. These topics will be used by different features. Here is the list of the topics for one device with their according rights:

Read only: - device/<interface name>/configs/desired - device/<interface name>/command/request/# Write only: - device/<interface name>/telemetries - device/<interface name>/configs/reported - device/<interface name>/command/response/+

More information on how these topics are used in their corresponding features are presented in the next sections.

Note that sending too many messages can have an impact on your infrastructure, it needs to be sized accordingly.

Telemetries

To send telemetries, device must send MQTT messages to the topic device/<interface name>/telemetries. The message format depends on the used codec.

Configurations

Configurations in MQTT use two different topics:

  • Desired configurations: device/<interface name>/configs/desired
  • Reported configurations: device/<interface name>/configs/reported

When sending configurations, desired or reported, in MQTT, all the configurations currently defined are sent. This means that not sending a configuration will remove it. For example, if this payload is sent as a desired configuration to the device:

{
    "temp": 12,
    "speed": 4
}

Kamea will consider that the configuration temp and the configuration speed has been set or updated on the device. But if afterward the devices sends:

{
    "temp": 12
}

Kamea will consider that the configuration speed has been removed from the device.

This behavior is the same for reported configurations.

All active configurations are sent every time, if some configurations are missing it means they have been removed.

One specificity of the desired configuration is that they are sent as "retain message". The retain message is the default message the device will receive when he subscribes, meaning that the last desired configurations sent will become the new default message. So if some desired configurations are set before the device subscribe, it will still receive the configurations on its first subscription.

Commands

Commands over MQTT use two topics: - To send the request to the device: device/<interface name>/command/request/# - To receive the result: device/<interface name>/command/response/+

The request topic includes the multi-level wildcard # meaning more topic segments can be added. And indeed, for command request, two topic levels need to be added dynamically. The first one is the name of the command, for example the command start. The second topic level will be a unique identifier that will be use internally by Kamea to handle concurrent commands. For the command start, the topic could be device/<interface name>/command/request/start/a61a6a71-2146-4eea-bc98-6d1058b7efc8. On the device side, there are two options: subscribe to device/<interface name>/command/request/# to handle all possible commands OR subscribe to each possible command independently. But keep in mind that subscribing to each command still requires to use the single-level wildcard + to receive the Kamea identifier.

It is mandatory that the device retrieves the Kamea identifier because the device needs to send it back when answering the command. It is simple to retrieve because the Kamea identifier is always the last topic segment in the command request. With the same logic than for the request, response topic must include the ID. The difference is that command name does not have to be specified again. The response topic to the previous example will be: device/<interface name>/command/response/a61a6a71-2146-4eea-bc98-6d1058b7efc8.

The API subscribes to device/<interface name>/command/response/+ and processes the payload sent to validate or not the command.

Connectivity

The Kamea API periodically sends a request to the broker to get the list of the currently connected devices. By default, this period is 10 seconds.

AMQP Consumers

AMQP Consumers are NodeJS workers dedicated to process the messages stored in the AMQP queues. These messages will be formatted before being forwarded to the bus to continue the ingestion cycle. The small size of these workers make them scalable quickly if traffic suddenly increases. At least one worker must be up at all time.

The purpose of AMQP Consumers is to extract the "ingestion information" that are needed by the rest of the ingestion cycle, from the message. These information includes: the payload of the message, the device information, the codec needed to read the payload and the timestamp at which the message has been received.

Technical details

This section provides technical details on the RabbitMQ setup. Note that reading it is optional since everything is automatically created and setup by the pipelines. It is here to help developers working on the system.

By default, TLS and mTLS communication are activated and the following official RabbitMQ plugins have been used (For more details on RabbitMQ plugins: https://www.rabbitmq.com/docs/plugins): - rabbitmq_mqtt: Enables the MQTT protocol (not activated by default) on the broker - rabbitmq_auth_mechanism_ssl: Enables a client to authenticate using only his TLS certificate - rabbitmq_auth_backend_http: Enables the use of an external backend to authenticate and authorize. This is only used for auto-provisioning. - rabbitmq_trust_store: Adds a trust store in the broker to keep all the whitelisted TLS certificates. This is used to validate the devices certificates when they authenticate against the broker.

To have all the plugins and features working you will need to set up different volumes and environment variables.

RabbitMQ environment

Environment variables: - RMQ_LOGIN: The login of the default admin account. - RMQ_PWD: The password of the default admin account. - RMQ_TLS_CERT_PATH: The path to where the TLS certificate of the MQTT broker is mounted. - RMQ_TLS_KEY_PATH: The path to where the private key of the MQTT broker is mounted. It is recommended to store the key in a secured storage. - RMQ_TLS_CA_CERT_PATH: The path to the broker Root CA certificate. Will be used for mTLS. - RMQ_AUTH_SECRET: The secret used to authenticate the broker against the Kamea API. - RMQ_TRUST_STORE_PATH: The path to the broker trust store. Will be used for mTLS. - KAMEA_API_URL: Kamea API URL.

Several Docker volumes must be mounted for: - The broker certificate. The mounting path has to be the same that the value of RMQ_TLS_CERT_PATH. - The broker private key. The mounting path has to be the same that the value of RMQ_TLS_KEY_PATH. - The broker Root CA certificate. The mounting path has to be the same that the value of RMQ_TLS_CA_CERT_PATH. - The trust store. At least one volume is needed. All whitelisted CA certificate have to be mounted in the mounting path defined in RMQ_TRUST_STORE_PATH. It's possible to do multiple mounts (one for each certificate for example) or only one including all the certificates.

Kamea API environment variables

To allow the Kamea API to communicate with RabbitMQ a set of environment variables is needed: - MQTT_CA_PATH: The path of RabbitMQ server certificate, used to establish TLS. - MQTT_API_CLIENT_ID: Kamea API session ID in the MQTT broker. Used to persist the Kamea API subscriptions. - RMQ_AUTH_SECRET: Secret used to authenticate the MQTT broker against the Kamea API. Must be the same value as the one provided in the variable RMQ_AUTH_SECRET of the RabbitMQ server. - MGMT_MQTT_INGESTION_URL: The MQTT URL of the RabbitMQ server. Example: mqtts://your-mqtt-domain.io:8883 - MQTT_BROKER_URL: The HTTP URL of the MQTT broker. Used to interact with the broker API to manage users and rights. Example: https://your-mqtt-domain.io:15671/api - RMQ_LOGIN: RabbitMQ user name - RMQ_PWD: RabbitMQ user password - USE_MQTT: Set to "true" to enable the MQTT module in the API. Any other value will disable the MQTT module.

The following variables are used in the API testing suite, but are not required for the API execution.

  • TEST_MQTT: Set to "true" to have the MQTT related tests run in the pipeline. The environment variable USE_MQTT also have to be set to "TRUE". Any other value will not run the tests in the pipeline.
  • MQTT_CERTIFICATES_PATH: Path to the certificates used for testing. Only used inside tests.

AMQP Consumers environment variables

The AMQP consumers need a set of environment variables: - MQTT_CA_PATH: The path of RabbitMQ server certificate, used to establish TLS. - MGMT_AMQP_INGESTION_URL: The AMQP URL of the RabbitMQ server. Only consists of its domain name. - RMQ_LOGIN: RabbitMQ user name. - RMQ_PWD: RabbitMQ user password. - BUS_CONNECTION_STRING: Connection string used to connect to the bus. - REPORTED_STATE_TOPIC_NAME: Name of the bus topic used to send the reported configurations. - RAW_DATA_TOPIC_NAME: Name of the bus topic used to send the telemetries.

Limitations

RabbitMQ not being a dedicated MQTT broker there are some limitations worth noticing. First, the Quality of Service 2 (QoS 2) is not available in RabbitMQ. But because the QoS 2 is adding a lot of extra communication it's usually not worth in a resource limited context. All QoS 2 subscriptions will be downgraded to QoS 1. Second, if a device subscribe to the same topic in QoS 0 and QoS 1 it might receive the messages in duplicate.

For more information, on the MQTT limits of RabbitMQ please refer to https://www.rabbitmq.com/docs/mqtt#limitations.