Skip to content

Data transfer

The data transferred in an MQTT message can be simple. For example, it might consist of a single sensor reading. However, in other cases it may make more sense to transfer several pieces of data at the same time. For example, a refrigerated truck might be part of a large fleet, all sending out periodic readings to allow users to monitor their conditions. Here, it would make sense for a message to contain all the data for the truck. This keeps related data together and removes the need to piece the data together from multiple message or to deal with each individual data item separately.

There are two commonly-used methods for encoding structure into a text-based message. One is XML which uses a syntax similar to HTML, and the other is JavaScrip Object Notation (JSON). Of these, JSON is more popular because it is simpler to manipulate with common programming languages.

JSON

The syntax of JSON is based on that used to define object in JavaScript. Virtually the same syntax is used to define dictionary objects in Python and associative arrays in Perl and PHP. The basic format is to define a series of key-value pairs where the key is essentially the name of a variable. A JSON object that represents the readings from a DHT22 sensor might look like this, for example:

1
2
3
4
{
    "temperature": 20.5,
    "humidity": 88
}

Notice that

  • The content of the JSON object is enclosed in braces
  • The keys are readable text strings
  • A value is associated with a key using a colon
  • Each key-value pair is separated from the next by a comma

A JSON value may be a nested object instead of just a number. This is illustrated in the example below which captures the fact that the two individual items are produced by a single sensor.

1
2
3
4
5
6
{
    "dht22": {
        "temperature": 20.5,
        "humidity": 88
    }
}

JSON also supports arrays through the use of square brackets. This is useful, for example, if you are reading values from several sensors at a time as illustrated below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
{
    "sensors": [
        "dht22": {
            "temperature": 20.5,
            "humidity": 88
        },
        "gps": {
            "latitude": 1.534,
            "longitude": 55.372
        }
    ]
}

Notice that

  • The main structure is still a set of key-value pairs
  • The value of sensors is an array of items
  • Each item in the array shown is a nested object
  • Array items are separated by commas

JSON is very flexible and the same data can be represented in different ways. For example, it may be more convenient for processing if the previous example made use of the array option for the sensor readings:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
{
    "sensors": [
        "dht22": {
            [
                {
                    "name": "temperature",
                     "value": 20.5
                 },
                 {
                    "name": "humidity",
                    "value": 88
                }
            ]
        },
        "gps": {
            [
                {
                    "name": "latitude",
                    "value": 1.534
                },
                {
                    "name": "longitude"
                    "value": 55.372
                }
            ]
        }
    ]
}

In this version of the structure, every individual sensor reading has the same format. When deciding on the most appropriate structure for your application, the most important consideration is how the data will be processed.

Validation

JSON structures can quickly become complex as illustrated above. Once you have decided on the structure you will use, it is useful to be able to check that the strings being produced by your code are correct. The first thing you need to check is whether a string is in fact valid JSON. With so many brackets for various kinds, it is easy to make a mistake. For simple syntax checking you can use an online resource such as JSON lint.

To exert greater control over the structure of your JSON messages, you can also define a JSON schema. This is essentially a set of rules that can be applied to any given string to see if it has the correct elements. Again, there are online tools that you can use for this including this one from Liquid Technologies.

To create a JSON schema requires some care. The structure of a schema uses syntax similar to that of a JSON document itself and is described here.

JSON documentation JSON documentation

JSON in Python JSON in Python

JSON with Particle devices JSON with Particle devices