ESP32 sending data to SCADA and MES

The starting mistake: thinking publishing is enough

Making the ESP32 publish to an MQTT topic takes ten minutes. Almost every tutorial stops there, which is why so many integrations die in the first meeting with the SCADA operator.

SCADA does not want your JSON. It works with tags: a named point with a defined type, unit, range, quality and timestamp. The operator expects to open a screen and see PANEL_03.MOTOR_A.TEMPERATURE with a value, unit and an indicator that the number is trustworthy now.

Delivering JSON with field names invented by whoever wrote the firmware transfers the work of guessing what everything means to the other side. That is where the integration stops.

This article is about what comes after publish works.

Topic modeling: the contract between both sides

The topic hierarchy is the first decision and the hardest one to change after production.

A convention that works goes from general to specific, with one fixed level per dimension:

Three rules prevent rework.

Never put the value in the topic name. Use one topic per measurement and put the value in the payload. motor03/temperature/78 creates a new topic for every reading and destroys wildcard subscriptions.

Use stable identifiers, not descriptions. Motor 03 remains 03 when the model changes. motor-weg-w22-2024 becomes a lie at the first maintenance.

No accents, spaces or mixed capitalization. The broker accepts them, but every consumer tool later becomes a headache.

The gateway timestamp is not enough

This is what separates an integration that passes an audit from one that does not.

SCADA timestamps data when it arrives. If the network stalls for two minutes, all readings from that interval arrive together with the same timestamp, and the history shows a step that never existed in the process.

Worse, if you have an offline buffer, six hours of stored data arrive with the reconnection time. The graph becomes useless exactly during the period you needed to investigate.

The payload must carry the instant of the reading:

Fields are short because the payload repeats thousands of times per day and every byte multiplies. t is the reading instant in seconds since 1970, v is the value, q is quality and u is the unit. For fast variables, publish t in milliseconds and agree on that unit with whoever configures the SCADA.

This returns us to the problem from the buffer article: the ESP32 has no clock at power-up. Without synchronized NTP, it cannot timestamp. The honest choice is to publish quality as uncertain while the clock is not synchronized, never to invent a time.

Data quality: the field nobody fills

Every serious industrial protocol has quality, and it is the field most often ignored by people coming from the web side.

The reason is simple: zero is a valid value. If the sensor fails and the firmware publishes 0, SCADA shows zero degrees and the operator believes it. With quality, it shows zero marked as bad, which is the difference between a false alarm and a wrong decision.

The convention inherited from OPC uses three ranges:

Always publishing 192 is the same as having no field. The value of quality is in its ability to change.

Heartbeat: distinguish stopped from dead

A value that has not changed for ten minutes can mean two opposite things: the process is stable, or the gateway is dead.

Without a heartbeat, SCADA cannot tell. The wrong answer is expensive both ways: a false alarm when everything is fine, or silence when the plant has stopped.

The solution has two layers.

The Last Will from the MQTT article covers sudden death: the broker publishes offline for you when the connection drops without a goodbye. For this to work, the Last Will must be retained and published on the same status topic described below, so it overwrites the last "online" the broker kept.

A periodic status topic, published every thirty seconds with retention, covers the rest:

The reset value comes from the watchdog article and is extremely valuable on the SCADA side: when a gateway starts reporting TASK_WDT repeatedly, maintenance finds it before the equipment stops.

Configure an alarm in SCADA for stale updates: if the status t is older than two minutes, the gateway is gone. An alarm for a "missing topic" does not work here, because with retention the broker returns the last status even when the gateway is dead.

Retention: what SCADA sees after a restart

A normal MQTT message is delivered and forgotten. If SCADA restarts, it has no value until the next publication; for an ambient temperature sensor publishing every five minutes, that means five minutes of an empty screen.

The retained flag makes the broker store the last message for each topic and deliver it to new subscribers. SCADA starts and immediately has the last known value.

The practical rule is: retain state, not events.

Retain the process value because the last known value has meaning, and retain the gateway status, which is state by definition.

Do not retain a momentary alarm because replaying an old alarm confuses the operator, and do not retain a command because replaying it executes the action again.

That last case is genuinely dangerous. A retained command can reactivate equipment when SCADA reconnects.

TLS in the plant: what changes

Communication with SCADA almost always requires TLS, and on the ESP32 this has three practical consequences.

It costs RAM. The handshake consumes tens of KB. On a tight firmware, the symptom is a reboot from lack of memory, not a network error, so people waste hours looking for a connectivity problem.

It requires a clock. Certificate validation depends on the date: a valid certificate looks expired if the board thinks it is 1970. Synchronize NTP before attempting TLS.

It requires the root certificate. It is embedded in the firmware and expires. One day the connection stops working by itself, without anyone changing anything. Record the expiry date with the equipment documentation; it is the kind of detail nobody remembers three years later.

Sparkplug B: when it is worth it

If your SCADA supports it, learn about it. Sparkplug B standardizes exactly what this article has been describing: topic hierarchy, data types, quality, and device birth and death.

The real advantage is the birth certificate: when it connects, the device publishes its complete list of tags with type and unit. SCADA discovers what the gateway offers without anyone registering tags by hand.

The cost is the binary payload, which needs a library on both sides and is much heavier on the ESP32 than a short JSON.

The honest recommendation is: if your SCADA already speaks Sparkplug and you have dozens of gateways, it is worth it. For a few devices with well-modeled JSON, the complexity is not worth it.

Tutorial: from publish to a tag on screen

Step 1: settle the model before writing code

Sit with the SCADA operator before writing the first line. Define the topic hierarchy, measurement names and units.

Changing this later means reconfiguring SCADA and reprogramming every gateway. It is the cheapest step to do and the most expensive to skip.

Step 2: synchronize the clock first

Configure NTP and wait for synchronization before publishing or opening TLS. Until it is synchronized, publish uncertain quality and leave out the t field: never invent a time.

Step 3: publish one measurement

Start with one topic. Confirm in SCADA that the tag appears with the correct value, unit and time.

One measurement working end to end is worth more than twenty publishing incorrectly.

Step 4: add quality and test degradation

Disconnect the sensor on purpose and confirm that the value arrives with bad quality and that SCADA shows it instead of displaying zero as a valid reading.

If SCADA cannot distinguish it, the problem is in the tag configuration on that side.

Step 5: heartbeat and absence alarm

Publish the periodic status and configure the stale-update alarm in SCADA. Then turn off the gateway and time how long it takes for the alarm to fire.

This is the test that proves a silent failure will be noticed.

Step 6: enable TLS

Only after everything works in clear text. Embed the root certificate, confirm that the clock is synchronized and watch free heap after connecting.

If the board reboots here, it is memory, not the network.

Step 7: test reconnection with buffered data

Stop the broker for a few minutes while the gateway keeps reading. When it returns, confirm that data arrives with the reading time, not the reconnection time.

This validates the timestamp, and almost nobody tests it.

Common errors and solutions

SCADA does not recognize the data. Modeling. The JSON must be agreed on first; names invented in firmware do not become tags by themselves.

Every value has the same time. The timestamp is applied on arrival. Publish the reading instant in the payload.

The operator saw zero and believed it. The quality field is missing or SCADA is not configured to display it.

False alarm when the value does not change. There is no heartbeat. Configure an alarm for stale updates on the status topic, not for an unchanged value.

SCADA restarts and has no data. State topics are not retained.

A command ran again by itself. The command was published retained. Never retain commands.

TLS stopped working without anyone changing anything. The root certificate expired or the clock was not synchronized.

The board reboots when TLS starts. Handshake RAM. Use a board with PSRAM or reduce the firmware.