Last week, we tested the integration of a new analytics module for one of the production sites — the system was supposed to collect data on chemical synthesis parameters in real time and send them to the server for building reports. Everything was going according to plan until... "this" happened.

The server crashed. The module is frozen. The operator on the shop floor started getting error after error. Everyone thought that the problem was with the network or overload — it turned out to be just one missing character in the JSON file. Yes, yes, a simple unshielded apostrophe.

This week, we introduced a new predictive analysis system for the composition of raw materials — the module sent data on the concentration of elements in JSON format. It would seem to be a standard story.

But one of the packages came with this JSON:

#json

{
  "compound": "NH3",
  "pressure": "null",
  "concentration": 0.24
}

Our system considered this as the actual value — zero pressure. And immediately transmitted the stop command to one of the key nodes. In just 7 minutes, the unit reacted as expected according to the safety protocol — emergency flow suspension, pressure relief, emergency ventilation.

💥 Production has stopped. 💥 Staff evacuated from the sector. One of the warehouses was automatically locked down. 💥 The materials accounting system has reset the balances for NH₃, KOH and HSO.

🔥 Closed the entire unit to the manual control system. 🔥 We raised the backup server on the last snapshot (thank our devops, who has OCD for backups). 🔥 We conducted a full audit of all incoming integrations — 14 points. Added a strict validation scheme for incoming JSON using the JSON Schema:

(
{
  "type": "object",
  "properties": {
    "pressure": {
      "type": "number",
      "minimum": 0.01
    }
  },
  "required": ["pressure"]
}

We have written a short technical specification for all contractors: if the pressure field contains null, "-", "n/a" or "0", we perceive this as a threat and block the entire package.

And most importantly , we have compiled our own internal document:

"7 scenarios when automation can become the enemy of production"

IT integration in the chemical industry is not just a "convenience". This is a responsibility for safety, for people, for raw materials, and for the reputation of the industry.

Industrial chemistry is accurate. But when it is described by lines of code, a 1-character tolerance can cause it to stop.

So now we have a different mindset: Every new parser, every data import, every exchange with a contractor, we start with a question:

"What happens if this value is false?""And if someone puts null? And if -1? And if ""?"

We restored everything in 2 hours. No one was injured. The products have not lost their quality. But this case will remain with us as a lesson in digital caution.

If you work in production, implement new modules, or develop industrial IoT, don't think that "JSON is just text." It is the language between man, machine and reality.

And we, the KAZCHEMY team, continue to make this language safe, understandable and sustainable.

 qazchem.kz

#KAZCHEMISTRY #digitalization of production #automation #jsonfail #data error #integration #IT industry #Real life history