Skip to main content

When the HMI Goes Dark: A Loss-of-View Tabletop for Small Food Processors

The operator can still hear the line running, but the HMI screen is blank. A second station has stopped updating. The historian shows no new data. Nobody yet knows whether the process has stopped, the displays have failed, or the values on another screen can be trusted.

This is a loss-of-view exercise. It tests how a small food or beverage processor would manage the process when normal digital visibility is missing or questionable. NIST lists blocked or delayed OT information, including loss of view and loss of control, among the incidents an OT system may face.[1]

Run the exercise at a table with a fictional scenario. Do not disconnect equipment, suppress alarms, change controller logic, test a shutdown, or ask an operator to run a live process without its normal displays. The plant’s operations, engineering, food-safety or quality, and environmental health and safety authorities decide what is safe. An exercise facilitator does not. International guidance hosted by CISA frames OT design and management around safety, security, and continuity of critical services.[3]

Put operating authority on the first page

Start with one process area and a simple statement: at 10:12 a.m., the primary HMI stopped updating; the backup display shows values that may be several minutes old; the line appears to be running; and the cause is unknown.

Before discussing malware, ask who has authority over the physical process. Name the shift lead, process owner, controls engineer, maintenance contact, IT incident lead, and food-safety or quality decision maker. Add environmental health and safety, facilities, the equipment vendor, and senior management where the process calls for them.

NIST says OT tends to put availability and safety ahead of confidentiality.[1] It also recommends input from operations, engineering, IT, system-support vendors, management, legal, and safety when an incident-response plan is prepared.[1] The exact group will vary by plant, but the exercise should make one boundary clear: IT can investigate systems and networks; it does not unilaterally decide that a physical process is safe to continue or restart.

Give each role a name and two backups. Record who can order a controlled stop, approve isolation of an HMI or network segment, call the vendor, release product, and authorize a return to normal operation. If the answer is “it depends,” write down what it depends on and who resolves the conflict.

Separate what is known from what is displayed

Hand the group a one-page process sheet with fictional details. Include the affected screens, controllers, alarms, historian, quality records, upstream and downstream dependencies, and any local instruments operators normally use. Do not use real addresses, passwords, controller programs, recipes, or network diagrams in the exercise packet.

Create three columns on a whiteboard:

  • known through an approved independent observation;
  • displayed but not yet trusted;
  • unavailable or unknown.

The distinction matters. A frozen screen can look normal. A working screen can still be showing stale or incomplete information. Ask the process owner which observations are approved for this equipment and who is qualified to make them. The answer may involve local indicators, product checks, equipment state, or another control station, but the tabletop should not invent an alternate operating method. If the approved procedure does not cover operation without the HMI, that is a finding.

Now add a second inject: an operator reports that a temperature value on a local approved indicator does not match the last HMI value. Ask what changes. Which process, safety, quality, or product-hold procedure applies? Who records the time and affected production window? Who is allowed to decide whether the line can continue in a reduced mode?

Make containment a joint decision

The facilitator then tells the team that IT sees unusual traffic involving the HMI. The tempting answer is “pull the cable.” Do not reward speed without process context.

NIST warns that physically isolating a system may hurt operational performance or safety and recommends a focused risk assessment to choose the response.[1] Ask the controls engineer and process owner what the HMI supports, whether another station depends on it, and what could happen if communication disappears completely. Ask IT what evidence might be lost, what traffic can be restricted without touching control communications, and whether a known-safe monitoring point is available.

The team should produce a short decision record:

  1. condition observed and time;
  2. process and people potentially affected;
  3. trusted and untrusted information;
  4. proposed containment action;
  5. operational and safety review;
  6. approver, operator, and verification method;
  7. fallback if the action makes conditions worse.

This is not a universal instruction to keep a suspected system connected. It is a way to prevent a cyber response from becoming an unauthorized process change.

Test the handoff to recovery

Move the clock forward. The affected HMI is isolated under the plant’s approved procedure, and the team has a known-good replacement image available in the fictional scenario. Recovery still needs an order.

NIST recommends measuring response plans against the service being provided, not only the compromised system.[1] Ask what service must return first: operator visibility, alarming, control, historian collection, recipe management, quality records, reporting, or something else. Let operations and engineering set the sequence. IT and the vendor can explain technical dependencies, but a successful boot screen is not proof that the process is ready.

For each recovery step, record the approved source for software and configuration, the expected version, who verifies integrity, which connections remain blocked, and the test that shows the function is behaving as intended. Keep product disposition and process release separate from technical restoration. The quality or food-safety owner may need evidence that the technical team does not own.

A live recovery test may be unsafe or impractical. NIST notes that OT recovery testing can be constrained by operations and safety; it suggests bench or offline testing where possible and says backup integrity should at least be verified when a full test cannot be run.[1] For the tabletop, identify one safe offline check the plant could perform later and the maintenance window, test equipment, vendor support, and approval it would require.

End with a restart gate, not a reboot

The final question is not “Is the HMI back?” It is “Who has enough trustworthy evidence to resume the intended operating state?”

Build a restart gate with plant-specific owners. It may require confirmation that the approved HMI image and configuration were used, communication paths match the approved design, alarms and time synchronization work, operator views agree with approved observations, temporary access is removed, evidence is preserved, affected product has an assigned disposition, and operations has accepted the handback.

CISA describes an incident-response plan as a leadership-approved document that defines roles and guides work before, during, and after a suspected or confirmed incident.[2] CISA also advises keeping printed plans and contacts because email, chat, or document storage may be unavailable during an incident.[2] Give the shift team a controlled paper copy of the loss-of-view call tree and decision sheet, with an owner and review date.

A 60-minute tabletop should leave a short work list, not a score. Assign each gap to an owner: unclear shutdown authority, missing vendor contact, unavailable drawings, unverified backup, no offline test method, uncertain product-hold trigger, or a restart approval that exists only in someone’s memory. CISA recommends tabletop exercises as a way to rehearse incident roles.[2] Fix a few concrete gaps, then run the same scenario on another shift.

For practical guidance on cyber risk and safer technology use, subscribe to Cyber & AI Insights.

If your plant wants help facilitating an OT incident-readiness tabletop without touching production, schedule a conversation with Your Data Guardian.

Sources

  1. NIST SP 800-82 Rev. 3: Guide to Operational Technology Security
  2. CISA: Incident Response Plan Basics
  3. CISA: Principles of Operational Technology Cyber Security