What Live Production Taught Me About Writing Hardware Drivers in Lua

Hardware Doesn't Read Your API Spec

When you write web software, you usually control both ends of the wire—or at least talk to a modern REST API that returns clean JSON and predictable HTTP status codes.

When you write Q-SYS Lua control drivers for high-output auditorium projectors and commercial displays in a 15,000+ weekly attendee venue, you quickly learn that hardware firmware lies. Endpoints change across firmware revisions, JSON payloads come back wrapped inside stray HTML tags, and sending a power command while a projector lamp is cooling down can lock up the control interface right before a live cue.

1. Extracting JSON from Noisy Firmware Responses

While building our Christie HD14K-M projector driver, standard JSON decoding kept failing intermittently. Inspecting the raw HTTP response revealed that the projector's embedded web server occasionally prepended HTML headers or trailing whitespace around the JSON body, and differed in where it exposed its RPC endpoint across firmware versions.

Instead of hardcoding a single path or trusting the response body, I built a candidate-endpoint probe (`/control`, `/cgi-bin/web.cgi`, `/web.cgi`, `/webapi`, `/rpc`) paired with a defensive decoder that strips out the outermost `{ ... }` block before parsing.

local function safeDecode(raw)
  if type(raw) ~= "string" or #raw == 0 then
    return nil
  end
  local ok, data = pcall(rapidjson.decode, raw)
  if ok and type(data) == "table" then
    return data
  end
  -- Firmware sometimes wraps JSON inside HTML or stray text; isolate outermost object
  local jsonChunk = raw:match("(%b{})")
  if jsonChunk then
    local okChunk, dataChunk = pcall(rapidjson.decode, jsonChunk)
    if okChunk and type(dataChunk) == "table" then
      return dataChunk
    end
  end
  return nil
end

2. Queued Auth Waiters & Client-Limit Backoff

Projector web servers have tiny connection pools. If a status poll and an operator button press both notice an expired session at the same millisecond and fire parallel login requests, the projector throws a 'maximum number of connected clients' error and drops both.

To prevent connection storms, I used a waiter queue (`loginWaiters`) so concurrent commands share a single in-flight authentication handshake, automatic single-retry replay when a session expires mid-command, and a 30-second cooldown timer (`loginBackoffUntil`) whenever the hardware reports a full client pool.

local function ensureAuthenticated(cb)
  if sessionCookie ~= nil and activeEndpoint ~= nil then
    cb(true)
    return
  end
  if nowSeconds() < loginBackoffUntil then
    cb(false)
    return
  end
  table.insert(loginWaiters, cb)
  if loginInFlight then
    return
  end

  loginInFlight = true
  tryLoginAt(1, function(ok)
    loginInFlight = false
    local waiters = loginWaiters
    loginWaiters = {}
    for _, waiter in ipairs(waiters) do
      waiter(ok)
    end
  end)
end

3. Reconciling Desired State Against Thermal Physics

A button in a UI is instantaneous; a dual-lamp auditorium projector is not. On our Sony Commons projectors, if a volunteer tapped 'Power Off' and immediately realized they needed the projector back on, sending a raw power toggle during the cooling cycle would simply be ignored by the hardware—leaving the Q-SYS UI out of sync with the room.

Rather than treating button presses as fire-and-forget events, I modeled power as a desired state (`desiredPowerState`) reconciled every 10 seconds against the projector's actual thermal state (`ON`, `STANDBY`, `STARTUP`, `COOLING`). If the projector is mid-transition, the script waits until the thermal cycle settles before issuing the toggle.

local function reconcilePowerRequest(state)
  if desiredPowerState == nil then
    return
  end

  if desiredPowerState and state == "ON" then
    desiredPowerState = nil
    return
  end

  if (not desiredPowerState) and state == "STANDBY" then
    desiredPowerState = nil
    return
  end

  -- Wait out physical lamp warmup/cooldown before issuing a toggle
  if state == "STARTUP" or state == "COOLING" then
    print("Projector in transitional state (" .. state .. "); waiting before retry.")
    return
  end

  if desiredPowerState and state == "STANDBY" then
    sendPowerToggle("power on request")
  elseif (not desiredPowerState) and state == "ON" then
    sendPowerToggle("power off request")
  end
end

4. Daisy-Chaining TCP Socket Fleets

For lobbies and common areas, we needed a single Q-SYS control block to drive arrays of Sony Bravia commercial displays over Sony Simple IP Control (TCP port 20060) without duplicating scripts for every TV.

By instantiating a Lua metatable object (`Display:new(ip)`) per IP in a comma-separated list, one Q-SYS control pin can broadcast power, volume, mute, and HDMI input commands (`*SCPOWR`, `*SCVOLU`, `*SCINPT`) across an entire wall of displays while aggregating their individual TCP socket states into a single health indicator.

Takeaway

Writing Lua drivers for live production hardware taught me habits I now bring to every full-stack TypeScript and cloud system I build: never trust external responses, deduplicate concurrent auth handshakes, respect physical and downstream rate limits, and reconcile desired state instead of firing blind imperative commands.