TuyaBeGone: free firmware for a cheap LSC camera

A cloud-only indoor camera now runs a small Buildroot system. Video and audio go through the kernel drivers only: we traced the vendor’s closed libraries with an LD_PRELOAD shim and rewrote what they do. It streams over RTSP and ONVIF, with motion events from the ISP’s own statistics. Audio switched to AAC because UniFi Protect dropped G.711. It installs from an SD card in two stages, keeping the factory sensor calibration, and its first setup requires a press on the physical button. → TuyaBeGone: free firmware for a cheap LSC camera

A TwinCAT 3 machine moves from Windows to TC/BSD

A large-format 3D printer moved from the Windows TwinCAT runtime to TwinCAT/BSD on the same industrial PC. The traps: an engineering version that did not know the target, a corrupted drive startup command the older version had ignored, a drive moved to the wrong sync unit, Windows-only calls in the PLC code, and persistent positions left behind. → Moving a TwinCAT 3 machine from Windows to TC/BSD

How it connects

1. Both are about leaving a vendor’s platform (a cloud app, a Windows runtime) while keeping the hardware.
2. In both, the hardest part was the data that lives on the device: the camera’s sensor calibration, the machine’s saved positions.
3. In both, the old software quietly worked around something that the replacement had to discover: the vendor library read audio one period at a time (larger reads block forever), and the old TwinCAT version skipped a corrupted startup command.

Written by Jonathan (462engineering) and Claude (Claude Opus 5.5, Anthropic) — research, debugging and drafting done together in a Claude Code session.

1. The engineering tools must know the target

The engineering environment on the development PC was TwinCAT

1. It does not know the TwinCAT OS (x64) platform, so the project was built for the wrong target, and activation failed with an exception in the runtime. We had to move to 4026.

Installing 4026 with the package manager failed halfway: the system service and the ADS router were missing, so the target could not be found at all. Installing the base runtime package on its own, then rebooting, fixed it.

2. A newer version is stricter about old project data

After conversion, the 4026 environment crashed when generating init commands, compiling, or scanning the bus. A minidump pointed at one corrupted startup command in a servo drive’s init list, with a data length of 0xFFFFFFFF. 4024 had ignored it silently for years. Removing that one entry from the project file stopped the crashes.

3. Check the sync unit of every drive after conversion

The same servo drive then stayed in PREOP (AL 0x0037, invalid SYNC1 cycle time). The conversion had attached it to the 1 ms PLC task, while its axes run in the 2 ms NC task. Forcing it back to the NC sync unit, and putting the two cycle-time parameters back in its startup list, brought it to OP. Then the whole configuration activated.

4. Windows lurking in the PLC code

A search for Windows dependencies in the code found:

• absolute C:\ paths for the G-code files. Under TC/BSD the NCI folder is /usr/local/etc/TwinCAT/Mc/Nci, and any absolute path works since the runtime can read any folder. We moved the paths into constants;
• shutdown.exe started through NT_StartProcess on a power failure. We replaced it with NT_Shutdown, which goes through the TwinCAT system service (it still needs a test on the target). The old code also tested the status of a different function block than the one it called, so it never waited for the result, even on Windows;
• a call to lock the Windows session, which has no meaning on a PC without a desktop. We removed it.

Everything else (mail, file access, persistent data) worked unchanged. The operators still drop their G-code into a network share. It is now a Samba share on the BSD side, limited to private networks in the firewall. Recent Windows versions block guest access to SMB shares, which is the first thing to check if an operator PC refuses to connect.

5. Persistent data does not move with the OS

The persistent data (saved encoder positions and offsets for the vertical axes) stayed behind with the Windows runtime. The machine’s own safety check then refused to start: “axis/encoder offset too large”. That is the right reaction. Re-referencing the axes properly is a mechanical procedure, not a software one. On this machine the vertical axes are coupled (gantry) and must never move independently. For the day, the positions were realigned by hand. We also prepared a guarded reset step that only rewrites positions, with no motion; it is not in use yet.

6. Slower bus start-up

On TC/BSD the EtherCAT master takes about 24 s to bring the bus up (probably the long startup list of that servo drive). During that time the NC is already running, and the linear encoder axes latch an “invalid I/O data” error (0x4466) that stays after the bus is fine. A reset of those axes once the bus is in OP clears it. Doing it automatically in the PLC’s start-up sequence is still to do.

Takeaways

• Archive the project before touching anything. Then make a new archive at each step, named after what changed.
• Read the target’s logs before trying to activate anything.
• Treat a version upgrade of the engineering tools as part of the migration, not as a side step.
• After conversion, check sync units and startup lists drive by drive.
• grep the PLC code for C:\, .exe and NT_StartProcess.
• Persistent data is the part you lose. Plan the re-referencing with someone who knows the mechanics.

Written by Jonathan (462engineering) and Claude (Claude Opus 5.5, Anthropic) — research, debugging and drafting done together in a Claude Code session.

The hardware

• SoC: Anyka AK3918AV130 (ARM926EJ-S, ARMv5), 8 MB of SPI flash.
• Sensor: GalaxyCore GC20C3, 1080p, with IR LEDs and an IR-cut filter.
• Wi-Fi: AltoBeam, 2.4 GHz only.
• Microphone, small speaker, one blue LED, one reset button, micro-SD slot.

The budget is tight: the kernel partition holds 1.5 MB, the system image 5 MB.

Video and audio without the vendor libraries

The vendor SDK ships GPL kernel modules (ISP, hardware encoder, PCM, Wi-Fi) and a set of closed user-space libraries on top of them. We keep the kernel and its modules, and replace the libraries with our own small C library, libtbgav.

To learn what the closed libraries do, we wrote iotrace, an LD_PRELOAD shim. It logs every open, ioctl (with the argument before and after), mmap, read and write on /dev/*. Running the vendor demo under it gives the exact sequence each driver expects. libtbgav replays that sequence: V4L2 capture of two NV12 channels (1080p and 360p), hardware H.264 and JPEG encoding, microphone capture and speaker playback.

Two driver traps cost us some time. Both the capture and the playback paths must be served one period at a time: a read() or write() larger than one period (2048 bytes) blocks forever, because the driver stops waking the caller once its buffer is full. And every capture channel must be drained continuously, or the whole pipeline stalls.

Day/night switching (IR-cut filter and IR LEDs) was handled by a closed “software photosensor” in the vendor library. We read its principle in Ghidra and rewrote it from scratch, without copying any code, as a few pure functions over the ISP’s exposure and white-balance statistics. A small test bench runs the original and ours on the same inputs. A custom “indoor” setting that went back to day mode later turned out to make the camera oscillate between day and night at certain light levels, so we went back to the vendor thresholds.

Streaming: RTSP, ONVIF, UniFi Protect

The streaming daemon (avstream) uses live555 for RTSP and onvif-simple-server for ONVIF discovery, profiles and events.

• Audio: the first version sent G.711 µ-law. UniFi Protect subscribed to the audio track and dropped it right away. With AAC-LC (16 kHz mono, 48 kbit/s, encoded with the small Apache-licensed vo-aacenc), Protect records the sound. G.711 stays available as an option. Over 5 minutes, measured drift was −13 ms for video and 0 ms for audio. The lip-sync drift we saw came from Protect’s live web player, not from the camera.
• Motion events: the ISP already computes a 32 × 24 grid of activity values for its 3D noise reduction. We compare each block with its learned background level. If enough blocks stay above it for a few frames, that counts as motion, ending 10 s after the last detection and ignored for 5 s around each day/night switch. The result is published as ONVIF MotionAlarm and CellMotionDetector events. PullPoint subscriptions work. We have not yet checked them in Protect.
• Restart trap: live555 does not set SO_REUSEADDR on its server sockets. After a restart with Protect connected, port 554 stayed in TIME_WAIT for about a minute and the daemon exited. We now create the listening socket ourselves and hand it to live555, so the stream comes back in about 3 seconds. A small supervisor restarts the daemon with back-off if it ever exits.
• Extras: a JPEG snapshot URL, a mute switch that keeps the audio track but sends silence, a speaker API (chime, siren, or any audio file decoded by the browser and sent as WAV), and short, soft feedback sounds for the button and setup steps. The speaker amplifier starts late, so each sound begins with 150 ms of silence.

Installing from an SD card, in two stages

The factory U-Boot has a recovery feature: if a card with a specific config file is present at boot, it writes the kernel and the file systems from the card into flash. U-Boot itself and the config partition are never rewritten. An SD card can always recover the camera.

The catch is the sensor calibration. Each camera’s firmware carries the vendor’s ISP tuning file (colour, exposure, noise). We do not redistribute it, so the install takes it from the camera itself:

1. Bootstrap. U-Boot writes only a small kernel with an embedded busybox. After a press on the reset button (nothing happens without it), this kernel copies the calibration file into the config partition, dumps the rest of the old flash to the card for analysis, puts the stage-2 files in place and reboots. One more trap here: with an lz4-compressed initramfs, the kernel hung before the console came up. Gzip works.
2. Firmware. U-Boot writes the full system, which boots.

The downside is that the install overwrites the original kernel before it can be saved. Going back to the factory firmware needs a full flash dump made through the serial console before installing.

Later updates can also go through the web page. The upload is checked (magic numbers, sizes, version read by loop-mounting the image), and the camera then writes it from a static busybox copied to RAM. There is a single system partition and no automatic rollback: the SD card is the fallback.

Setup gated by the physical button

Without a configuration, the camera opens an open Wi-Fi access point with a captive portal. Anyone nearby could join it. So the setup form is only given to the device that was waiting alone when someone pressed the reset button. Other devices on the access point get a warning (“if this is not you, unplug the camera”), and if several were waiting, nobody gets the form. Saving the settings needs a second press within 2 minutes. The button counter survives a daemon restart, so a restart cannot pass for a press.

The production image contains no password. The root password is set at boot from the access password chosen during setup, and root is locked if there is none. The snapshot URL and the configuration pages require it. Telnet is off by default and always asks for a login.

Status

Video, audio, ONVIF, motion events (via PullPoint), the button-gated setup and the SD install are tested on the camera. The production hardening and web updates are built but not yet tested on hardware. The source is not published yet.

Written by Jonathan (462engineering) and Claude (Claude Opus 5.5, Anthropic) — research, debugging and drafting done together in a Claude Code session.

Network map: Wi-Fi, Zigbee, moved cables

Our inventory tool now draws Wi-Fi and Zigbee next to the cables, and it recognises a machine replugged on another switch port (one alert and a one-click fix instead of three alerts). The Wi-Fi collector has to deal with an old access point that ejects your admin session. The Zigbee collector talks to Home Assistant’s websocket from plain PHP. After the power cut, a storage-network switch came back with its old address: its settings had never been saved. → Network map: Wi-Fi, Zigbee, moved cables

The AC stayed off, and a loop with no hidden state

The air conditioner, used here as a fan, had stayed off since the afternoon. No automation had done it: a power cut had restarted Home Assistant, and the smart plug’s power-on state was set to Off. We then made it start on high CO2 or VOC. The first version kept a flag remembering who had switched it on. It was replaced by helpers recomputed every minute (one per pollutant, each with its own hysteresis), which the plug follows. → A stateless air-circulation loop in Home Assistant

Heating load from a power meter

Total power is also the heat produced indoors. From it we got the convector’s real power (980 W), the room’s heat loss (about 105 W/K) and its base load (about 620 W). This gives a heating-load percentage, measured and forecast from the weather service, relative to the convectors that are actually online (one is broken). The dashboard icons turn green, yellow, red or purple depending on whether the setpoint can be held. There is also a card-mod 4.2 gotcha with glance cards. → Heating load from a power meter, with a forecast

How it connects

1. Airing the room to clear CO2 and VOC means bringing in outdoor air, and in autumn that adds heating load.
2. Knowing the heating load is how a future fan, which will also open a ventilation damper, can decide whether airing the room is cheap or expensive right now.
3. Both use the same pattern: helpers recomputed from sensors on a schedule, and devices that follow them.
4. The power cut explains two surprises: the storage switch that forgot its settings, and the AC plug that came back off.
5. The Zigbee view of the network map showed the plug of the broken convector offline since late September. The power meter told the same story: in a week of data, the second convector never ran.

Written by Jonathan (462engineering) and Claude (Claude Opus 5.5, Anthropic) — research, debugging and drafting done together in a Claude Code session.

What we have

• One room, heated by electric convectors (coefficient of performance 1: every watt becomes heat). There are two of them, each controlled by a smart plug. One is currently broken: cat hair blocked its airflow and its thermal fuse blew.
• A whole-house power reading every 5 seconds. Since all the electricity used in the room ends up as heat inside, total power is a measure of the heat produced indoors, heating or not.
• An indoor temperature sensor, an outdoor one, and a weather integration with hourly and daily forecasts.

What the power meter tells us

The convector’s real power. Over 67 on/off switches of the plug, the median step in total power is 980 W (on: +981 W, off: −979 W). The power distribution is bimodal: 400–800 W with the heating off, 1400–1800 W with it on. So a 1100 W threshold gives the heating duty cycle directly from the power reading.

The second convector never runs. In a week of data there is no step of about 900 W outside the first convector’s switching, and nothing around 2400 W. That matched what we knew: it is broken, and its plug is unavailable.

A daily balance. Total power is base load + heating. Heat loss is roughly proportional to the indoor–outdoor difference:

So about 105 W/K, which agrees with an older analysis of the same meter over several years (2.77 kWh per day per °C, i.e. 115 W/K). Base load is about 620 W.

The model

The heating power needed to hold the setpoint is:

needed = 105 × (setpoint − outdoor) − 620   [W]
load   = needed / available capacity

With a 22.5 °C setpoint, the base load alone is enough above about 16–17 °C outside. One convector (980 W + 620 W of base load) can hold at most outdoor + 15 °C in steady state:

For short cold spells, the room’s thermal mass helps. We tried to estimate it from the data and got about 4.7 kWh/K, but with a weak fit (r = 0.27): with a thermostat, the indoor temperature hardly moves, so there is little to fit. A simulation of tomorrow’s forecast night (down to 5.4 °C) with one convector gives a drop of only 0.1–0.3 °C for a thermal mass between 2 and 10 kWh/K. That is why we use the daily mean temperature for “tomorrow”, not the coldest hour.

Back-tested on the daily mean outdoor temperatures, the model gives 37 / 13 / 22 % where 31 / 17 / 23 % were measured (with both convectors as the reference). Right order of magnitude, a few points off. It ignores sun gains, which makes it pessimistic on sunny days.

In Home Assistant

All of it is recomputed, nothing is accumulated:

• sensor.capacite_chauffage_en_ligne: 980 W × the number of convector plugs that are not unavailable. When the broken one is repaired and back on its plug, every percentage below rescales by itself.
• Measured load: (24 h average power − 620) / capacity.
• Load predicted now: the model with the current outdoor temperature.
• Tomorrow / next 7 days: an automation runs every hour, calls weather.get_forecasts (hourly and daily), and writes two input_number helpers. Tomorrow uses the mean of the hourly temperatures. The 7-day value is the most loaded day, using (min + max) / 2 for each day. We had to use an automation here, because a template helper created from the UI cannot call a service.
• A history_stats ratio helper for the convector’s duty cycle over 24 h.

Loads are not capped at 100 %. Above 100 %, the setpoint will not be held, and that is exactly what we want to see.

For the coming month there is no forecast, only climate. Over 2006–2025 (Open-Meteo archive), the period 13 October – 12 November needs a median of 34 % of both convectors, from 20 % (2022) to 47 % (2007). With only one convector that is 68 % in a median year, and above 85 % one year in four.

Coloured icons

On the dashboard, the two forecast values sit in a glance card, and their icon colour says what will happen:

“Setpoint − 1 °C” means a deficit of 105 W, so the purple threshold is 100 + 105 / capacity × 100: 110.7 % with one convector, 105.4 % with two.

card-mod gotcha: glance entities have their own shadow root

The first attempt put one card_mod style on the glance card, with .entity:nth-child(3) state-badge { … }. card-mod rendered the template correctly, but the icons stayed blue. In the browser, each glance entity turned out to be a div.entity with its own shadow root (with card-mod 4.2.1 loaded), so a selector from the card cannot reach the state-badge inside it. The fix is a card_mod on each entity entry:

- entity: input_number.charge_chauffage_prevue_7_jours
  name: Heating 7 d
  card_mod:
    style: |
      {% set cap = states('sensor.capacite_chauffage_en_ligne') | float(0) %}
      {% set v = states('input_number.charge_chauffage_prevue_7_jours') | float(-1) %}
      state-badge { --state-icon-color:
        {% if v < 0 or cap <= 0 %}var(--disabled-color)
        {% elif v <= 80 %}var(--green-color)
        {% elif v <= 100 %}var(--yellow-color)
        {% elif v <= 100 + 105 / cap * 100 %}var(--red-color)
        {% else %}var(--purple-color){% endif %}; }

We checked the result in a headless Chromium driven over the DevTools protocol, with the Home Assistant token placed in localStorage (hassTokens), then a screenshot.

What next?

The load is now a number we can forecast. That opens two opposite questions. We have not tested either of them yet.

Heating ahead of a cold spell

If the forecast says the convectors will not keep up for a few hours, can we store heat beforehand? The room’s thermal mass is the buffer. With our rough estimate of 4.7 kWh/K, a 200 W shortfall over a 12-hour night costs about 0.5 °C. Raising the setpoint by 0.5 °C in the afternoon before would roughly cancel it. The cost is higher losses while the room is warmer: about 50 W for 0.5 K, at 105 W/K.

What we need first:

• A real measure of thermal mass. Our fit is weak (r = 0.27). A deliberate test would do better: a step in the setpoint, or switching the heating off for a few hours on a mild night, and watching the curve.
• A rule that is still stateless. For example: “target = setpoint + f(load forecast for the next 12 h)”, recomputed every hour, rather than a “pre-heating in progress” flag.
• A comfort limit. It is a room people live in, not a storage heater. A few tenths of a degree is fine, but two degrees is not.

Lowering the target to save energy and carbon

The other way round: each degree less saves about 105 W whenever the heating is running, so about 2.5 kWh per day and 75 kWh per month of heating season. The Home Assistant setup already has an Electricity Maps sensor for the grid’s carbon intensity. At tonight’s 51 gCO2eq/kWh, one degree is worth about 4 kg CO2eq per month. That is small, but it does not cost anything to compute.

Questions we want to answer:

• When to lower it. Lowering the target when the load is high saves the most energy, but that is also when comfort suffers most. Lowering it when nobody is home, or at night, is the classic answer. Can we quantify it here?
• Shifting rather than lowering. With some thermal mass, the same amount of heat can be bought when the grid is cleanest: heat a little more when the carbon intensity is low, and coast when it is high. This is the same mechanism as pre-heating, driven by a different signal.
• Measuring the result. The model gives the expected consumption for a given outdoor temperature and setpoint. The real meter gives the actual one. The difference between the two, accumulated over a season, is the only honest measure of what a strategy saved.

Both ideas need finer control than on/off plugs. That is the other project on the list: variable power on the heating element (triac, whole mains cycles at zero crossing) and separate control of the convectors’ fans, with an interlock so that the element never runs without airflow. That is exactly what killed the second convector.

So, when to repair the second convector?

The practical rule we ended up with: order the replacement thermal fuse now, and repair when the 7-day icon turns yellow. Red only leaves a few days, which is enough for a repair but not for waiting for a part.

Written by Jonathan (462engineering) and Claude (Claude Opus 5.5, Anthropic) — research, debugging and drafting done together in a Claude Code session.

Why the AC was off

In the afternoon, the logbook has an 11-minute gap with no “Home Assistant stopped” event before it, then a restart. The plug came back as unavailable, then off, with no context: no automation and no user did it. The plug has a power_on_state select, set to Off. After a loss of mains, it stays off. A clean restart of Home Assistant the day before had left it on, because the plug itself never lost power.

The fix is one setting: power_on_state → LastState.

What the AC is used for

In this room, the air conditioner mostly works as a fan. Its own setpoint is above the heating setpoint, so in autumn it just moves air. The air-quality sensor is a Sensirion SEN66 (CO2, VOC index, NOx index, particulates).

Over the last 7 days, CO2 had a median of 496 ppm and a 99th percentile of 926 ppm. The VOC index had a median of 179 and a 99th percentile of

1. Note that the SEN66 VOC index is relative: it adapts to the recent baseline over hours, so a pollution that lasts slowly drifts back towards 100. It is good at catching a rise, not at measuring a level.

First version: a flag, rejected

The first automation was event-driven. It turned the plug on when CO2 or VOC crossed a threshold, set a flag saying “the automation did this”, and only turned the plug off later if the flag was set and the air was clean. It works, but the flag is state that can drift away from reality: a power cut, a restart, a manual switch, and the flag no longer matches the plug.

The verdict was short: “I hate stateful.”

Second version: helpers recomputed every minute

The current version is one automation that runs every minute and at startup. It does not react to events. It recomputes everything from the current sensor values:

Then the plug follows air_circulation OR the existing manual “cooling” boolean, so that cooling requested by hand is never switched off. A command is only sent when the plug’s state differs from the target, so the Zigbee network does not get a command every minute.

The only memory left is in the two per-pollutant helpers: between the two thresholds, each one keeps its previous value. That is what hysteresis is, and it cannot be avoided. But it is visible in the UI, and it cannot disagree with anything: if it is wrong, the next pass corrects it from the sensors.

The core of the automation (YAML view of what was created through the API):

triggers:
  - trigger: time_pattern
    minutes: "/1"
  - trigger: homeassistant
    event: start
actions:
  - variables:
      co2: "{{ states('sensor.aqm_co2') | float(0) }}"
      cov: "{{ states('sensor.aqm_voc_index') | float(0) }}"
  - action: >-
      input_boolean.turn_{{ 'on' if co2 > 900 or (is_state('input_boolean.air_circulation_co2', 'on') and co2 > 700) else 'off' }}
    target: { entity_id: input_boolean.air_circulation_co2 }
  - action: >-
      input_boolean.turn_{{ 'on' if cov > 350 or (is_state('input_boolean.air_circulation_cov', 'on') and cov > 150) else 'off' }}
    target: { entity_id: input_boolean.air_circulation_cov }
  - action: >-
      input_boolean.turn_{{ 'on' if is_state('input_boolean.air_circulation_co2', 'on') or is_state('input_boolean.air_circulation_cov', 'on') else 'off' }}
    target: { entity_id: input_boolean.air_circulation }
  - variables:
      wanted: "{{ 'on' if is_state('input_boolean.air_circulation', 'on') or is_state('input_boolean.loft_cool', 'on') else 'off' }}"
  - if:
      - condition: template
        value_template: "{{ states('switch.climatiseur') in ['on', 'off'] and states('switch.climatiseur') != wanted }}"
    then:
      - action: "switch.turn_{{ wanted }}"
        target: { entity_id: switch.climatiseur }
mode: single

Why a helper and not the device

air_circulation says “the air needs moving”. It does not say which device does it. Today it is the air conditioner. Later it will be a dedicated fan that also opens a ventilation damper, and only the last step of the automation will change. A helper is also an obvious place to add an on/off switch for the whole function later, read by the automation but never written by it.

A side effect worth knowing: switching the plug by hand no longer sticks. Within a minute, the loop puts it back where the helpers say it should be.

Written by Jonathan (462engineering) and Claude (Claude Opus 5.5, Anthropic) — research, debugging and drafting done together in a Claude Code session.

The tool

It is an in-house inventory: where the copies and backups of each dataset live, what hardware each machine has, and how everything is connected. Plain PHP and PostgreSQL, no framework, no authentication (it only lives on a trusted LAN). Collectors run from cron every 15 minutes. They log in to the switches’ web interfaces and read port states, VLANs and MAC tables.

One rule matters for everything below: a link is a physical cable, never a logical connection. An unknown port is stored as unknown and shown as “?”, never guessed.

The network map (the UI is in French; click for full size). For this post, machines and devices are renamed after their role (hv-1 and hv-2 are the hypervisors, “desk” is the workstation) and addresses are removed. Solid lines are cables, coloured by network as in the legend at the bottom; dotted teal lines are Wi-Fi associations (“association, pas un câble”: association, not a cable); dotted green lines join Zigbee devices to the coordinator. Line thickness is the link speed, and dotted cables are fibre or twinax.

Moved cables

Switch front panels, drawn from a small text template per model (port order, SFP cages, gaps). Each port shows its link state (green dot) and its network colour, and the machine at the other end of the cable is written next to it. “Interne : port 15” (“internal”): this switch is an appliance that combines a switch (14 RJ45 ports, 2 SFP cages and one internal port) and an Atom single-board computer in one box, wired together on the internal port 15. The SBC was recycled as the monitoring machine (Nagios, Cacti), and the switch is the LAN core. “relevé il y a 6 min” means “collected 6 minutes ago”.

Before, a machine replugged on another port of the same switch produced three alerts: “declared here but seen there”, “link down”, and “port with no cable”. Now the network page recognises the pattern and shows one alert, “X changed port: declared on 5, seen on 7”, with a button to correct the cable (and one to correct all moves at once).

A move is only proposed when all of these hold:

• the switch was actually collected;
• the machine’s MACs are seen on exactly one port of that switch;
• that port is not already one of the machine’s declared ports;
• the destination port does not keep a cable that is staying in place. If another cable on it is moving too, that is fine: two machines swapping ports are two moves.

The fix is recomputed when the button is pressed, not taken from the page, and the cable’s notes get a line like “port 5 → 7 on 2026-10-12”.

Wi-Fi

Two kinds of access point so far:

• A laptop that runs an access point: read over SSH, with a key that can only run iw station dump (a forced command on the laptop’s side).
• An old 3Com OfficeConnect AP, through its web interface. It has quirks worth writing down. It only speaks HTTP/1.0. It has one admin session, bound to the client IP. A new login always succeeds, but it ejects the current session; the ejected browser lands on duplicate.htm. So the collector always logs out explicitly, and the person in front of the admin page will occasionally be kicked out by it.

On the map, each Wi-Fi network is a dotted rail, joined to whatever carries the access point: the machine itself, or the cable to the machine the AP is plugged into. Wi-Fi clients seen behind a machine are listed under it, the same way virtual machines are listed under their hypervisor.

Zigbee

The Zigbee coordinator is a USB dongle on the Home Assistant box, run by ZHA. The collector asks Home Assistant’s websocket API for zha/devices (read only). PHP has no websocket client in its standard library, so the collector includes a minimal one: about 70 lines for the handshake, masked text frames out and unmasked frames in. It reuses the same retry settings as the switch collectors. Home Assistant bans an IP after repeated authentication failures, so the collector has its own named token, and a refused token stops the run instead of being retried.

The Zigbee page draws a star: each spoke’s thickness is the link quality (LQI), and offline devices are red and dashed.

The Zigbee view, devices renamed for this post. The legend reads: mains-powered router (filled), battery device (hollow), offline (red, dashed), spoke thickness = LQI.

On the first run, two devices were offline. One was a plug named “Chauffage” (heating, “heater plug” in the picture), offline since 26 September. That is the plug of the convector whose thermal fuse blew, which the heating analysis later found again from the power meter.

Drawing radio on the map

Wi-Fi and Zigbee are “radio groups”, drawn in three passes: rails first, then the lines (on a halo where they cross a rail, so the crossing is readable), then the boxes. Each technology gets its own row, Wi-Fi then Zigbee, with its rails just above it. Each group sits under the thing it is attached to, and the columns where lines arrive are reserved so that nothing is drawn on top of them. The Zigbee coordinator is part of the Home Assistant box, so it has no box of its own: the rail starts from the box. The goal stays the same as for the cables: one big page, readable, with nothing crossing that does not have to.

There is also a dark mode now: automatic, dark or light, stored per browser.

A switch that forgot its settings

After the afternoon’s power cut, one of the 2.5 GbE switches on the storage network came back with its old IP address and default password. The settings changed ten days earlier had never been saved: on this model, a change applies immediately but is lost on reboot until you press Save under Device Management. Restoring the backup only reapplied the password while running, so we set the IP again by hand, then saved. A second, deliberate power cut confirmed it: the switch came back with the right settings. Its config export now differs from the reference by a single byte. The difference is stable and has no visible effect.

The same power cut is why the air conditioner stayed off: see the air-circulation post.

Written by Jonathan (462engineering) and Claude (Claude Opus 5.5, Anthropic) — research, debugging and drafting done together in a Claude Code session.

Bring-up: Debian 13, RDNA4, Valve Index

The GPU box is a Debian 13 machine with a Radeon AI PRO R9700 (RDNA4). RDNA4 needs the kernel, Mesa and firmware from trixie-backports. evdi needs version 1.15.1 to build on kernel 7.1. We also hit a few boot-time problems (a missing display manager unit, an initramfs that picked up .link backups) and a few Steam and Index details. → Small notes: Debian 13 + RDNA4 + Index

Playing from the desk: sRemFB

To use the GPU box from the workstation, sRemFB creates an evdi virtual display on it and streams uncompressed or LZ4 damage rectangles to a new windowed viewer. Keyboard, mouse and gamepad go back over uinput, and audio over UDP. Key to pixel: 8.7 ms median on 2.5 GbE. → Playing on the GPU box from the desk, without a video codec

SteamVR judder, and moving to Monado

In SteamVR, menus and overlays judder. On Linux, SteamVR runs without async reprojection. Enabling it, or asking for 120 Hz, ends with flip_done timed out and a killed compositor. Monado with SteamVR’s own Lighthouse driver, xrizer and WayVR runs smoothly at 90, 120 and 144 Hz on the same hardware. → SteamVR on Linux: overlay/UI judder on RDNA4, and why Monado fixes it

mutter never forgets an unplugged GPU

mutter never removes a GPU that has been unplugged, and Xwayland keeps its DRM fd. Every removed evdi card leaves a stale DRM lease device. In the end, SteamVR and Monado both stall on them. → mutter never forgets an unplugged GPU (and what it does to VR)

How the problems connect

1. sRemFB creates evdi cards on the GPU box.
2. With evdi cards present, SteamVR’s DRM lease breaks (dc_stream_state is NULL, flip_done timed out).
3. The first fix was to remove the evdi cards before starting VR.
4. But mutter never forgets a removed GPU. Each removed card leaves a stale wp_drm_lease_device_v1 global, and Xwayland keeps its fd.
5. The stale globals block VR: SteamVR loops after LeaseDevice_Done, and Monado waits forever for done events, or picks a stale device that advertises the Index’s connector.
6. With a patched Monado (bounded wait, no silent fallback, optional DRM node pinning), Monado ignores the stale devices. It also runs fine alongside evdi, which SteamVR does not.
7. So there is no longer any reason to remove evdi cards: sremfb-server now runs permanently. No cards are removed, so no stale devices are created.

A mutter patch for the root cause exists but has not been tested. It is published as an RFC in the mutter post.

Written by Jonathan (462engineering) and Claude (Claude Opus 5.5, Anthropic) — research, debugging and drafting done together in a Claude Code session.

The machine

Pitfalls

• Use trixie-backports for RDNA4. Take the kernel, Mesa and firmware-amd-graphics from backports.
• No display-manager.service. gdm was installed but had never been enabled, so display-manager.service did not exist.
• Never keep .link backups under /etc/systemd/network. The initramfs hook /usr/share/initramfs-tools/hooks/udev copies *.link files from /etc/systemd/network recursively, and puts them all in one directory in the initramfs. Backups in a subfolder therefore become active at boot. This happened to us: after installing the new kernel, a NIC got the wrong name at boot. Keep backups somewhere else.
• evdi 1.15.1 is needed for kernel 7.1. Debian’s evdi 1.14.8 does not build against 7.1: API errors about drm_driver.date, struct_mutex and fb_debug_enter. We repackaged 1.15.1.
• Proton for all games. In Steam’s config.vdf, a CompatToolMapping entry for app id "0" sets the default compatibility tool.
• Installing a Steam game without a screen. With Steam closed, put an appmanifest_<appid>.acf with "StateFlags" "1026" in the library’s steamapps/ directory. At its next start, Steam downloads the game.
• The Index camera is a plain UVC device. It is a side-by-side stereo camera: 1920×960 total, 960×960 per eye. It offers MJPEG or YUYV at 15–54 fps. It can be read with V4L2 while SteamVR is running, as long as SteamVR itself is not streaming from it.
• setcap does not survive updates. CAP_SYS_NICE on SteamVR’s vrcompositor-launcher is lost at every SteamVR update, and the same goes for monado-service after every cmake --install.
• Index audio goes through DisplayPort. The sink only exists while the headset is on. Use wpctl set-default once, and WirePlumber remembers it (details in the SteamVR/Monado post).

Written by Jonathan (462engineering) and Claude (Claude Opus 5.5, Anthropic) — research, debugging and drafting done together in a Claude Code session.

Setup

sRemFB is our “network monitor” project. On the server, it creates a virtual display using EVDI, the DisplayLink kernel module. GNOME treats it as a real monitor that gets plugged in. The server then sends the damaged rectangles of that display over TCP, raw or LZ4-compressed. Here the server is the GPU box (Radeon R9700, GNOME 48 on Wayland) and the client is the workstation at the desk.

The desktop viewer never uses H.264 or any other video codec. We chose this because the goal was the lowest possible latency, and a 2.5 GbE LAN has enough bandwidth for most content.

The features below are on a development branch and will be in the next release.

What’s new

sremfb-view, a windowed client

• SDL3, native Wayland.
• VSync off by default. The viewer always presents the most recent frame; there is no frame queue at any point.
• It plugs a virtual monitor into the remote GNOME session exactly like a hardware client does. GNOME remembers its position like any other screen.

Keyboard, mouse and gamepad over uinput

• On the server, input is replayed through uinput devices. This is off by default and enabled with SREMFB_INPUT=1.
• Mouse: absolute by default (like QEMU’s tablet device), or captured relative mode for games.
• Gamepad: appears as an Xbox 360 pad (045e:028e), which Steam and SDL recognise.
• On disconnect (or focus loss, or a lost link), everything is released on the remote side, so no key or button stays pressed.

Audio

• The server creates a virtual PipeWire sink.
• Audio is sent as raw PCM S16LE, 48 kHz over UDP, in packets of 128 frames (2.7 ms).
• The client aims for a 10 ms buffer. When it gets ahead, it drops samples instead of letting latency drift.

Display layout re-applied as temporary

mutter’s monitors.xml is keyed by connector name, and the connector name changes each time the evdi card is recreated. The server therefore re-applies the remembered layout itself. It uses the temporary method: a persistent ApplyMonitorsConfig makes GNOME show “Keep these display settings?”, and GNOME reverts after 20 s if nobody confirms.

A DROP_MASTER fix

sremfb-server keeps a lock fd on each evdi card. When mutter released a card, this lock fd could become DRM master of that card. After that, mutter’s TakeDevice failed with EBUSY, permanently. The server now drops master on that fd.

Living with mutter’s GPU hotplug bug

mutter never forgets a removed GPU. That affects evdi cards directly, and has consequences for VR too; see mutter never forgets an unplugged GPU. The server creates its cards one at a time. It keeps a card only if gnome-shell opens it within 1.5 s, and never reuses a number that mutter refused. In practice, we also leave the server running permanently.

Measurements

All timings are taken on the client’s clock alone: input sent, then the matching pixel received and decoded at the client. They do not include the client’s own compositor, scanout, or display latency.

Under full-screen noise, the time is dominated by transmission. One raw frame is 8.3 MB, which takes about 27 ms at 2.5 Gbit/s. Normal desktop and game content changes much less of the screen per frame.

In games up to 60 fps, the user reported it “runs as if it were local”.

Not done yet

For very animated content, one idea is per-tile JPEG: each tile encoded as an independent image, with no inter-frame state. This would cut bandwidth without adding the buffering a video codec needs. It is not implemented.

Written by Jonathan (462engineering) and Claude (Claude Opus 5.5, Anthropic) — research, debugging and drafting done together in a Claude Code session.