Files
andy b52b780f2b fix(da07): queue outbound commands one-in-flight - the station drops burst writes
Hardware-found: toggling two device rows' Active and refreshing showed only the
second toggle landed. The 2026-06-12 capture explains it - 16 burst channel
writes drew only 2 Z1 ACKs; the station processes one inbound frame at a time
and silently drops the rest. The legacy never burst: MakeCommand only queued,
and each inbound Z popped exactly one command (Main.frm SendCommand).

The controller now queues commands and keeps one in flight: Z1 confirms and
advances; Z0 retransmits; a Z2 idle while unconfirmed means the frame was
dropped, so it retransmits (all queued commands are idempotent - this improves
on the legacy, which lost silent drops), capped at 3 transmissions then dropped
with errorOccurred. Link frames (data-frame ACKs, idles) bypass the queue. The
queue is cleared on stop() so a closed port cannot wedge it; a refresh queued
behind writes re-arms the load-settle timer at transmit time.

pendingWritesChanged(depth) drives a new footer WORKING dot - the modern
version of the legacy "Working n" status caption, per user request: lit while
settings changes await the station's acknowledgement.

The simulator now Z1-ACKs every command frame like the real station (applying
the command BEFORE acking, so synchronous pumping cannot reorder same-field
writes) - its perfect burst handling is exactly why this bug was sim-invisible.
CIM_DA07_CAPTURE now also records outbound frames ("> "-prefixed) so the next
hardware session sees both sides of the handshake.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 10:44:59 -04:00

16 KiB
Raw Permalink Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What this is

A modern Python 3 + PySide6 rebuild of a legacy VB6 instrument tool, the DA-12 Monitoring Station Service Tool — a serial/RS-232 utility that configures and monitors CIMTechniques DA-12 sensor stations during production and in the field. The DA-12 VB6 baseline (DA12-Service.exe, Main.frm, Calib.frm, Support.bas, DA-Service.vbp, etc.) now lives at cim_suite/modules/da12/legacy/ — read it to confirm protocol behavior, never modify it.

Bigger direction: DA-12 is now module #1 of the unified CIMTechniques Service Suite — a monorepo under import root cim_suite/ with a shared core package and a shell launcher, so customers install/sign/update one app. See docs/SUITE-ARCHITECTURE.md.

Module #2 — DA-07 ("eLink"): rebuilt 2026-06-02 at cim_suite/modules/da07/ following the same five-layer pattern (--module da07 --simulate). Its protocol is a different dialect of the same CIMTechniques family, so it has its own da07/protocol/ rather than reusing DA-12's: frames are ~<payload><checksum>\r (checksum = sum of ASCII incl. ~, mod 256), multi-byte numbers are little-endian, and channel values are IEEE-754 floats (vs DA-12's big-endian fixed-point). It also has an extra hierarchy level — Station → Devices (pods) → Channels. Its I/O model is a polled handshake, not a stream (DA-12 streams {...} frames unsolicited): the station sends a refresh reply one frame per ACK and interleaves Z2 idle polls into it. The controller must both ACK every inbound data frame with Z1 (da07/domain/controller.py::_process) and answer the station's Z2 idles with Z2 (_handle_poll; Z0=NAK→resend) — miss either and the load stalls partway. This was hardware-verified 2026-06-03; the simulator models the stream via handshake=True. (A real station also sends an unmodelled M frame the controller ACKs past.) The DA-07 also never echoes settings writes, so every controller set_* applies the value to the local model optimistically (hardware-found 2026-06-12; without this, periodic ~H/~G frames rebuild the tabs from the stale model and revert every edit) — and outbound commands are queued one-in-flight, paced by the station's Z replies (hardware-found same day: the station drops burst writes — 2 of 16 ACKed; the footer "Working" dot shows the queue draining, mirroring the legacy MakeCommand/SendCommand queue and its "Working n" caption). Bugs found against real DA-07 hardware are logged in docs/DA07-FIELD-NOTES.md — read it before debugging this module. The DA-07 VB6 baseline is at cim_suite/modules/da07/legacy/ (read-only). Spec: docs/superpowers/specs/2026-06-02-da07-service-tool-rebuild-design.md. Deferred production features and hardware-verification items are tracked in docs/BACKLOG.md (BL-E1..E3) and docs/HARDWARE-VERIFICATION.md (DA-07 section).

Module #3 — IOModbus: rebuilt 2026-06-03 at cim_suite/modules/iomodbus/ following the same five-layer pattern (--module iomodbus --simulate). It is the odd one out: a standard, config-driven Modbus RTU master, not a CIMTechniques proprietary stream. It has no fixed protocol of its own — a bundled text catalog (resources/IOModbus.txt, parsed by protocol/regdef.py) describes every supported device as Modbus register maps, and everything shown/written is derived from it. Its own iomodbus/protocol/: Modbus CRC-16 (crc.py), the 11 register data-type codecs (codecs.py — big-endian regs, signed-16 bipolar, the device-specific -19999 offset, normal/reversed floats), an 8-byte request builder and a transaction-scoped ResponseAssembler (pdu.py). The key structural difference from DA-12/DA-07: the link is half-duplex request/response, so the controller (domain/controller.py) drives a timer-stepped scan + poll state machine with an _awaiting interlock (send one request → wait for its response or a timeout → advance; the response handler never re-sends, so the synchronous simulator can't recurse). _tick is a plain method so tests drive it deterministically. The simulator (transport/simulator.py) is an in-memory Modbus slave bank seeded from the catalog (pull-based: write(request)→reply; tick() only mutates state). The IOModbus VB6 baseline is at cim_suite/modules/iomodbus/legacy/ (read-only; the IOBuilder/ companion edits the catalog). Spec: docs/superpowers/specs/2026-06-03-iomodbus-service-tool-rebuild-design.md. Deferred features (LAN/Modbus-TCP, Debug/Test/RegInfo windows, IOBuilder) and HW-verification items are tracked in docs/BACKLOG.md (BL-I1..I3) and docs/HARDWARE-VERIFICATION.md (IOModbus section).

Cross-module — Device settings repository: a suite-wide SQLite store at %LOCALAPPDATA%\CIMTechniques\Service Suite\devices.db (separate devices.sim.db under --simulate) that change-logs DA-12/DA-07 configuration settings keyed by station MAC (first sighting + on change; live values excluded). Pure store at cim_suite/core/repository/store.py; a Qt RepositoryRecorder debounces controller signals and writes only changed config fields; per-module domain/repo_snapshot.py adapters flatten models into namespaced (key, label, value, display) tuples; a reusable SettingHistoryDialog shows per-row + whole-device history. IOModbus is out of scope. Right-click a history row → "Set to this value" reverts a setting to any recorded value (module-agnostic dialog + per-module setting_writer; read-only fields disabled). Spec: docs/superpowers/specs/2026-06-06-device-settings-repository-design.md.

All three rebuilds are functionally complete against their simulators. The outstanding milestone is verification against real DA-12, DA-07, and Modbus hardware (see Hardware verification below).

Commands

# Setup (Windows, Python 3.11+)
py -m venv .venv
.venv\Scripts\python -m pip install -e ".[dev]"

# Run against the in-memory simulator (no hardware) — the normal dev path
.venv\Scripts\python -m cim_suite.shell.app --module da12 --simulate
# Against real hardware / a saved port
.venv\Scripts\python -m cim_suite.shell.app --module da12 --port COM3
# Open the suite launcher landing page (module card grid)
.venv\Scripts\python -m cim_suite.shell.app

# Tests & lint
.venv\Scripts\python -m pytest -q
.venv\Scripts\python -m pytest tests/core/test_codecs.py -q          # one file
.venv\Scripts\python -m pytest tests/core/test_codecs.py::test_name  # one test
.venv\Scripts\python -m ruff check cim_suite tests

# Build BOTH distribution artifacts (per-user installer + portable zip) in one shot.
# Version is single-sourced from cim_suite.__version__. The installer needs Inno Setup 6
# (without it, build.ps1 warns and produces the portable zip only). Code signing is gated
# on $env:CIM_SIGN_CERT (see docs/RELEASE-PACKAGING.md); skipped if unset.
powershell -ExecutionPolicy Bypass -File packaging\build.ps1

# Verify the frozen exe actually booted (GUI app — use Start-Process -Wait for the exit code)
$env:SUITE_SELFTEST="1"; $p = Start-Process .\packaging\dist\CIM-Service-Suite\CIM-Service-Suite.exe -ArgumentList "--module","da12","--simulate" -Wait -PassThru; echo "exit=$($p.ExitCode)"; Remove-Item Env:\SUITE_SELFTEST

The whole test suite runs headless with no hardware (SimulatedStation speaks the real wire protocol; UI tests use pytest-qt offscreen). Keep pytest green and ruff clean — both are part of "done" here.

Architecture

Five layers, strictly ordered. The protocol core is pure (no I/O, no Qt) and the device is swappable — this is the central discipline, not an accident.

Layer Package Role
Shared core cim_suite/core protocol/codecs.py (hex/scale primitives + FieldReader); transport/base.py + serial_transport.py (BaseTransport interface + pyserial reader thread); ui/theme/ (design system tokens → fonts → stylesheet → apply_theme); ui/table_tab.py; config.py (generic, app-name-keyed); module.py (the Module Protocol + ComingSoonModule). Extracted conservatively — only obviously-shared pieces (rule of three).
DA-12 Protocol (pure) cim_suite/modules/da12/protocol framing (extract {...} bodies), decoder (body → messages dataclass), encoder (build outbound command strings). No I/O. Fully unit-tested.
DA-12 Transport cim_suite/modules/da12/transport SimulatedStation (in-memory fake DA-12, same BaseTransport interface). SerialTransport now lives in cim_suite/core/transport.
DA-12 Domain cim_suite/modules/da12/domain models (in-memory SensorTable/LimitsTable/StatsTable/StationSettings, plain Python); calibration, logger; StationController (the DA-12 hub — see below).
DA-12 UI cim_suite/modules/da12/ui MainWindow + 5 tabs (Station, Sensors, Alarm Limits, Statistics, Calibration), dialogs. Thin views: render models, call controller methods on edit.
Shell cim_suite/shell app.py (entry point); registry.py (static module list); launcher.py (card landing page — grid of module cards); window.py (SuiteWindow hosting the launcher or the active module, with a "☰ Suite" back action).

StationController is the hub (domain/controller.py): it owns the models and the transport, decodes inbound frames into model updates, emits high-level Qt signals (sensorsChanged, limitsChanged, etc.) the UI subscribes to, and exposes one method per outbound command. Data flow:

transport (thread) → controller._rawReceived signal → _process → framer → decode → models → *Changed signals → UI
UI edit → controller.set_*  → encoder → transport.write

Qt threading — the key trick: the reader thread feeds the controller via the internal _rawReceived signal connected with AutoConnection (default). It delivers synchronously when emitted from the GUI thread (simulator/tests need no event loop) and queues onto the GUI thread when emitted from the serial reader thread (thread-safe). Preserve this — don't add explicit Qt.QueuedConnection or locks around it.

Entry point quirk: cim_suite/shell/app.py uses relative imports within the package, so it must run as a module (python -m cim_suite.shell.app). The PyInstaller bundle entry is packaging/suite_launcher.py, which imports the package absolutely — running app.py directly as the frozen entry crashes with "attempted relative import." Don't "fix" this by flattening imports.

VB6-fidelity rules (where silent wrong-data bugs hide)

This is a reverse-engineered instrument protocol. The codecs intentionally mimic VB6 semantics — when touching protocol/codecs.py, decoder.py, or encoder.py, keep these exact:

  • Signed hex. Station values are scaled integers in hex, parsed as signed 16-bit (CInt("&H"...)): &H8000 = 32768, &HFFFF = 1. Use the signed helpers; never plain int(s, 16).
  • Int() floors toward −∞, not toward zero. encode_scaled uses math.floor for this reason (matters for negative calibration offsets). Python's int() would be wrong.
  • Sentinels are per-field, not global. 8000/FFFF8000 means "no value" in value/limit fields but is a legitimate number in the scale/offset (AddFloat) fields — hence the separate decode_float vs decode_scaled. Don't unify them.
  • Each field can have its own scale (the format-code argument, 06 = decimal places). Map them individually from the parse routine, mirroring UpdateService in Main.frm.

The general version of these traps is documented in docs/VB6-MIGRATION-PLAYBOOK.md §5 — read it before reverse-engineering any new protocol field or the next app.

Wire protocol shape

Inbound: the station streams {<TypeLetter><field0>\t<field1>...} frames. Decoder type letters: A full sensor record, B average update, C/I current value, D/E station setting lines, F status frame, G alarm limits, H statistics. Outbound commands are built in encoder.py (e.g. {Y} refresh-all, {X} refresh stats, {V} reset, {T...} set clock, and {<letter><fmt><id><payload>} settings writes). The controller's set_* methods map UI grid columns to these letters.

Hardware verification (still pending)

Five protocol details can only be confirmed against a real DA-12 and are each isolated to a small, clearly-commented spot: station clock epoch/format, sensor type/Calc enum codes, the 16-channel message/letter mapping, and especially the F status-frame layout (decoded but unverified — decoder._decode_status is flagged, and the type-F counters are deliberately not surfaced in the UI until a real capture exists). Work through docs/HARDWARE-VERIFICATION.md with hardware connected. The repo has no ground-truth frame captures yet (Exceptions.txt/VB187.tmp are a crash log and a form backup) — the first real frames you capture should be saved as test fixtures.

Conventions & docs

  • Branches: main is now the Python rebuild mainline (the rebuild was promoted to main on 2026-06-02). rebuild/python-pyside6 still exists, pointing at the same commit — either branch is fine to keep working on; they diverge from wherever you commit next. The original VB6 baseline is preserved per-module under cim_suite/modules/<id>/legacy/ (read-only), not as a branch. Commit per logical step; keep tests green.
  • Dropped legacy features (wireless sensors, OP-05 annunciator, Debug tab, factory serial-number write) are documented with exact restore locations in docs/superpowers/specs/2026-06-01-da12-service-tool-rebuild-design.md. Don't silently re-add or assume they're gone for good — they're restorable on request.
  • Data locations (never next to the exe): config JSON, calibration history (DACal.csv), and per-sensor logs (DA-Logs\<serial>.txt) all live under the user app-data folder via cim_suite/modules/da12/config.py.
  • The doc pair: docs/BACKLOG.md is the live to-do list; docs/REBUILD-STATUS.md is point-in-time status. They cross-link — keep them in sync when finishing an item.
  • UI edit loop: when repopulating a table from the model, block its itemChanged signal so programmatic fills don't fire phantom "user edited" events.
  • Design system: all UI follows the Instrument design system — docs/Instrument Design Spec.md is the authoritative spec (token/type/component values) and docs/DESIGN-SYSTEM.md is the working guide with the per-module adoption checklist. Implemented as a self-contained theme layer at cim_suite/core/ui/theme/ (two-theme tokens → IBM Plex fonts → type styles → stylesheet → apply_theme) plus the component kit at cim_suite/core/ui/kit/ and window chrome at cim_suite/core/ui/chrome/. Never hard-code colors/fonts/px or call setStyleSheet in module code (tests/core/test_design_discipline.py enforces this); custom paints read theme.current() at paint time and repaint on themeChanged. The mockup HTML is visual reference only — never pixel-measure it.
  • Spreadsheet export: every data screen can be exported to .xlsx; the engine is cim_suite/core/export/ and the per-module adoption checklist is in docs/EXPORT.md.
  • Versioning & changelog: SemVer, single-sourced from [project] version in pyproject.toml (cim_suite.__version__ reads it; the frozen build bundles the file; the Inno Setup installer takes it via ISCC /DAppVersion=... — see Commands). Commits are Conventional Commits (<type>: <desc>) — validated by .githooks/commit-msg (install once with python scripts/setup_hooks.py). The deterministic python scripts/bump.py computes the next version from commit prefixes (fix=patch, feat=minor, BREAKING CHANGE/!=major) — never let an LLM pick the bump. The pure logic lives in scripts/release/ (tested in tests/release/); git I/O is isolated in scripts/release/gitio.py. Two records, two audiences: git history is the technical record; CHANGELOG.md is plain-language release notes for non-technical users (also shown via the launcher's "What's new" dialog). The changelog prose is the only LLM-assisted step — draft it with the /release-notes command, and never commit it unreviewed; accuracy over flattery (don't invent user benefits a commit doesn't support).