Reverts optimistic send. The browser times the press with performance.now() and
sends it once on release, so the firmware sees exactly the press that was made.
Optimistic send fired a speculative tap at pointerdown plus a held press if the
button was still down. It was 152 ms faster (505 vs 657 ms click-to-visible at
400 ms RTT) but it guessed, and a wrong guess sent both presses for the firmware
to act on. Raising the threshold to 900 ms hid the symptom without removing the
failure mode, and it also made hold-to-repeat unreachable, since the server
released the key after a fixed 900 ms no matter how long you held.
Measuring costs the click duration in latency and buys exactness plus real
hold-to-repeat. Verified against the firmware:
taps under 400 ms 120/250/390 ms -> cursor +1, submenu never opens
holds from 400 ms 500/900/1500 ms -> cursor +3/+8/+15
MENU tap 120/300/390 ms -> menu opens, submenu stays shut
One correction to my own expectations along the way: I first recorded the multi-step
moves at 500 and 800 ms as failures. They are not. App/misc.c has
key_repeat_10ms = 8, so past 400 ms the firmware auto-repeats every 80 ms, and the
counts match (duration - 400) / 80. That is what a real radio does when you hold a
button, so the note on FIRMWARE_HELD_MS now says not to filter it out.
MIN_HOLD_MS returns as the floor for a measured press, since a very fast click can
measure below the debounce window. LONG_PRESS_AFTER_MS and LONG_PRESS_MS are gone
with the scheme that needed them.
Three things reported from actual use, plus the bug the logging exposed.
1. Keys are logged (source "key"), including refusals and presses while powered
off. Without this there was no way to tell "the key never arrived" from "the
firmware did something else with it" -- which is exactly what was needed below.
2. /stream resends the current frame every IDLE_FRAME_INTERVAL_S even when nothing
changed. Change-detection alone made a static screen indistinguishable from a
dead connection, and a client joining mid-idle stayed blank. Measured: 6 frames
in 10 idle seconds, where before it was 0.
3. Off now actually blanks the screen. The dark panel moved to a .screenwrap
wrapper and the <img> is hidden; setting a background on the <img> alone did
nothing visible, because the image kept painting the last frame over it.
Then the reported bug: in the menu, UP/DOWN behaved like another MENU press. The
key log made it diagnosable and the cause was mine -- LONG_PRESS_AFTER_MS was set
to the firmware's own 400 ms boundary, but a deliberate click runs 100-500 ms, so
ordinary clicks sent tap AND held and the firmware acted on both:
held DOWN auto-repeated, gMenuCursor 3 -> 12 from one click
held MENU entered the submenu, gIsInSubMenu 0 -> 1
The UI threshold is now 900 ms, well clear of any click, and FIRMWARE_HELD_MS is a
separate constant so the two are not conflated again. Verified against the real
firmware: 120/300/500/800 ms clicks each move the cursor exactly +1 with
submenu=0, while a deliberate 1400 ms hold still auto-repeats (+9).
Three sources into one buffer: power events from the supervisor, QEMU's stderr
(which run.sh and the tests used to discard), and firmware serial, which the
machine model tags SERIAL. default_launcher now captures stderr rather than
sending it to DEVNULL, which is what made the last two reachable.
The pane is a fixed-height scroll box as asked: 180px with overflow-y:auto, so it
never grows with content -- older lines move up out of view and you scroll back to
read them.
Two details that make that usable rather than annoying:
- Autoscroll only sticks when you are already at the bottom. Otherwise a new line
arriving would yank the view away from whatever you had scrolled up to read.
- MAX_LOG_LINES caps the <pre> as well. The box is fixed-height either way, but an
unbounded DOM node would still grow memory across a long session.
Verified on the live server: power on produced power/qemu/serial lines including
"UV-K5 Firmware, EGZUMER-F4HWN+NR7Y c91cec95", each Reset logs the event and the
banner reappearing, and since= never resent a line. A capacity-500 buffer fed 2000
lines keeps exactly 500 and does not replay evicted entries to a stale cursor.
Collects power events, QEMU stderr, and firmware serial output. Bounded and in
memory on purpose: an unbounded buffer in a long-running server is a slow leak,
and anyone wanting a permanent record can redirect the server's stderr.
Entries carry a monotonic seq so a polling client can ask for "anything after N"
and receive each line exactly once, including after eviction has dropped older
entries -- a test covers that case specifically, since an index-based cursor would
silently repeat or skip lines there.
Stream decoding is lenient: serial bytes can be garbage before the firmware
configures the port, and losing the stream to one bad byte would be worse than a
replacement character.
Two changes, both aimed only at latency, since that is what the link makes
expensive.
1. Send on pointerdown instead of pointerup. Waiting for release left the network
idle for the entire click. The duration is unknown at that moment, so the
speculative request asks for a short press; holding past LONG_PRESS_AFTER_MS
sends a second, deliberately long press, which is how held events stay
reachable.
2. TAP_MS 200 -> 60 ms. The server blocks for hold_ms before replying, so this is
latency the user pays directly. 200 ms was a guess that gave back most of what
change 1 saved.
The 60 ms is measured, and the sample size mattered: at 4 trials per value 30 ms
looked reliable, but at 12 trials 20 ms registered only 5/12 while 30 ms was 12/12.
The nominal 20 ms debounce is not sufficient alone because KEYBOARD_Poll samples
each column 8 times wanting 2 matching reads. 60 ms is double the proven floor.
Click-to-visible at 400 ms RTT, where one round trip is an unavoidable 400 ms:
original (2 requests, on release) 2525 ms (+2125 over the floor)
one request, on release 657 ms (+257)
current (1 request, on press) 505 ms (+105)
Long press still works: a 60 ms press opens the menu (gScreenToDisplay 0 -> 1)
while a 900 ms press is treated as held and correctly does not, so the firmware
still distinguishes them.
Also drops MIN_HOLD_MS, now dead: the browser no longer measures press duration,
so there is no measurement to clamp. Its test asserted only that the string
appeared, which would have kept passing over dead code.
The firmware has been printing all along and nothing was listening. USART1 has no
real model here -- it is one of the logging catch-all stubs -- so every byte went
into qemu_log_mask(LOG_UNIMP) and vanished.
Two things were needed, and the second was not in the plan:
1. Print USART1 DR writes (+0x04, per the vendor CMSIS header) as SERIAL lines.
2. Report TXE|TC in USART1 SR. This is the part I had missed. UART_Send() in
App/driver/uart.c spins on LL_USART_IsActiveFlag_TXE() with a bounded timeout
and *skips the byte* when the flag never sets. A stub returning 0 for SR meant
the firmware discarded its own output before it ever reached DR -- the only
write arriving was UART_Init()'s priming zero. So step 1 alone produced
nothing, which is why the first attempt looked like "the build has no logging".
Then a bug of my own: the priming byte is 0x00, I buffered it, and fprintf("%s")
stopped at that NUL and printed an empty line while all 46 bytes sat behind it.
NULs are now dropped, and a line flushes on CR as well as LF.
Verified: SERIAL UV-K5 Firmware, EGZUMER-F4HWN+NR7Y c91cec95
keypad_test.py still passes, which matters because this file is where removing
three fprintfs once silently deleted the keypad.
The server now owns the QEMU process by default but does not launch it. You open
the page to a dark screen and press On, which is the behaviour asked for: like
walking up to a machine rather than finding it already booted.
This inverts the flag from the plan. Owning the process has to be the default,
since it is the only way On/Off can work at all; --attach is the opt-in for joining
a run.sh instance, where Off is refused.
Two tests guard the intent rather than the wiring: one asserts main() has --attach
and not --own-emulator, another asserts main() never calls power_on(), so a future
edit cannot quietly restore auto-boot.
Verified on the live server with no QEMU running beforehand:
startup 0 QEMU processes, powered=false, frame.png 503, page 200
On 1 QEMU process, powered=true, frame.png 200 (2920 bytes)
Off 0 QEMU processes, frame.png 503, page still 200, keys 409
On again 1 QEMU process, powered=true, frame.png 200
The web server stays up across Off, which is what you asked for: the screen goes
dark and waits for the next person to press On.
Buttons sit above the LCD as asked, with a state label that turns green when the
guest is up. Powered off dims the panel via a screen-off class, so a dark screen
is the signal rather than a frozen last frame.
Off asks for confirmation: it ends the guest, and a stray click should not do that
silently. Buttons disable while a power action is in flight, since On and Off take
a couple of seconds and a double click would race.
The stream is restarted after every power action. The old multipart response ends
when the emulator goes away, so without a fresh src the image would stay blank
after On.
Refusals are surfaced in the status line rather than swallowed -- that is how the
409 for an adopted emulator becomes visible instead of looking like a dead button.
49 unit tests, and the generated page passes node --check.
POST /api/power/{on,off,reset,pause,resume}, and every route now tolerates there
being no emulator: /api/status reports powered:false, /frame.png returns 503,
keypresses return 409 with "press On first". Previously create_app required a live
client and the whole page would 500.
The client is fetched through the supervisor per request rather than captured once,
because a power cycle replaces it and the captured one goes stale.
After any power action the pump is rebound, so Off actually goes dark instead of
freezing on the last frame.
Off is refused with 409 for an adopted emulator: we did not start that process.
Reset is allowed either way, since system_reset ends nothing.
Verified over HTTP with a real supervisor and real QEMU:
start powered=False qemu=0 frame=503
key 409 as expected
On powered=True qemu=1 frame=2920 bytes
Reset qemu=1 (process survived)
Off powered=False qemu=0 frame=503
On again powered=True qemu=1 frame=2920 bytes
The web server stayed up throughout, which is the requested behaviour.
Power on/off cannot live inside the QMP connection: QMP quit destroys the socket
a later power on would have to arrive through. So something outside it has to be
able to spawn the process again.
Off then On is a cold boot -- process replaced, guest from reset -- which is the
behaviour asked for: like cutting mains power and restoring it. system_reset is the
warm alternative and keeps the process.
adopt() is for attaching to a run.sh instance. power_off then refuses, because we
did not start that process. is_running() also reports False for a process that
exited on its own, rather than trusting our own bookkeeping.
power_off tolerates quit raising: the socket usually drops before the reply
arrives, so that is success rather than an error. The launcher clears a stale
socket first, since QEMU failing to bind presents as On doing nothing.
Verified against real QEMU: starts with 0 processes, On gives 1, Reset keeps the
same one, Off returns to 0, On again cold boots. 14 unit tests with a fake
launcher, so they need no emulator.
/stream and /frame.png now read the pump's shared buffer. A slow client falls
behind in frames rather than in QMP reads, and reconnects no longer multiply the
load on the emulator.
/frame.png returns 503 when there is no frame rather than raising, because that is
a real state: the emulator can be powered off and the page still has to load. The
same reason create_app now tolerates client=None.
Measured on the live server with 4 concurrent streams, which is what a reconnecting
browser produces: keypress latency went from 6 ms avg to 5 ms, so -1 ms, i.e. noise.
All four clients received the same ~27 frames. /stream first byte in 4 ms.
Note this rules out my earlier guess: I had assumed concurrent streams were
starving keypresses, and the numbers said otherwise both before and after. The
pump is worth having for constant QMP load, not because contention was the
slowness.
One thread grabs the LCD at a fixed rate into a shared buffer; HTTP clients serve
from that buffer. Before, every /stream iteration issued its own QMP reads, so
load scaled with client count and reconnects, and a slow reader could stall the
grab loop.
Encodes only when the framebuffer bytes change, and exposes generation so a client
can tell a new frame from the same one without comparing bytes.
rebind() is here for the power work in a later task: the emulator comes and goes
under the server, and rebind(None) blanks the screen instead of leaving a stale
frame that looks live. The run loop re-checks the grabber identity under the lock
after a read, so a rebind landing mid-read cannot be undone by the frame it was
already fetching.
A client that raises is swallowed on purpose -- a dead emulator must not kill the
pump, since power may come back.
Verified against a real emulator: 5000 latest() calls in 1 ms with no QMP traffic,
generation flat at 1 over a second of static screen, guest still running, and
rebind(None) going dark. 9 unit tests.
The browser now times the press with performance.now() and posts hold_ms once,
instead of sending down and up as two requests. Halves the round trips per key and
makes press duration independent of the link.
Deletes test_sends_down_and_up_not_just_tap: it asserted the behaviour being
replaced, so keeping it would have meant asserting the bug.
MIN_HOLD_MS is injected into the page from webui.py so the two agree on the floor,
which exists because anything under the firmware's 20 ms debounce does not register
at all -- a very fast click still has to ask for 60 ms.
Verified over a simulated 400 ms link: three keys in 1.58 s where the old design
needed ~2.45 s, and a 120 ms press opens the menu (gScreenToDisplay 0 -> 1) with
DOWN then moving the cursor. The generated page also passes node --check.
POST /api/key now accepts {"key": "MENU", "hold_ms": 120} and holds the key for
exactly that long, locally.
The old design sent down and up as two requests so the browser would own press
duration. That is correct on loopback and broken over a real link: the round trip
between the two requests *is* the press duration. Measured against this server at
400 ms RTT, an intended tap arrived as a 407 ms hold, and since the firmware reads
400 ms as held (key_repeat_delay_10ms = 40), every short press was dispatched to
the hold path where MAIN_Key_MENU does nothing. Jitter either side of that
threshold is why it felt intermittent rather than simply broken.
Verified at 400 ms simulated RTT:
one request 531 ms total, firmware saw 120 ms -> short press
two requests 816 ms total, firmware saw 408 ms -> held (the bug)
hold_ms is clamped to MAX_HOLD_MS, rejects negatives and non-numbers, and defaults
to TAP_MS. A deliberate 900 ms hold is preserved, so long-press events still work.
down and up stay for scripting on a fast link.
The rules restricting port 8080 to DN42 were applied by hand and existed only in
the live kernel tables -- lost on reboot, with the port then wide open and nothing
in the repo to say it ever had been restricted.
The script is idempotent (checks before adding) and has remove/show. show prints
packet counters, which is the part that matters: this host's INPUT policy is
ACCEPT, so a rule that only allows DN42 does nothing at all. The final DROP is
what restricts anything, and a rising DROP count is the only proof it works
rather than the traffic simply not arriving.
Currently observed: 2203 accepted from DN42 v4, 103 from v6, 22 dropped.
README gets a section with the endpoint table, the two constraints that will
otherwise surprise someone (single QMP client, no authentication), and the reason
frames go through memsave rather than pmemsave or gdb.
AGENTS.md gets the run instructions plus a new entry under 'Things that already
went wrong' for the pmemsave trap: it takes a physical address, returns zeros for
gFrameBuffer, and reports success. The web UI was built on it initially because a
benchmark showed it was fast -- the benchmark never checked the contents. Worth
recording as the general lesson, not just the specific fix.
Boots its own emulator on a private socket and port, so it neither disturbs a
run.sh session nor fights it for the single-client QMP socket.
The unit tests stub QMP and therefore cannot catch a wrong QMP command. That is
not hypothetical: pmemsave and memsave differ only in physical versus virtual
addressing, and pmemsave fails by silently returning zeros. Check 2 asserts the
frame has lit pixels for exactly that reason.
Six checks, all passing: PNG renders, screen has content (1883 lit pixels), a
MENU tap changes the screen, the guest is still running after streaming, PTT is
rejected with 400, and /stream is multipart with a PNG part.
Waits 16 s before the first press, past power-save entry, so it also covers keys
working once the radio is asleep.
Flask app on the existing QMP socket. GET / serves the page, GET /stream is a
multipart PNG sequence at up to 15 fps, POST /api/key drives the keypad model.
The keypad takes discrete down/up rather than a fixed-duration tap, because the
firmware reads anything past 400 ms as a held key and dispatches it differently.
Letting the browser own the timing is what makes both short and long presses
reachable; tap stays available for scripting and is pinned between the 20 ms
debounce and the 400 ms hold, with a test asserting that range.
The stream only re-encodes when the framebuffer bytes change. The LCD is static
most of the time, so idle CPU stays near zero.
Unknown key names are rejected with 400 before reaching QMP, so a PTT button
cannot be added by accident. The front end releases on pointerleave,
pointercancel and blur, and /api/release-all is the safety valve for a key that
somehow stays down.
Verified against a live emulator over HTTP: /frame.png renders the dual-VFO main
screen, a MENU tap opens the menu at 01/79, and two DOWN presses reach 03/79.
19 unit tests, 41 across the whole suite.
Rejects unknown names in the server rather than letting them reach QMP as an
error. A test parses keypad_key_names out of qemu/py32f071.c and fails if the
lists drift, so adding a key to the model cannot silently leave the web UI
behind.
PTT is deliberately absent: the model wires it separately on GPIOC, so the press
property rejects the name.
memsave, not pmemsave. The plan specified pmemsave on the strength of a timing
measurement that never checked the contents; it turns out pmemsave takes a
*physical* address and silently returns zeros for gFrameBuffer's virtual
address. No error, no warning -- just a permanently blank screen.
Caught by rendering a real frame and finding 0 lit pixels where the gdb path
reported 1693. With memsave the count matches exactly, and the image reads
correctly: both VFOs at 18.00000 MHz, PS/DWR/CL status bar.
Two tests guard the decisions rather than the current text: the stub client
raises if pmemsave is ever used, and a source check rejects subprocess/popen so
frame reads cannot regress onto gdb, which would halt the guest.
Measured 1.35 ms per frame with the guest still reporting status running.
Lifted from tools/screenshot.py so the web UI cannot drift from the CLI
screenshotter. Verified unpack() is identical to the original across 200
randomised trials, and the PNG IHDR matches byte for byte.
encode_png() returns bytes instead of writing a file, and drops to zlib level 6:
at streaming rates the CPU saving beats the last few bytes on loopback.
Wraps the connect/negotiate/command dance with a lock, since only one client
can hold the QMP socket at a time. Events interleave with replies, so command()
reads until it sees the matching return rather than trusting the first message.
Connection failure carries the reason and the likely cause: key.py already
holding the socket.
AGENTS.md still carried a stale entry telling the reader to *lengthen* key
holds when a press seems ignored, which is the opposite of the fix and is
what broke the tooling in the first place. Replaced with the correction and
a pointer to the right section.
The keypad heading also claimed the hold time was the only cause. There were
two: the 2500 ms hold in key.py, and row_out missing volatile. Both are now
listed up front with a link to the detail.
Adds the regression test to the places someone would actually look: the
"How to run it" section in AGENTS.md, the layout listing, and a build step
in the README noting that a clean build is not evidence the keypad works,
since the -O2 dead-code elimination produces no warning.
The previous commit removed three TRACE fprintfs from py32f071.c as
cleanup. That silently broke the keypad completely -- no press reached the
UI, and nothing warned about it.
Root cause is dead-code elimination, not the printing.
qdev_init_gpio_out_named() is inlinable and only records the row_out array;
the lines are filled in later by qdev_connect_gpio_out_named() from board
code, which GCC cannot see. At -O2 GCC therefore proves every element is
still NULL, sees that qemu_set_irq() returns immediately on a NULL irq, and
deletes the body of keypad_update_rows() along with all five calls to it. No
row line is ever driven and the firmware's scan reads all-high.
From the object code:
callers reaching keypad_update_rows
plain none -- the calls are gone
volatile keypad_key_changed, keypad_col_changed, keypad_set_press,
keypad_reset, uvk5_machine_init
keypad_col_changed compiles to a store and a ret with no call at all; with
volatile it ends in jmp keypad_update_rows. Declaring row_out volatile fixes
it at the cause. 10/10 on the press test, 3/3 on keypad_test.py, no build
warnings.
Scoped rather than assumed: PY32GpioState::out is not affected. Marking it
volatile too gives a byte-identical object file, because py32_gpio_write()
is only reachable through a MemoryRegionOps function-pointer table so GCC
cannot enumerate its callers. It stays plain.
Adds tools/keypad_test.py: boots its own instance on private ports and
checks that a short MENU press opens the menu, DOWN moves the cursor, and a
held key is visible to the scan. This is what should have caught the
breakage before it was pushed.
Docs corrected. The breakage had been written up as "power save stops the
keypad scan" and called a gap in the model; it was neither. AGENTS.md now
records the mechanism, the measurements, the objdump check, and the two
measurement traps that made this hard: reading gKeyReading0 after releasing
the key (always KEY_INVALID), and trusting a gdb breakpoint on
KEYBOARD_Poll (with the guest stopped the scan's delays cost no guest time,
so Poll returns KEY_MENU on a build where it fails when running free).
README screenshots regenerated from the current build.
key.py held every key for 2500 ms, on the assumption that guest time runs
fast during delays so a press needs a long wall-clock hold. That is wrong
for this path, and it is why the keypad looked dead.
The two SysTick mechanisms are separate. poll-boost accelerates counter
*reads* so SYSTICK_DelayUs converges; it does not speed up interrupt
delivery. Interrupts drive SysTick_Handler -> gNextTimeslice ->
APP_TimeSlice10ms -> CheckKeys at close to real time, so the firmware's
thresholds hold in wall clock as written: 20 ms to register a press,
400 ms to count as held.
2500 ms is ~250 ticks, six times past the long-press threshold, so every
press was dispatched as a hold. MAIN_Key_MENU acts only on a short release
and returns early when bKeyHeld is set, so nothing happened. Confirmed by
reading gDebounceCounter mid-hold: 317 after a 3 s hold, which also proves
the timeslice was running all along.
Now HOLD_MS=200 and LONG_HOLD_MS=900. Verified with screenshots: the menu
opens and UP/DOWN move through it.
Also here:
- Drop the three TRACE fprintfs. They fired on every keypad poll and
buried the console; the matrix is confirmed working.
- Drop a redundant forward declaration of py32_spi_xfer_byte, silencing
the only build warning.
- Document the real remaining gap: power save (~6 s after boot) stops the
keypad scan and the model does not wake from it. Includes the two dead
ends already ruled out by experiment, so nobody repeats them.
- Add README screenshots captured from guest memory.
Downloaded verbatim from apache.org, not transcribed.
qemu/py32f071.c stays GPL-2.0-or-later as its header says: it builds into
QEMU and derives from QEMU's device models, so it cannot be relicensed.
README notes the split.
Notes for whoever works on this next, weighted toward what the code does not
say: that the firmware is the reference and must never be edited to suit the
emulator, that register layouts come from the vendor CMSIS header rather than
inference, and that the way to find the next peripheral worth modelling is to
watch where the firmware stops.
Records the mistakes that already cost time here, each with the symptom that
made it look like something else: GDB breakpoints halting the guest (which reads
as 'the keypress does nothing'), writing the SysTick counter back while
accelerating it (which hangs the delay loop outright), lowering the clock to
speed up busy-waits (measured, 32x, nowhere near enough), unnamed qdev GPIO lines
sharing one namespace, and a probe script whose own regex silently matched
nothing.
Also states plainly what the emulator cannot answer, so a passing test is not
mistaken for evidence about radio behaviour.
Adds a QEMU machine for the Puya PY32F071 (Cortex-M0+) so Quansheng UV-K5 V3
firmware can run on a PC. The firmware boots to its main loop in about five
seconds and the LCD contents are readable.
Register layouts come from the vendor CMSIS header shipped with the firmware
rather than guesswork. Modelled: RCC, GPIO, ADC, both SPI controllers, DMA1 and
the PY25Q16 flash; everything else answers through a logging catch-all, which is
how the next thing worth modelling gets identified.
Seven things had to be right before it would boot, each found by watching where
the firmware stopped: flash aliased at the application offset, clock ready bits,
self-clearing ADC calibration, SPI transfer flags, DMA-driven flash reads,
SysTick poll acceleration, and the bit-banged transceiver bus idling low.
SysTick needs explanation. SYSTICK_DelayUs polls the counter and accumulates
differences; under emulation a register read costs far more relative to guest
time, so a measured 120 ms delay would have taken about 7.7 hours. Lowering the
clock does not help because the bottleneck is loop iterations, not counter speed.
Reporting a value that runs ahead of the real counter does, via a new poll-boost
property on SysTick. Guest time therefore runs fast during delays: fine for
exercising menus and control flow, wrong for judging signal timing.
Also includes the host build of the CW timing chain (harness, stubs, shim,
tests), which compiles app/cwkeyer.c and app/cwmacro.c unmodified against stub
drivers with a virtual clock and scripted paddle input.
Known gap: keypad rows reach the firmware's scan and KEYBOARD_Poll returns the
right key code, but the UI does not react yet.
Not modelled, and not intended to be: radio behaviour. The transceiver chip has
no public datasheet, so keying envelopes and emissions need real hardware.
mckero