mirror of
https://github.com/MCKero6423/uv-k5-v3-emulator.git
synced 2026-10-02 03:15:36 +00:00
Emulator: multiboot slots from the page, flash controller, portable tests
flash controller: store ACR/OPTKEYR instead of swallowing them, which is what stopped the factory bootloader from starting slots over the firmware's own serial protocol (0x0720 family); uvk5_socket/uvk5_testenv so a fresh checkout skips instead of failing; web UI slot table and Multiboot button; quick start, CONTRIBUTING, and stop tracking firmware images and radio dumps
This commit is contained in:
mckero committed
1 parent
ee80939c78
commit
2667e046e8
54 files changed
+5345
-347
No files matched your search
+38
@@ -22,3 +22,41 @@ build/
|
||||
# Python bytecode from the tools tests.
|
||||
__pycache__/
|
||||
*.pyc
|
||||
|
||||
# Firmware is not ours to redistribute, and the localised builds in particular belong
|
||||
# to whoever made them. tools/fetch_firmware.py puts a release in assets/firmware/ on
|
||||
# request; nothing there is committed.
|
||||
assets/firmware/
|
||||
|
||||
# The radio's EEPROM dump: settings and calibration from a real radio, which is not a
|
||||
# build artifact. The tests build their own images from assets/pristine/ instead.
|
||||
work/data.bin
|
||||
work/f4hwn/
|
||||
work/**/*.img
|
||||
work/**/*.bin
|
||||
work/**/*.elf
|
||||
work/**/*.uf2
|
||||
|
||||
# Scratch output from a debugging session. The scripts stay: they document how this
|
||||
# machine is driven, and running one is how you reproduce what they describe.
|
||||
work/**/*.log
|
||||
work/**/*.png
|
||||
work/**/*.js
|
||||
work/**/*.tmp
|
||||
work/py*/
|
||||
work/serial_probe.py
|
||||
work/serial_write_probe.py
|
||||
work/qmp.py
|
||||
work/qmp_bridge.py
|
||||
|
||||
# Junk that a Windows redirect created once: literal "%SystemDrive%" directories full
|
||||
# of cache databases. They were never meant to be here.
|
||||
%SystemDrive%/
|
||||
**/%SystemDrive%/
|
||||
# work/ is scratch -- images, logs, QEMU stderr, captures. The four scripts and the
|
||||
# README are the parts worth keeping, so whitelist those rather than trying to list
|
||||
# every extension a debugging session produces (a rule that missed one let four
|
||||
# QEMU stderr logs get staged).
|
||||
work/*
|
||||
!work/*.ps1
|
||||
!work/README.md
|
||||
@@ -76,8 +76,9 @@ save byte, `0x0E70` for the VFO indices, and so on. No metadata, no directory, n
|
||||
checksum -- just an address that the code and the data both have to agree on. When
|
||||
a setting reads back wrong, suspect the offset before suspecting the transport.
|
||||
|
||||
The ~15 s to reach the main loop is emulation overhead. A real radio is up in about
|
||||
a second.
|
||||
Boot time is emulation overhead. Measured on this machine: first pixels at ~1.6 s and
|
||||
a drawn main screen at ~3.6 s after QEMU starts, which is the "~5 s" the README quotes.
|
||||
A real radio is up in about a second.
|
||||
|
||||
## Ground rules
|
||||
|
||||
@@ -147,6 +148,36 @@ Two things about it that matter when working on this repo:
|
||||
Its tests: `tools/test_uvk5_*.py` and `tools/test_webui.py` need no emulator,
|
||||
`tools/test_webui_e2e.py` boots its own.
|
||||
|
||||
A firmware can also be loaded from the page rather than from the command line:
|
||||
`POST /api/firmware` takes the image as its request body, stores it in
|
||||
`work/firmware/`, and boots it -- restarting the emulator if it was running. The
|
||||
image's **shape** is read out of the image (`tools/uvk5_image.py` on the host,
|
||||
`uvk5_sniff_app_offset()` in the machine): an *application* image is linked for
|
||||
`0x08002800`, a *full-flash* image starts at `0x08000000`, and address 0 has to alias
|
||||
the matching base. Getting that wrong is silent -- the image lands 0x2800 bytes off and
|
||||
the first fetch reads whatever data is there -- which is why it is not a flag and not a
|
||||
file-name convention. A file that is not an image is refused without disturbing the
|
||||
running radio.
|
||||
|
||||
Two things about that path are worth knowing, both found the hard way:
|
||||
|
||||
- **The flash image travels in the environment, not in `-M`.** Through the launcher,
|
||||
QEMU rejected `-M uv-k5-v3,flash-image=...` with "unsupported machine type": the
|
||||
identical argv started fine when run by hand, `-M help` in the *same* context listed
|
||||
the machine, the argv `repr` was clean, and the environment diffed down to nothing
|
||||
conclusive. The property still works when it is set, so both are supported; the
|
||||
launcher now passes the bare machine name plus `UVK5_FLASH_IMAGE`, which the model
|
||||
reads as a fallback. The root cause is unexplained -- do not "clean this up" without
|
||||
re-testing a power-on from the page.
|
||||
- **The screen is read from the display controller, not from guest RAM.** The panel
|
||||
model keeps the controller's own display RAM (8 pages of 128 columns), and the web
|
||||
page renders that, so the picture is right for *any* firmware -- builds sharing an
|
||||
ancestor still differ in their display logic, and the multi-system release keeps its
|
||||
image somewhere else entirely. Do not apply the driver's `0xA1` segment reverse on
|
||||
top of the data: measured at one instant against the guest's own framebuffer, 8153 of
|
||||
8192 pixels agree with no mirroring and 6557 with it. `memsave` of `gFrameBuffer`
|
||||
remains the fallback for an emulator built without the panel model.
|
||||
|
||||
## The flash bugs: four faults, one symptom
|
||||
|
||||
"The frequency will not change" and "flash forgets everything after power off"
|
||||
@@ -251,6 +282,16 @@ reported `IDR=0x0000` for several rounds because its regex did not match gdb's
|
||||
output format at all. The register was fine; the reader was broken. Cross-check
|
||||
with `tools/gpiob_dump.sh`, which uses a different path.
|
||||
|
||||
The same trap one layer further out: **a redirect can change the encoding.** Three
|
||||
probe runs under `qemu ... 2> probe.log` reported zero SPI transfers, zero flash
|
||||
reads and zero chip-select changes, and "the firmware never touches SPI" was written
|
||||
down as a finding. PowerShell 5.1 writes `2>` as UTF-16LE, so every ASCII line a
|
||||
probe printed had a NUL between each character and a `startswith("LCDW")` filter
|
||||
could never match it. Decoding the same file as UTF-16 showed a complete ST7565 init
|
||||
sequence and 48 distinct settings reads. Before believing an empty probe, check that
|
||||
the probe *can* be seen: read the file, count its bytes, or write it from `cmd /c`,
|
||||
which does not re-encode.
|
||||
|
||||
**QMP `pmemsave` is physical, `memsave` is virtual.** The framebuffer symbols are
|
||||
CPU virtual addresses, so `pmemsave` on `gFrameBuffer` returns a block of zeros
|
||||
and reports success -- a blank screen with nothing logged anywhere. The web UI was
|
||||
@@ -259,6 +300,136 @@ benchmark never checked the *contents*. Measure the thing you actually care
|
||||
about: the bug surfaced only when a rendered frame came back with 0 lit pixels
|
||||
where the gdb path reported 1693.
|
||||
|
||||
### The page is generated by an f-string, so check the script it serves
|
||||
|
||||
The web UI is one f-string. A stray backslash in a JavaScript string literal therefore
|
||||
produces a page whose **whole** `<script>` fails to parse, and the only symptom is that
|
||||
the status line sits on "connecting..." forever while every endpoint still answers
|
||||
`curl` correctly. That shipped once: `.split('\\')` came out as `.split('\')`, an
|
||||
unterminated string, and the page was dead from a browser's point of view while every
|
||||
test passed.
|
||||
|
||||
`test_webui.TestPageScriptParses` extracts the served script and runs `node --check`
|
||||
on it now. Test the artifact you ship, not the code that builds it.
|
||||
|
||||
### A probe needs to be able to see the thing it is looking for
|
||||
|
||||
Three separate rounds of "the firmware never touches the flash" were all the probe's
|
||||
fault, and each one looked like a finding:
|
||||
|
||||
- A probe filtered on `address >= 0x0C0000`, so every frame without an address -- write
|
||||
enable, and the sector erase that actually erases -- was dropped. "0 writes" was the
|
||||
filter, not the firmware.
|
||||
- A handshake was given 1.5 s to answer and the firmware needed about 4 s to enter its
|
||||
serial mode. "No reply" was the timeout.
|
||||
- Why a probe can be invisible at all: PowerShell 5.1 writes `2>` as UTF-16LE, so every
|
||||
line had a NUL between each character and no filter could ever match.
|
||||
|
||||
Before believing an empty probe, make it print something you know is there.
|
||||
|
||||
### The flash model wrote the whole image back on every chip-select release
|
||||
|
||||
2 MB per release is nothing for a settings save. It is ruinous for the multi-system host
|
||||
interface, which programs a slot 200 bytes at a time
|
||||
(`App/app/uart.c`, `0x0724`, 12-byte header plus data): one 114 KB firmware became ~600
|
||||
full rewrites, on the vCPU thread, and the *guest* -- and every host tool talking to it --
|
||||
waited for each one. Measured: a single 64-byte slot write took six seconds.
|
||||
|
||||
The first fix was a 200 ms time-based throttle, which was wrong: it trades a slow test for
|
||||
silently losing the last window of writes on a hard kill. The model now tracks the changed
|
||||
byte range and writes only that, in place, which is both fast and the more faithful
|
||||
behaviour -- real NOR does not make an interrupted program atomic. The exit notifier still
|
||||
writes everything.
|
||||
|
||||
### The serial link carries the firmware's own screen stream
|
||||
|
||||
`K5Viewer` streams the display out of USART1. A host client that reads only while it is
|
||||
waiting for a reply backs the socket up, and the **guest then blocks** writing to it: a
|
||||
slot transfer started losing replies partway and a single small write took seconds. The
|
||||
fix is a reader thread that drains continuously and lets the waiting code look at what has
|
||||
been reassembled -- on the radio's side the same rule applies to whatever talks to it.
|
||||
|
||||
Also on that path: the firmware's receive buffer is 256 bytes
|
||||
(`App/driver/uart.c: UART_DMA_Buffer[256]`), so a 240-byte chunk plus framing overran it
|
||||
and every frame was dropped in silence; 200 fits. And the serial *session* times out after
|
||||
~6 s without a `0x0514` (`gSerialConfigCountDown_500ms = 12`), which a long transfer
|
||||
crosses -- measured by re-handshaking: the writes resume immediately.
|
||||
|
||||
With those four, `tools/uvk5_slots_serial.py` writes a slot through the firmware itself and
|
||||
the device validates the CRC.
|
||||
|
||||
### A serial client that connects after boot misses everything
|
||||
|
||||
`-serial tcp:host:port,server=on,wait=off` **discards** what the guest writes until a
|
||||
client connects. The firmware prints its banner in the first seconds, so a client that
|
||||
attaches "once QEMU is up" -- four seconds later, say -- sees an empty port and it looks
|
||||
exactly like a guest that never booted. Four rounds of "the bootloader sends nothing"
|
||||
were that, not the bootloader.
|
||||
|
||||
Connect first, then let the guest run. The same trap applies to the 0x0518 flood a
|
||||
bootloader emits while waiting for a host: it is continuous, so a late client *does* see
|
||||
it -- which is why the mistake survived as long as it did, showing up only for the
|
||||
one-shot startup output.
|
||||
|
||||
Two related habits, both learned here:
|
||||
|
||||
- **Check that the probe can see something you know is there.** A USART register probe
|
||||
reported zero accesses, and the obvious reading was "the bootloader never programs the
|
||||
USART". The application, run through the same probe, reported 2239 -- which is what
|
||||
said the probe worked and the bootloader really was silent.
|
||||
- **When a documented observation stops reproducing, treat the note as unverified.** The
|
||||
bootloader's Moto-mode flood was written down from a run that is no longer reproducible
|
||||
with the current build and image. Re-derive it before relying on it.
|
||||
|
||||
### A bare host:port is not a scheme
|
||||
|
||||
`uvk5_socket.connect` split its argument on ":" to find a scheme, so the endpoint the
|
||||
supervisor, the web UI and the README all pass -- a plain `127.0.0.1:4444` -- became
|
||||
scheme `127.0.0.1`, empty port, and an empty host. The connect then sat there until its
|
||||
deadline. What that looks like from outside is "the page cannot power the emulator on",
|
||||
while a QEMU started by hand with the identical command line answers QMP in half a
|
||||
second, and the guest boots happily in the background the whole time.
|
||||
|
||||
Two things made it hard to see: the same helper also accepts `tcp:host:port`, so the
|
||||
tests that used that form passed, and the failure is a *timeout* rather than an error, so
|
||||
it reads as a slow or wedged emulator. `test_uvk5_socket` now covers every form that
|
||||
reaches `connect`, and the lesson generalises: **when a helper accepts several spellings,
|
||||
test each one** -- the one nobody tests is the one everybody passes.
|
||||
|
||||
The other half of the same fault was real and independent: QEMU's stderr had to be
|
||||
drained from the moment it started. The firmware streams its display down that pipe, 64 KB
|
||||
fills in about a second, and QEMU blocks writing to it -- which stops its main loop, so
|
||||
QMP never answers either. Measured both ways: with the pipe drained, QMP accepts in 0.5 s;
|
||||
with it left unread, never.
|
||||
|
||||
### A register you swallow is a hang the next program waits on
|
||||
|
||||
The factory bootloader would not start at all: no serial output, and the PC probe sampled
|
||||
`0x08000f38` on every single sample. That address is inside the bootloader, and the two
|
||||
instructions there are
|
||||
|
||||
0x0f38: ldr r2, [r1] ; r1 = 0x40022000, the flash controller
|
||||
0x0f3a: lsls r2, r2, #30
|
||||
0x0f3c: lsrs r2, r2, #30 ; r2 = ACR & 3, the LATENCY field
|
||||
0x0f3e: cmp r2, #1 ; waiting for one wait state
|
||||
0x0f40: bne 0x0f38
|
||||
|
||||
The flash controller model added earlier treated `ACR` and `OPTKEYR` as writes to
|
||||
ignore -- it returned early, so the generic path never stored them, so `ACR` read back
|
||||
zero forever and the bootloader spun before it ever configured its UART. Returning `false`
|
||||
lets the value be stored, and the PC immediately moved into the application (`0x08013ea0`)
|
||||
and serial output appeared.
|
||||
|
||||
Two lessons, both general:
|
||||
|
||||
- **A write-only register is still a register.** The application never read `ACR` back, so
|
||||
its absence was invisible for as long as only the application ran. The next program to
|
||||
touch the same peripheral found it at once.
|
||||
- **"It used to work" is a bisect instruction.** The bootloader's Moto-mode flood had been
|
||||
observed before the flash controller was modelled, and stopped reproducing afterwards.
|
||||
The right move was to ask what changed between those two runs, not to distrust the
|
||||
earlier note.
|
||||
|
||||
## The keypad: two real bugs, both fixed
|
||||
|
||||
The old note here said "keys reach the firmware but the UI does not react" and
|
||||
@@ -416,10 +587,15 @@ Two related facts, both confirmed by experiment, so nobody spends time on them:
|
||||
mismatch on a fresh image, and writes the settings sector.
|
||||
`PY25Q16_WriteBuffer` erases the whole 4 KB sector before reprogramming, so a
|
||||
byte planted at `0x00A00B` is gone before the read at `settings.c:169` sees it.
|
||||
- **Guest-side settings changes do not persist.** The emulated PY25Q16 loads the
|
||||
image into RAM at realize time and never writes back, so anything the firmware
|
||||
saves is lost on restart. Adding a flush would be the fix if persistent
|
||||
settings are ever wanted. Nothing needs it today.
|
||||
- **Settings do persist now, which changes how to test.** The PY25Q16 model loads
|
||||
the image at realize time, keeps it in RAM, and writes it back over a temp file when
|
||||
CS is released or the process exits, so *every session leaves `assets/flash.img`
|
||||
changed*. Measured after one real session: the settings block at `0x00A000`, which
|
||||
starts life as all `0xFF`, held the guest's settings, `0x8000..0x8800` had moved,
|
||||
and the file differed from the pre-session copy in 2239 bytes. Diff against
|
||||
`assets/pristine/` (or a copy you kept) instead of assuming a fresh image, and power
|
||||
the emulator off before restoring it. On Windows this silently did nothing until
|
||||
`rename()` was replaced by `g_rename()` -- see the portability section.
|
||||
|
||||
Useful here: `tools/scan_trace.sh` (what the scan reads), `tools/key_result.sh`
|
||||
(what Poll returns), `tools/trace_run.sh` (the TRACE points).
|
||||
@@ -505,6 +681,12 @@ memory-map addresses match the model's `#define`s, that every long flag a doc pa
|
||||
a tool actually exists in it, and that documented firmware `file:line` references still
|
||||
point at what the prose claims.
|
||||
|
||||
Two things the checker itself needed before it could run anywhere but the author's
|
||||
machine: every read is `encoding="utf-8"` (the default is the locale codec, and on
|
||||
Windows that is GBK, which cannot decode the Chinese docs at all), and the firmware
|
||||
tree path comes from `UVK5_FW_DIR` rather than being hardcoded, so the `file:line`
|
||||
checks can be pointed at whatever tree you have.
|
||||
|
||||
The flag check earned its own lesson. Its first version matched only to the end of the
|
||||
line, so on a wrapped command like
|
||||
|
||||
@@ -851,3 +1033,90 @@ down and the write offset has to come from the difference.
|
||||
write-1-to-start bits (like `ADC_CR2_CAL`) must never be stored set
|
||||
4. Rebuild, run, and check with `tools/where.sh` that the firmware moved past
|
||||
where it used to stop
|
||||
5. When you add a stub to `py32_stubs[]`, **bump `PY32_NUM_STUB`**. Forgetting used
|
||||
to be silent: the device was never realized, the address stayed unmapped, and the
|
||||
only symptom was that nothing changed. A `QEMU_BUILD_BUG_ON(ARRAY_SIZE(...) !=
|
||||
PY32_NUM_STUB)` next to the table makes it a build error now. Two holes were found
|
||||
that way, both fatal to the multi-system release (see the portability section):
|
||||
`0x40007400` = `DAC1_BASE` and `0x1FFF3000` = `UID_BASE`, neither of which any
|
||||
firmware-visible list mentioned. That is why the whole APB/AHB peripheral space now
|
||||
has a **low-priority catch-all** behind the named devices: an unnamed register
|
||||
answers and logs instead of aborting, and a data abort on real hardware that
|
||||
answers is a model bug, not a discovery.
|
||||
|
||||
## Finding the display buffers in a new firmware
|
||||
|
||||
`gFrameBuffer` and `gStatusLine` move between builds and **neither is 128-byte
|
||||
aligned**, so an aligned guess renders a picture that is wrong in a way that looks
|
||||
like a font or a font-loading problem: 0x3E bytes off and every row becomes "tail of
|
||||
the previous row + head of this one", which splits glyphs at a fixed column and hides
|
||||
the status line behind frame content. Do not eyeball it -- the firmware source says
|
||||
exactly where they are.
|
||||
|
||||
1. Dump SRAM (QMP `memsave`, or `python work/qmp.py dump 0x20000000 0x4000 out.bin`).
|
||||
2. Pick bitmaps whose contents *and* placement are both known from the source
|
||||
(`App/bitmaps.c` with `App/ui/status.c` and `App/driver/st7565.c`):
|
||||
`gFontPowerSave` is copied to status +0, `gFontDWR` to +18, `gFontPttClassic` to
|
||||
+54, `BITMAP_BatteryLevel1` to +111 (`LCD_WIDTH - 17`); `BITMAP_VFO_Default` is
|
||||
`memcpy`'d to offset 0 of a **frame** line.
|
||||
3. Search SRAM for those byte strings. Only one base makes all four status offsets
|
||||
agree at once, and the VFO arrow's address *is* the frame buffer. They must then
|
||||
differ by exactly `FRAME_LINES * LCD_WIDTH` = 896, which is what says the search
|
||||
converged. For the 5.9.0.CN build: frame `0x200012BE`, status `0x2000163E`.
|
||||
|
||||
Self-check once you have them: frame line 3 is the middle separator the UI memsets, so
|
||||
it should be entirely zero; and with both VFOs on one frequency, frame lines 0/1 equal
|
||||
lines 4/5 while lines 2 and 6 differ, because only the active VFO's info line has
|
||||
content.
|
||||
|
||||
## Portability: what Windows actually broke
|
||||
|
||||
The machine and the tools are portable C and Python; the *packaging* was Linux-only.
|
||||
Four failures, each invisible until something depended on it:
|
||||
|
||||
* **`rename()` does not replace an existing file on Windows.** The flash write-back
|
||||
writes a temp file and renames it over the image, so every settings save failed with
|
||||
`cannot replace`, settings never reached disk, and the stderr storm held the main
|
||||
loop long enough that QMP never sent its greeting -- which surfaced only as "power
|
||||
on failed: timed out". `g_rename()` (needs `<glib/gstdio.h>`) gives the POSIX
|
||||
behaviour on both platforms.
|
||||
* **A Windows QEMU cannot create a unix socket**, so QMP has to travel as
|
||||
`tcp:host:port`; `uvk5_qmp.py`, `key.py` and the supervisor's launcher accept both
|
||||
forms now.
|
||||
* **`qemu/py32f071.c` does not compile against a stock QEMU 7.2** without
|
||||
`#include "qapi/visitor.h"` for `visit_type_uint64`; `qom/object.h` does not pull it
|
||||
in transitively.
|
||||
* **`-kernel foo.bin` loads in the wrong place.** `armv7m_load_kernel()` puts a raw
|
||||
binary at the base it is handed, which on this machine is the flash *alias*, so the
|
||||
image lands 0x2800 bytes high and the first fetch faults. `tools/bin2elf.py` wraps
|
||||
the release `.bin` in an ELF32/ARM header with the right program header.
|
||||
|
||||
**The external flash is partitioned, and the main firmware reads it.** Only
|
||||
`0x00A0xx` showed up in a 26 s capture once, which looked like "the firmware does
|
||||
not use the flash at all" -- wrong twice over: the first run was defeated by a
|
||||
PowerShell UTF-16 redirect, the second by capping the probe at 80 reads. With the cap
|
||||
lifted (4000) and a menu opened so Chinese text is drawn, one boot produces 3168
|
||||
reads: the settings block, individual glyphs in the user font packs at `0x0A0000`
|
||||
and `0x0E0000`, and a **1024-step walk of a 32 KB font table at `0x1E0000`**, 32
|
||||
bytes per step. The layout, derived from the tooling at
|
||||
`gitee.com/oldlicn/betula-multi-system-tool` rather than from its partition-map
|
||||
image:
|
||||
|
||||
| offset | size | contents |
|
||||
| --- | --- | --- |
|
||||
| 0x000000 | 128 KB | bootloader + settings (`0x00A0xx`) + calibration (`0x010000`) |
|
||||
| 0x020000 | 4 x 128 KB | firmware slots (the tool ships "clear 0x20000-0x40000" through "0x80000-0xA0000") |
|
||||
| 0x0A0000 | 256 KB | user font pack, 16x16 |
|
||||
| 0x0E0000 | 64 KB | user font pack, 8x8 |
|
||||
| 0x100000 | 1 MB | factory resource block, including the 32 KB table at `0x1E0000` |
|
||||
|
||||
Sixteen official 128 KB restore files reassemble into a real 2 MB image. Adding its
|
||||
`0x100000-0x200000` region to `assets/flash.img` **changes what the firmware
|
||||
renders**, so that data is live, not decoration. Which source supplies which text is
|
||||
still open: the 16-pixel glyphs on screen match neither the pack at `0xA0000` (2 of
|
||||
24 cells) nor the table at `0x1E0000` (0 of 8) byte for byte.
|
||||
|
||||
Panel settings are a fifth, different case: contrast and inversion are not in the
|
||||
framebuffer at all, so nothing that renders `gFrameBuffer` can show them.
|
||||
`TYPE_ST7565` models the controller's own registers and `tools/uvk5_lcd.py` applies
|
||||
the inversion to the picture; see README.md.
|
||||
+209
-3
@@ -65,7 +65,8 @@ bootloader 区域，所以应用在它之后。`armv7m_load_kernel()` 之所以
|
||||
没有校验和 —— 只有一个代码和数据必须约定一致的地址。**所以某个设置读回来不对时，
|
||||
先怀疑偏移，再怀疑传输层。**
|
||||
|
||||
进入主循环要约 15 秒，那是模拟开销。真机大约一秒就起来了。
|
||||
启动耗时是模拟开销。本机实测：QEMU 启动后约 1.6 秒出现第一个像素、约 3.6 秒画出主界面，
|
||||
也就是 README 里写的"约 5 秒"。真机大约一秒就起来了。
|
||||
|
||||
## 基本规则
|
||||
|
||||
@@ -126,6 +127,28 @@ bootloader 区域，所以应用在它之后。`armv7m_load_kernel()` 之所以
|
||||
它的测试：`tools/test_uvk5_*.py` 和 `tools/test_webui.py` 不需要模拟器，
|
||||
`tools/test_webui_e2e.py` 会自己启动一个。
|
||||
|
||||
固件也可以**从页面加载**，不必走命令行：`POST /api/firmware` 把请求体当作镜像，存进
|
||||
`work/firmware/` 并启动它（模拟器正在运行就先重启）。镜像的**形态**是从镜像自己读出来的
|
||||
（主机侧 `tools/uvk5_image.py`，机器侧 `uvk5_sniff_app_offset()`）：*应用镜像*链接在
|
||||
`0x08002800`，*整片镜像*从 `0x08000000` 开始，而地址 0 必须别名到对应的基址。判断错的
|
||||
症状是**静默**的 —— 镜像整体偏 0x2800 字节，第一次取指读到的是随便什么数据 —— 所以这里
|
||||
既不用标志位，也不按文件名约定。不是镜像的文件会被拒绝，且不打扰正在运行的电台。
|
||||
|
||||
这条路上有两件事是踩出来的：
|
||||
|
||||
- **flash 镜像走环境变量，不走 `-M`。** 经启动器启动时，QEMU 会用
|
||||
"unsupported machine type" 拒绝 `-M uv-k5-v3,flash-image=...`：同一份 argv 我手跑就
|
||||
正常，`-M help` 在**同一上下文**里能列出这台机器，argv 的 `repr` 干净，环境变量也
|
||||
逐项比过、没有定论。属性本身仍然可用，所以两条路都保留；启动器现在只传机器名，另用
|
||||
`UVK5_FLASH_IMAGE`，模型把它作为属性缺省的回退。根因未知 —— 没重新验证过"从页面开机"
|
||||
之前，别把它当成冗余"清理"掉。
|
||||
- **画面取自显示控制器，而不是 guest RAM。** 面板模型保存控制器自己的显示 RAM
|
||||
（8 页 × 128 列），页面渲染的是它，所以**任何**固件都显示正确 —— 同一祖先改出的各个版本
|
||||
显示逻辑不同，多系统那版更是把图像放在完全不同的位置。**不要**在数据之上再叠加驱动的
|
||||
`0xA1` 段反转：同一时刻与 guest 自己的帧缓冲对比，不镜像时 8192 像素里对上 8153，
|
||||
镜像后只剩 6557。读 `gFrameBuffer`（`memsave`）的旧路径保留为"没有面板模型的模拟器"
|
||||
的回退。
|
||||
|
||||
## flash 的那些 bug：四个故障，一个症状
|
||||
|
||||
"频率改不了"和"关机后 flash 什么都不记得"看起来是两个抱怨。实际是**一个根因加上路上顺带
|
||||
@@ -211,12 +234,120 @@ bootloader 区域，所以应用在它之后。`armv7m_load_kernel()` 之所以
|
||||
`IDR=0x0000`，因为它的正则**根本不匹配** gdb 的输出格式。寄存器是好的；读取器是坏的。
|
||||
用走另一条路径的 `tools/gpiob_dump.sh` 交叉验证。
|
||||
|
||||
同一个陷阱再往外一层：**重定向会改变编码。** 三次 `qemu ... 2> probe.log` 的探针都报告
|
||||
SPI 零传输、flash 零读取、片选零变化，而"固件根本不碰 SPI"被当成了结论写下来。
|
||||
PowerShell 5.1 的 `2>` 按 UTF-16LE 写文件，于是探针打出的每一行 ASCII 里每个字符之间都夹着
|
||||
NUL，`startswith("LCDW")` 永远匹配不上。把同一个文件按 UTF-16 解码，看到的是完整的 ST7565
|
||||
初始化序列和 48 个不同的设置读取。**在相信一个"什么都没发生"的探针之前，先确认探针能被看见**：
|
||||
读一下文件、数一下字节，或者用不会重新编码的 `cmd /c` 来写。
|
||||
|
||||
**QMP `pmemsave` 是物理地址，`memsave` 是虚拟地址。** 帧缓冲符号是 CPU 虚拟地址，
|
||||
所以对 `gFrameBuffer` 用 `pmemsave` 会返回一整块零**并报告成功** —— 一片空白屏幕，
|
||||
而且哪里都没有日志。网页界面最初就是建在 `pmemsave` 上的，因为一个计时基准说它更快；
|
||||
**那个基准从来没有检查过内容**。要测量你真正在意的东西：这个 bug 是在渲染出的一帧
|
||||
返回 0 个亮像素、而 gdb 路径报告 1693 时才浮出水面的。
|
||||
|
||||
### 页面是 f-string 生成的，所以要检查它真正吐出的脚本
|
||||
|
||||
网页 UI 是一整条 f-string。JavaScript 字符串字面量里多一个或少一个反斜杠，**整段**
|
||||
`<script>` 就无法解析，而唯一的现象是状态栏永远停在 "connecting..."——同时所有接口用
|
||||
`curl` 测又都正常。这真的发布过一次：`.split('\\')` 变成了 `.split('\')`，一个没闭合的
|
||||
字符串，于是从浏览器看页面是死的，而每个测试都通过。
|
||||
|
||||
现在 `test_webui.TestPageScriptParses` 会把服务端真正输出的脚本抽出来交给 `node --check`。
|
||||
**要测你交付出去的那个产物，而不是生成它的代码。**
|
||||
|
||||
### 探针必须真的能看到它在找的东西
|
||||
|
||||
有三轮"固件从不碰 flash"的结论，全是探针的错，而且每一轮看起来都像个发现：
|
||||
|
||||
- 探针按 `address >= 0x0C0000` 过滤，于是所有**没有地址**的帧——写使能、以及真正执行
|
||||
擦除的扇区擦除——都被丢掉了。"0 次写"是过滤器造成的，不是固件。
|
||||
- 握手只给了 1.5 秒，而固件需要约 4 秒才进入串口模式。"没有应答"是超时造成的。
|
||||
- 探针为什么会完全看不见：PowerShell 5.1 的 `2>` 会写成 UTF-16LE，每行字符之间都夹
|
||||
一个 NUL，任何过滤都匹配不上。
|
||||
|
||||
在相信一个空探针之前，先让它打印出一个你确定存在的东西。
|
||||
|
||||
### flash 模型每释放一次片选就把整份镜像写回
|
||||
|
||||
对一次设置保存来说 2 MB 不算什么；但对多系统的主机接口就是灾难——它是按 200 字节一块
|
||||
编程槽位的（`App/app/uart.c` 的 `0x0724`，12 字节头 + 数据）：一份 114 KB 固件会变成约
|
||||
600 次全量重写，而且跑在 vCPU 线程上，于是**客人**和所有跟它对话的主机工具都要等。实测：
|
||||
一次 64 字节的槽位写入花了六秒。
|
||||
|
||||
第一版修法是加 200 毫秒节流，那是错的：它用"强杀时静默丢掉最后一段写入"换来了测试变快。
|
||||
现在模型记录**改动区间**并只写那一段（就地写），既快又更忠实——真机 NOR 的编程本来就不是
|
||||
原子的。退出时的通知器仍然整份写回。
|
||||
|
||||
### 串口链路上跑着固件自己的屏幕流
|
||||
|
||||
`K5Viewer` 会把屏幕从 USART1 推出去。主机客户端如果只在等应答时才读，socket 就会积压，
|
||||
**客人随后会阻塞**在写串口上：槽位传输会在中途开始丢应答，一次很小的写入也要好几秒。
|
||||
正确做法是用一个读线程持续排空，等待逻辑只去看已经重组好的帧——在电台那一侧，任何跟它
|
||||
对话的程序都适用同一条。
|
||||
|
||||
这条路上还有两点：固件的接收缓冲是 256 字节
|
||||
（`App/driver/uart.c: UART_DMA_Buffer[256]`），240 字节数据加上帧头就超了，于是每一帧都被
|
||||
静默丢弃，200 就合适；以及串口**会话**在约 6 秒没有 `0x0514` 后会超时
|
||||
（`gSerialConfigCountDown_500ms = 12`），长传输一定会跨过它——用重新握手实测：写入立刻恢复。
|
||||
|
||||
这四点都修好后，`tools/uvk5_slots_serial.py` 能通过固件本身写入槽位，并由设备校验 CRC。
|
||||
|
||||
### 开机后才连上的串口客户端会错过一切
|
||||
|
||||
`-serial tcp:host:port,server=on,wait=off` 在**没有客户端连接时会丢弃**客人写出的字节。
|
||||
固件的横幅是在最初几秒打印的，所以"等 QEMU 起来再连"（比如四秒后）会看到一个空端口——
|
||||
和"客人根本没启动"长得一模一样。有四轮"引导什么也不发"其实是这个原因，不是引导。
|
||||
|
||||
先连接，再让客人跑。同一条陷阱也适用于引导等待主机时持续发出的 0x0518 洪水：它是持续的，所以
|
||||
晚连的客户端**也能看到**——这正是这个错误能存活这么久的原因，它只对一次性启动输出显形。
|
||||
|
||||
另外两个在这里学到的习惯：
|
||||
|
||||
- **先确认探针能看到一个你确定存在的东西。** 一个 USART 寄存器探针报告零次访问，最自然的读法
|
||||
是"引导从不配置串口"。而用同一个探针跑应用，得到 2239 次——这才说明探针是好的、引导确实是沉默的。
|
||||
- **当一条记录下来的观察不再能复现时，把它当成未验证。** 引导的 Moto 洪水是从一次现在已无法用
|
||||
当前构建与镜像复现的运行里写下来的；要依赖它之前先重新推导。
|
||||
|
||||
### 裸的 host:port 不是"scheme"
|
||||
|
||||
`uvk5_socket.connect` 会按 ":" 切分参数来找 scheme，于是监管进程、网页和 README 一直传的
|
||||
`127.0.0.1:4444` 被读成：scheme 是 `127.0.0.1`、端口为空、主机为空。连接就一直等到超时。
|
||||
从外面看到的现象是"网页开不了机"，而用完全相同命令行手敲的 QEMU 半秒就应答 QMP，客人则在后台
|
||||
好好跑着。
|
||||
|
||||
两件事让它难被发现：同一个助手也接受 `tcp:host:port`，所以用那种写法的测试都是通的；而失败
|
||||
形式是**超时**而不是报错，看起来像"模拟器很慢或卡住"。现在 `test_uvk5_socket` 覆盖了所有会走到
|
||||
`connect` 的写法。推广一句：**当一个助手接受多种写法时，每一种都要测**——没人测的那一种，往往
|
||||
就是所有人都在用的那一种。
|
||||
|
||||
同一处的另一半是**另一个独立真问题**：QEMU 的 stderr 必须从启动那一刻就被排空。固件把屏幕流从
|
||||
那根管道推出去，约一秒就能填满 64 KB，QEMU 写不进去就阻塞——主循环停了，QMP 自然也不应答。
|
||||
两种做法都量过：排空时 QMP 0.5 秒应答；不排空时永远不应答。
|
||||
|
||||
### 你吞掉的寄存器，就是下一个程序死等的东西
|
||||
|
||||
出厂引导**完全起不来**：没有任何串口输出，PC 探针每一次采样都停在 `0x08000f38`。那地址在引导
|
||||
区里，两条指令是：
|
||||
|
||||
0x0f38: ldr r2, [r1] ; r1 = 0x40022000，FLASH 控制器
|
||||
0x0f3a: lsls r2, r2, #30
|
||||
0x0f3c: lsrs r2, r2, #30 ; r2 = ACR & 3，就是 LATENCY 字段
|
||||
0x0f3e: cmp r2, #1 ; 等 1 个等待周期
|
||||
0x0f40: bne 0x0f38
|
||||
|
||||
早前加进来的 FLASH 控制器模型把 `ACR` 与 `OPTKEYR` 当成"可忽略的写"——直接提前 return，
|
||||
于是通用路径从未保存它们，`ACR` 永远回读 0，引导在配置串口之前就死等。改成 `return false`
|
||||
让数值被保存后，PC 立刻进入应用区（`0x08013ea0`），串口也开始有输出。
|
||||
|
||||
两条都通用：
|
||||
|
||||
- **只写寄存器也是寄存器。** 应用从不回读 `ACR`，所以只要只有应用在跑，这个缺陷就是隐形的；
|
||||
下一个碰同一个外设的程序立刻撞上。
|
||||
- **"以前是好的"就是一条二分指令。** 引导的 Moto 洪水在 FLASH 控制器建模**之前**被观察到，
|
||||
之后就不再复现。正确的动作是问"这两次之间改了什么"，而不是怀疑早先的记录。
|
||||
|
||||
## 键盘：两个真 bug，都已修复
|
||||
|
||||
这里原来的笔记写的是"按键到达了固件但界面不反应"，并且归咎于机器模型。结果发现有**两个
|
||||
@@ -349,8 +480,12 @@ GCC 能看到全部调用者。如果一个模型的输出神秘地不起作用
|
||||
`SETTINGS_InitEEPROM` 会比较 flash `0x00A160` 处的版本字符串，在一个新镜像上发现不匹配，
|
||||
于是写入设置扇区。而 `PY25Q16_WriteBuffer` 会在重新编程之前**擦除整个 4 KB 扇区**，
|
||||
所以埋在 `0x00A00B` 的字节在 `settings.c:169` 的读取看到它之前就已经没了。
|
||||
- **guest 侧的设置改动不会持久化。**（历史条目：现在 flash 会写回文件了，
|
||||
见 flash 章节的第 1 条修复。）
|
||||
- **guest 侧的设置改动现在会持久化，所以测试方式要变。** PY25Q16 模型在 realize 时读入镜像、
|
||||
留在 RAM 里，并在片选释放或进程退出时经临时文件写回，于是**每个会话都会改动
|
||||
`assets/flash.img`**。实测跑完一次真实会话之后：本来全是 `0xFF` 的设置区 `0x00A000`
|
||||
已经装着 guest 的设置，`0x8000..0x8800` 也变了，整个文件与开会话前的副本相差 2239 字节。
|
||||
所以不要假设镜像是干净的 —— 要和 `assets/pristine/`（或你自己留的副本）对比，还原前先断电。
|
||||
在 Windows 上这一步直到把 `rename()` 换成 `g_rename()` 之前都是静默失败的，见可移植性一节。
|
||||
|
||||
这里有用的工具：`tools/scan_trace.sh`（扫描读到了什么）、`tools/key_result.sh`
|
||||
（Poll 返回了什么）、`tools/trace_run.sh`（那些 TRACE 点）。
|
||||
@@ -427,6 +562,10 @@ GCC 能看到全部调用者。如果一个模型的输出神秘地不起作用
|
||||
文档传给某个工具的每个长参数是否真的存在于该工具中、
|
||||
以及文档里的固件 `file:line` 引用是否仍指向正文声称的东西。
|
||||
|
||||
校验器自己也需要两处改动才能在作者的机器之外运行：所有读取都显式用 `encoding="utf-8"`
|
||||
（默认是区域编码，Windows 上是 GBK，中文文档根本解不开），以及固件树路径来自 `UVK5_FW_DIR`
|
||||
环境变量而不是写死，这样 `file:line` 那几项检查可以指向你手上任何一份固件树。
|
||||
|
||||
参数检查本身也留下了一个教训。它的第一版只匹配到行尾，所以对一条这样换行的命令
|
||||
|
||||
python3 tools/screenshot.py --frame-addr 0x200013DC \
|
||||
@@ -732,3 +871,70 @@ guest 照样在跑，但事后 `REG_0C` bit 0 仍然是置位的：固件没有
|
||||
3. 警惕自旋循环：任何固件会轮询的标志都必须**能够变化**，
|
||||
而写 1 启动的位（比如 `ADC_CR2_CAL`）**绝不能被存成置位状态**
|
||||
4. 重新构建、运行，并用 `tools/where.sh` 确认固件越过了它原来停住的地方
|
||||
5. 往 `py32_stubs[]` 里加条目时**必须同时改 `PY32_NUM_STUB`**。忘了改以前是**静默**的：
|
||||
那个设备根本不会被 realize，地址仍是未映射，唯一症状就是"什么都没变"。现在表后面有一条
|
||||
`QEMU_BUILD_BUG_ON(ARRAY_SIZE(...) != PY32_NUM_STUB)`，会在编译期拦住。
|
||||
这样找到过两个洞，都对多系统固件是致命的（见可移植性一节）：`0x40007400` = `DAC1_BASE`、
|
||||
`0x1FFF3000` = `UID_BASE`，两者都不在任何面向固件的清单里。所以现在整个 APB/AHB 外设空间
|
||||
在具名设备之后还有一层**低优先级兜底**：没名字的寄存器会应答并记日志，而不是抛异常 ——
|
||||
真机有应答、模拟器却数据异常，那是模型的 bug，不是新发现。
|
||||
|
||||
## 在新固件里找显示缓冲区的地址
|
||||
|
||||
`gFrameBuffer` 和 `gStatusLine` 在不同构建之间会移动，而且**两者都不是 128 字节对齐的**，
|
||||
所以按对齐地址去猜，渲染出来的图会以"像字体问题"的方式出错：偏 0x3E 字节时，每一行都变成
|
||||
"上一行的尾部 + 本行的头部"，字形会被固定在某一列劈开，状态行也被帧内容盖住。别用眼睛判断
|
||||
—— 固件源码会明确告诉你它们在哪。
|
||||
|
||||
1. 抓 SRAM（QMP `memsave`，或 `python work/qmp.py dump 0x20000000 0x4000 out.bin`）。
|
||||
2. 挑"内容和落点都能从源码确定"的位图（`App/bitmaps.c` 配合 `App/ui/status.c`、
|
||||
`App/driver/st7565.c`）：`gFontPowerSave` 复制到状态行 +0、`gFontDWR` +18、
|
||||
`gFontPttClassic` +54、`BITMAP_BatteryLevel1` +111（`LCD_WIDTH - 17`）；
|
||||
`BITMAP_VFO_Default` 由 `memcpy` 写到**帧行**偏移 0。
|
||||
3. 在 SRAM 里搜这些字节串。只有唯一一个基址能让那四个状态行偏移同时成立，而 VFO 箭头的地址
|
||||
**就是**帧缓冲基址。两者必须正好相差 `FRAME_LINES * LCD_WIDTH` = 896 —— 这就是"搜索收敛了"
|
||||
的判据。5.9.0.CN 这份固件：帧缓冲 `0x200012BE`，状态行 `0x2000163E`。
|
||||
|
||||
拿到之后的自检：帧行 3 是界面 `memset` 掉的中缝，应当整行全 0；两个 VFO 同频时，
|
||||
帧行 0/1 与 4/5 相同，而 2 与 6 不同（只有活动 VFO 的说明行有内容）。
|
||||
|
||||
## 可移植性：Windows 到底弄坏了什么
|
||||
|
||||
机器和工具本身是可移植的 C 与 Python；**包装**是 Linux 专用的。四个失败，每一个在有人依赖它
|
||||
之前都是不可见的：
|
||||
|
||||
* **`rename()` 在 Windows 上不覆盖已存在的文件。** flash 回写是"写临时文件再改名覆盖"，
|
||||
于是每一次保存设置都失败（`cannot replace`），设置永远到不了磁盘，而 stderr 刷屏把主循环
|
||||
挤住到 QMP 连问候语都发不出来 —— 最终只表现为 "power on failed: timed out"。
|
||||
`g_rename()`（需要 `<glib/gstdio.h>`）在两个平台上都给出 POSIX 语义。
|
||||
* **Windows 版 QEMU 无法创建 unix socket**，所以 QMP 必须以 `tcp:host:port` 传输；
|
||||
`uvk5_qmp.py`、`key.py` 和 supervisor 的启动器现在两种形式都接受。
|
||||
* **`qemu/py32f071.c` 对原版 QEMU 7.2 编译不过**，缺 `#include "qapi/visitor.h"`
|
||||
（`visit_type_uint64`）；`qom/object.h` 不会间接带入它。
|
||||
* **`-kernel foo.bin` 会加载到错的地方。** `armv7m_load_kernel()` 把裸二进制加载到给它的基址，
|
||||
而在本机那个基址是 flash 的**别名区**，于是镜像高 0x2800 字节，第一次取指就 fault。
|
||||
`tools/bin2elf.py` 把发行版 `.bin` 套上带正确程序头的 ELF32/ARM。
|
||||
|
||||
**外置 flash 是有分区的，而且主固件确实在读它。** 有一次 26 秒的抓取里只看到 `0x00A0xx`，
|
||||
于是"固件根本不用 flash"被当成结论写下来 —— 错了两次：第一次被 PowerShell 的 UTF-16 重定向
|
||||
骗了，第二次是我把探针**截断在 80 次读**。把上限放到 4000、并打开菜单让固件画汉字之后，
|
||||
一次启动就产生 3168 次读：设置区、`0x0A0000`/`0x0E0000` 用户字体包里的单个字模，以及
|
||||
**`0x1E0000` 处一张 32 KB 字体表被按 32 字节一步连续走完 1024 次**。分区如下（由
|
||||
`gitee.com/oldlicn/betula-multi-system-tool` 的配套数据反推，不是靠它那张分区图）：
|
||||
|
||||
| 偏移 | 大小 | 内容 |
|
||||
| --- | --- | --- |
|
||||
| 0x000000 | 128 KB | 引导 + 设置（`0x00A0xx`）+ 校准（`0x010000`） |
|
||||
| 0x020000 | 4 × 128 KB | 固件槽（工具里正好有"清空 0x20000-0x40000"到"0x80000-0xA0000"） |
|
||||
| 0x0A0000 | 256 KB | 用户字体包 16x16 |
|
||||
| 0x0E0000 | 64 KB | 用户字体包 8x8 |
|
||||
| 0x100000 | 1 MB | 出厂资源区，其中 `0x1E0000` 是那张 32 KB 字体表 |
|
||||
|
||||
官方十六份 128 KB 恢复文件可以拼回真机整片 2 MB 数据。把它的 `0x100000-0x200000` 区补进
|
||||
`assets/flash.img` 之后，**固件画出来的内容会变** —— 说明那块数据是活的，不是装饰。
|
||||
还没定论的是"哪块文字由哪个来源提供"：屏幕上的 16 像素字模与 `0xA0000` 的字模包
|
||||
（24 格里对 2 格）、与 `0x1E0000` 的表（8 格里 0 格）都不能逐字节对上。
|
||||
|
||||
面板设置是第五种、不一样的情况：对比度和反显根本不在帧缓冲里，所以任何渲染 `gFrameBuffer`
|
||||
的东西都显示不出来。`TYPE_ST7565` 建模控制器自己的寄存器，`tools/uvk5_lcd.py` 把反显应用到
|
||||
画面上；见 README.md。
|
||||
@@ -0,0 +1,65 @@
|
||||
# Contributing
|
||||
|
||||
Short version: build it, run the tests, and do not commit firmware or radio data.
|
||||
|
||||
## Get it running
|
||||
|
||||
QEMU_SRC=~/src/qemu-7.2 bash tools/setup_qemu.sh # patch a QEMU tree and build it
|
||||
python3 tools/fetch_firmware.py # a release image to run
|
||||
python3 tools/make_flash.py # the flash image it reads settings from
|
||||
bash tools/run_tests.sh -q # fast; no emulator needed
|
||||
bash tools/run_tests.sh # everything; needs the tree above
|
||||
|
||||
`tools/run_tests.sh` checks the build first and stops if it failed. That matters: `ninja`
|
||||
leaves the previous binary in place, so a suite run against a broken build reports
|
||||
results for code that was never compiled. It has happened here twice.
|
||||
|
||||
The emulator tests boot their own QEMU on private ports, so they do not disturb a
|
||||
`run-webui.ps1` or `run.sh` session. A missing QEMU, firmware or gdb makes a test
|
||||
**skip** with a message, never fail -- a missing prerequisite is not a regression, and
|
||||
a suite that fails on a fresh checkout teaches people to ignore it.
|
||||
|
||||
Two helpers keep that honest: `tools/uvk5_socket.py` gives every test an endpoint that
|
||||
works where the platform has unix sockets and TCP where it does not, and
|
||||
`tools/uvk5_testenv.py` finds a QEMU, a firmware and a gdb, skipping with a reason when
|
||||
one is absent. Use them rather than hardcoding a path or a socket family.
|
||||
|
||||
## What not to commit
|
||||
|
||||
- **Firmware of any kind**, including released images, localised builds and bootloader
|
||||
dumps. `tools/fetch_firmware.py` fetches what a test needs into `assets/firmware/`,
|
||||
which is ignored.
|
||||
- **Anything from a real radio.** `work/data.bin` is an EEPROM dump with settings and
|
||||
calibration; the tests build their own images from `assets/pristine/`.
|
||||
- **Scratch under `work/`** -- images, logs, captures. The four `.ps1` scripts there are
|
||||
tracked on purpose.
|
||||
|
||||
If you find any of that already in the history, say so before pushing: removing it in a
|
||||
new commit does not remove the objects.
|
||||
|
||||
## House rules
|
||||
|
||||
These come from mistakes that already cost time, and [AGENTS.md](AGENTS.md) has the long
|
||||
version of each. The ones worth repeating:
|
||||
|
||||
- **Never edit the firmware to make the emulator work.** The firmware is the reference;
|
||||
if something does not run, the model is wrong. A fix in firmware source makes every
|
||||
later test meaningless.
|
||||
- **Instrument the model, not the guest.** A breakpoint stops the machine and changes what
|
||||
you are measuring. Put a probe in `qemu/py32f071.c` and read the output.
|
||||
- **A probe has to be able to see what it looks for.** Three rounds of "the firmware never
|
||||
touches the flash" were a probe filtering on an address field that write-enable and
|
||||
erase frames do not have.
|
||||
- **Check the build succeeded before believing a test.** See above.
|
||||
- **Document what you got wrong**, in `AGENTS.md` and `AGENTS.zh-CN.md` together. The two
|
||||
are kept in step; `tools/check_docs.py` checks the heading structure and the tool names.
|
||||
- **Never invent data.** Audio and RF are not modelled because the MCU never sees them,
|
||||
not because nobody got round to it.
|
||||
|
||||
## Documentation
|
||||
|
||||
Every document has a Chinese pair (`README.md` / `README.zh-CN.md`, `AGENTS.md` /
|
||||
`AGENTS.zh-CN.md`) with the same heading structure. `python3 tools/check_docs.py` checks
|
||||
that the tools and flags a document names exist, that the heading pairs match, that the
|
||||
memory-map addresses match the model, and that firmware `file:line` references still point
|
||||
at what the prose claims. It runs as part of `run_tests.sh -q`.
|
||||
@@ -38,6 +38,7 @@ has no public datasheet, so its driver is the only specification available.
|
||||
| --- | --- |
|
||||
| Boot to main loop | works, ~5 s |
|
||||
| LCD contents | readable via `tools/screenshot.py` |
|
||||
| Display contrast / inversion | panel settings, read from the controller; inversion also changes the rendered picture |
|
||||
| SPI flash, settings, calibration | works, and persists across power cycles |
|
||||
| Frequency entry | works, stored per band and kept |
|
||||
| Keypad and menu navigation | works, including waking from power save |
|
||||
@@ -83,6 +84,9 @@ keypresses silently stop working. Run the test after touching that code;
|
||||
*.zh-CN.md Chinese translations, kept in step
|
||||
docs/screenshots/ LCD captures used in this README
|
||||
tools/ run, screenshot, inject keys, probe state
|
||||
bin2elf.py wrap a release .bin so QEMU can load it as a kernel
|
||||
make_flash.py build assets/flash.img; --blob puts extra data (the
|
||||
font packs a Chinese build needs) at chosen offsets
|
||||
keypad_test.py keypad regression test, boots its own instance
|
||||
test_flash_persist.py flash writes survive a power cycle
|
||||
test_freq_entry.py a typed frequency takes effect and persists
|
||||
@@ -114,6 +118,66 @@ keypresses silently stop working. Run the test after touching that code;
|
||||
kept because they are quick to reach for, not because they are polished)
|
||||
harness/, stubs/, shim/, tests/ host build of the CW timing chain (stage A)
|
||||
|
||||
## What is not in this repository
|
||||
|
||||
Two things are deliberately absent, and neither should be committed:
|
||||
|
||||
- **Firmware.** Released images, localised builds and bootloader dumps belong to whoever
|
||||
made them, not to this project. `tools/fetch_firmware.py` fetches a release from the
|
||||
upstream archive into `assets/firmware/` when a test or a run needs one, and that
|
||||
directory is ignored.
|
||||
- **Anything read off a real radio.** `work/data.bin` is an EEPROM dump -- settings and
|
||||
calibration from somebody's hardware. It is not a build artifact. It is ignored now, and
|
||||
the tests build their own flash images from `assets/pristine/` instead.
|
||||
|
||||
`assets/pristine/flash-pristine.img.gz` and `assets/calibration.bin` do ship: they are a
|
||||
2 KB synthetic pair that `tools/make_flash.py` assembles into a flash image, not data
|
||||
from a radio.
|
||||
|
||||
The rest of `work/` is scratch -- images, logs, captures from a debugging session. The
|
||||
four scripts in it are tracked on purpose, because they document how this machine is
|
||||
driven; everything else is ignored.
|
||||
|
||||
**If any of that is already in the history, removing it now is not enough.** The objects
|
||||
stay reachable, so publishing this repository needs the history filtered
|
||||
(`git filter-repo`) or a fresh one. Check before pushing:
|
||||
|
||||
git log --stat -- work/data.bin assets/firmware
|
||||
|
||||
## Quick start
|
||||
|
||||
Five minutes from a checkout to a running radio on a web page.
|
||||
|
||||
# 1. A QEMU 7.2 tree, patched and built. Building by hand means copying three files
|
||||
# into the tree, editing Kconfig and meson.build, then configure and ninja; this
|
||||
# is those same steps (see Building below for what it does).
|
||||
QEMU_SRC=~/src/qemu-7.2 bash tools/setup_qemu.sh
|
||||
|
||||
# 2. A firmware to run. Firmware is not redistributed here -- this fetches a release
|
||||
# from the upstream project's archive into assets/firmware/ and prints its hash.
|
||||
python3 tools/fetch_firmware.py
|
||||
|
||||
# 3. The external flash image the firmware keeps its settings in.
|
||||
python3 tools/make_flash.py
|
||||
|
||||
# 4. Run it.
|
||||
python3 tools/webui.py --qemu ~/src/qemu-7.2/build/qemu-system-arm \
|
||||
--elf assets/firmware/f4hwn.fieldops.v6.0.0.bin # then open http://127.0.0.1:8080/
|
||||
|
||||
`--frame-addr` and `--status-addr` default to one known build and move between builds;
|
||||
see [Web remote control](#web-remote-control) for how to find them. On Windows,
|
||||
`work/run-webui.ps1` wraps step 4 with this machine's paths.
|
||||
|
||||
Drop any `.bin` on the page to boot it. The page's **Firmware slots** table reads and
|
||||
writes the multi-system firmware's four slots in the flash image, and **Multiboot**
|
||||
restarts holding MENU so its boot menu comes up -- in a build that has one: the page
|
||||
labels builds that do not, because there the button can do nothing at all.
|
||||
|
||||
Then check it still works:
|
||||
|
||||
bash tools/run_tests.sh -q # ~15 s, no emulator
|
||||
bash tools/run_tests.sh # everything; needs the tree from step 1
|
||||
|
||||
## Building
|
||||
|
||||
Needs a QEMU 7.2 source tree, `meson`, `ninja`, `libfdt-dev`, `libglib2.0-dev`,
|
||||
@@ -155,6 +219,7 @@ that was never compiled. Individual tests still run standalone:
|
||||
python3 tools/test_flash_persist.py
|
||||
python3 tools/test_freq_entry.py
|
||||
python3 tools/test_serial_rx.py
|
||||
python3 tools/test_slot_serial.py
|
||||
python3 tools/test_bk4819.py
|
||||
bash tools/test_bk4819_readback.sh
|
||||
python3 tools/test_smeter.py
|
||||
@@ -241,10 +306,22 @@ Endpoints, if you want to script it:
|
||||
| `POST /api/release-all` | release every key and PTT, if one ever sticks |
|
||||
| `POST /api/power/<action>` | `on`, `off`, `reset`, `pause`, `resume` |
|
||||
| `GET /api/logs?since=N` | log entries after cursor N, with client IPs |
|
||||
| `GET /api/status` | QMP `query-status`, plus a `speaker` field |
|
||||
| `GET /api/status` | QMP `query-status`, plus `speaker`, `panel` and `firmware` |
|
||||
| `GET /api/firmware` | the loaded image, and how it will be loaded |
|
||||
| `POST /api/firmware` | body is a `.bin` or `.elf`; boots it and restarts the emulator |
|
||||
| `GET /api/slots` | the firmware slots in the flash image the emulator uses |
|
||||
| `POST /api/slots/<n>` | body is a `.bin`; writes it into slot `n` and restarts |
|
||||
| `POST /api/slots/<n>/erase` | erase slot `n` |
|
||||
| `POST /api/flash` | body is a flash image; use it from now on |
|
||||
|
||||
Frames are read with QMP `memsave`, about 1.35 ms each, and the guest keeps
|
||||
running throughout. Two details there are easy to get wrong:
|
||||
Frames now come from the display controller's own memory: a QMP `qom-get` on the
|
||||
panel's `gram` property, so the picture is right whoever wrote the firmware and
|
||||
wherever it keeps its buffers. Builds that share an ancestor still differ in their
|
||||
display logic -- the multi-system release keeps its image somewhere else entirely.
|
||||
|
||||
The older path reads `gFrameBuffer` and `gStatusLine` out of guest RAM with QMP
|
||||
`memsave` (~1.35 ms per frame) and remains as the fallback for an emulator built
|
||||
without the panel model. The two cautions below apply to that path:
|
||||
|
||||
- **`memsave`, not `pmemsave`.** The framebuffer symbols are CPU virtual
|
||||
addresses. `pmemsave` treats its argument as physical and returns a block of
|
||||
@@ -260,6 +337,64 @@ Two constraints worth knowing before you use it:
|
||||
- **There is no authentication.** Anyone who reaches the port has full control of
|
||||
the emulated radio. It binds loopback by default for that reason.
|
||||
|
||||
### Loading firmware from the page
|
||||
|
||||
Drop a `.bin` anywhere on the page, or use the Firmware control, and the server
|
||||
stores it and boots it. No ELF wrapping, no address to look up.
|
||||
|
||||
Two shapes exist and they need different load addresses:
|
||||
|
||||
| shape | how it is recognised | load address |
|
||||
| --- | --- | --- |
|
||||
| application | reset handler past `0x08002800` | `0x08002800` |
|
||||
| full-flash | reset handler inside the bootloader region (`0x08000000`..`0x080027ff`) | `0x08000000` |
|
||||
|
||||
An `.elf` carries its own program headers and needs neither. This is read out of the
|
||||
image's first two words -- `tools/uvk5_image.py` on the host, `uvk5_sniff_app_offset()`
|
||||
in the machine -- rather than taken from a flag or a file name, because getting it
|
||||
wrong is silent: the image lands `0x2800` bytes off and the first fetch reads whatever
|
||||
data is there. A file that is not a bootable image is refused with 400 and the radio
|
||||
keeps running what it had.
|
||||
|
||||
Uploads live in `work/firmware/` (`UVK5_UPLOAD_DIR` moves it), and while the emulator
|
||||
is running an upload restarts it, since the image is chosen when QEMU is spawned.
|
||||
|
||||
Both shapes are verified end to end here: an application `.bin`, an `.elf`, and a
|
||||
full-flash image whose bootloader entry branches to the application at `0x08002800`
|
||||
all reach the same drawn screen.
|
||||
|
||||
### Firmware slots, and the multi-system release
|
||||
|
||||
The v6.0.0 release keeps a boot menu and four firmware slots in the external flash: hold
|
||||
MENU at power-on and it lists them, and choosing one reflashes the internal flash from that
|
||||
slot and resets. Both halves are reachable from the page.
|
||||
|
||||
- **Firmware slots** shows one row per slot with the name, version, size and whether the
|
||||
header CRC-32 matches the image. Write a `.bin` into a slot, or erase one. Edits go to a
|
||||
working copy of the flash image (`work/firmware/flash-current.img`), never to the file the
|
||||
server was started with, and the emulator is restarted to pick them up.
|
||||
- **Multiboot** (or Shift+M) restarts the emulator with MENU held *from reset*. The page
|
||||
cannot do that with key events, because the firmware samples the keypad in the first
|
||||
milliseconds after reset. On the machine it is `-M uv-k5-v3,boot-key=MENU` or
|
||||
`UVK5_BOOT_KEY`, held for `UVK5_BOOT_KEY_MS` (8 s by default: the boot path can spend
|
||||
20 s adopting the running firmware into slot 0 before anything samples the keypad).
|
||||
- `tools/uvk5_slots.py` does the same offline: write a slot into a flash image, and print
|
||||
what each slot holds.
|
||||
|
||||
The layout is the firmware's, from `App/driver/mb_flash.h`: slot 0 at `0x020000` backs up
|
||||
the internal image, slots 1..4 follow at `0x040000` in 128 KiB steps, the image starts one
|
||||
4 KiB sector into the slot, and the 64-byte header carries magic `FMB1`, the image size and
|
||||
a CRC-32. The firmware's own `0x0720`..`0x0727` serial commands write slots the same way,
|
||||
which is what the Windows tools use.
|
||||
|
||||
Two behaviours look like the emulator misbehaving and are not. A **corrupt** active-state
|
||||
marker next to a valid slot 0 makes the firmware halt on a `STATE ERROR` screen to protect
|
||||
Main, and a **missing** marker makes it adopt the running firmware into slot 0 -- reflashing
|
||||
the external flash -- before the menu appears. Writing a slot erases those marker sectors so
|
||||
it can decide again. The internal flash is programmable in the model (`0x40022000`:
|
||||
unlock, page erase, program, EOP, never busy), so restoring a slot really does replace the
|
||||
image the CPU executes after the reset.
|
||||
|
||||
### Reaching it from elsewhere
|
||||
|
||||
The deployment here runs the server on loopback and puts nginx in front of it for
|
||||
@@ -336,7 +471,8 @@ Register layouts come from the vendor CMSIS header shipped with the firmware
|
||||
SPI2 0x40003800 flash
|
||||
ADC1 0x40012400
|
||||
|
||||
Modelled: RCC, GPIO, ADC, both SPI controllers, DMA1, TIM2, and the PY25Q16 flash.
|
||||
Modelled: RCC, GPIO, ADC, both SPI controllers, DMA1, TIM2, the PY25Q16 flash, and
|
||||
the ST7565 display controller's own settings (contrast, inversion, display on/off).
|
||||
Everything else answers through a logging catch-all — the log is how the next
|
||||
thing worth modelling gets identified.
|
||||
|
||||
@@ -405,6 +541,48 @@ on a host would let the tests drift from what the radio runs. The debounce in
|
||||
consecutive reads to register a press, immediate release) is part of the timing
|
||||
behaviour under test.
|
||||
|
||||
## The display controller's own settings
|
||||
|
||||
Contrast (`SetCtr`) and display inversion (`SetInv`) are commands to the ST7565, not
|
||||
framebuffer content — `0x81 <value>` and `0xA6`/`0xA7` — so `gFrameBuffer` does not
|
||||
change and anything that renders that buffer shows no effect at all. That is why there
|
||||
is a small `TYPE_ST7565` behind SPI1 (A0 on PA6, CS on PB2, the pins
|
||||
`App/driver/st7565.c` uses). It parses the command stream and exposes three
|
||||
read-only properties:
|
||||
|
||||
qom-get /machine/panel invert # last of 0xA6 / 0xA7
|
||||
qom-get /machine/panel contrast # the value following 0x81
|
||||
qom-get /machine/panel display-on # last of 0xAE / 0xAF
|
||||
|
||||
`tools/uvk5_lcd.py` applies the inversion when it renders, because that effect is
|
||||
fully determined, so the menu entry is visible in the web UI. Contrast is analogue —
|
||||
how dark the glass gets — and is only reported. `/api/status` carries all three and
|
||||
the page shows them beside the speaker glyph. `display-on` is reported but not acted
|
||||
on: whether a software reset (`0xE2`) clears that latch is not certain, and blanking
|
||||
the screen on a guess would be worse than leaving the image alone.
|
||||
|
||||
## On Windows
|
||||
|
||||
The emulator, the models and the tools are portable; the packaging was not. Four
|
||||
things differ, and all four are handled in-tree now:
|
||||
|
||||
- **QMP over TCP.** A Windows build of QEMU cannot create a unix socket, so an
|
||||
endpoint may be `host:port` as well as a path — `tools/uvk5_qmp.py`,
|
||||
`tools/key.py` and `tools/uvk5_supervisor.py` all accept both.
|
||||
- **One build fix.** MSYS2's mingw-w64 packages build QEMU 7.2 as-is except that
|
||||
`qemu/py32f071.c` needs `#include "qapi/visitor.h"` for `visit_type_uint64`, which
|
||||
a stock tree does not pull in transitively.
|
||||
- **A release `.bin` is not a kernel image.** `armv7m_load_kernel()` loads a raw
|
||||
binary at the address it is handed, which here is the flash *alias*, so a `.bin`
|
||||
lands 0x2800 bytes too high and never boots. `tools/bin2elf.py` wraps it in an
|
||||
ELF32/ARM header with the right program header, which is what `-kernel` wants.
|
||||
- **The Chinese font packs live in the SPI flash**, not in the firmware:
|
||||
`tools/make_flash.py --blob 0:pack.uf2` places every UF2 block at its own target
|
||||
address. Without that the font area reads as 0xFF.
|
||||
|
||||
`work/` holds a Windows bring-up record: the launcher scripts, the frame addresses
|
||||
proven against the firmware source, and the failures that cost time.
|
||||
|
||||
## Licence
|
||||
|
||||
Apache 2.0, see [LICENSE](LICENSE).
|
||||
|
||||
+154
-3
@@ -32,6 +32,7 @@ VFO 重算了状态。在真机上你只看到"什么都没发生"，在这里
|
||||
| --- | --- |
|
||||
| 启动到主循环 | 可用，约 5 秒 |
|
||||
| LCD 内容 | 可用，经 `tools/screenshot.py` |
|
||||
| 显示对比度 / 反显 | 面板级设置，从控制器读取；反显还会改变渲染出的画面 |
|
||||
| SPI flash、设置、校准数据 | 可用，且断电保留 |
|
||||
| 频率输入 | 可用，按波段分别存储并保留 |
|
||||
| 键盘与菜单导航 | 可用，含从省电模式唤醒 |
|
||||
@@ -71,6 +72,9 @@ VFO 重算了状态。在真机上你只看到"什么都没发生"，在这里
|
||||
*.zh-CN.md 中文翻译，与英文版同步维护
|
||||
docs/screenshots/ 本 README 用到的 LCD 截图
|
||||
tools/ 运行、截图、注入按键、探查状态
|
||||
bin2elf.py 把发行版 .bin 包成 QEMU 能当内核加载的 ELF
|
||||
make_flash.py 生成 assets/flash.img；--blob 可把额外数据（中文版
|
||||
需要的字体包）放到指定偏移
|
||||
keypad_test.py 键盘回归测试，自己启动实例
|
||||
test_flash_persist.py flash 写入能跨断电保留
|
||||
test_freq_entry.py 输入的频率生效并保留
|
||||
@@ -102,6 +106,57 @@ VFO 重算了状态。在真机上你只看到"什么都没发生"，在这里
|
||||
留着是因为随手就能用，不是因为它们打磨过）
|
||||
harness/, stubs/, shim/, tests/ CW 时序链的宿主机构建（阶段 A）
|
||||
|
||||
## 仓库里没有什么
|
||||
|
||||
有两类东西是刻意不放的，也都不应该提交：
|
||||
|
||||
- **固件**：发行镜像、汉化/改版构建、引导 dump 都属于它们的作者，不属于这个项目。
|
||||
需要时用 `tools/fetch_firmware.py` 从上游归档取一份到 `assets/firmware/`，该目录已被忽略。
|
||||
- **任何从真电台读出来的数据**：`work/data.bin` 是一份 EEPROM dump——别人机器上的设置与校准。
|
||||
它不是构建产物。它现在已被忽略，测试改用 `assets/pristine/` 自己拼 flash 镜像。
|
||||
|
||||
`assets/pristine/flash-pristine.img.gz` 与 `assets/calibration.bin` 是会随仓库分发的：它们是
|
||||
一对 2 KB 的**合成**数据，由 `tools/make_flash.py` 拼成 flash 镜像，不是电台里的数据。
|
||||
|
||||
`work/` 里其余都是临时产物——镜像、日志、抓取结果。里面那四个脚本是**刻意跟踪**的，因为它们
|
||||
记录了这台机器怎么驱动；其他内容一律忽略。
|
||||
|
||||
**如果这些东西已经在历史里了，现在删掉并不够。** 对象仍然可达，所以要把仓库公开就得先清理历史
|
||||
（`git filter-repo`）或另起一个仓库。推送前先查一下：
|
||||
|
||||
git log --stat -- work/data.bin assets/firmware
|
||||
|
||||
## 快速开始
|
||||
|
||||
从克隆到网页上跑起电台，大约五分钟。
|
||||
|
||||
# 1. 把机器模型打进 QEMU 7.2 源码树并编译。手工做法是拷三个文件、改 Kconfig 与
|
||||
# meson.build、再 configure 和 ninja；这个脚本就是那几步（下面"构建"一节有说明）。
|
||||
QEMU_SRC=~/src/qemu-7.2 bash tools/setup_qemu.sh
|
||||
|
||||
# 2. 拿一份可运行的固件。仓库不再分发固件——这个工具会从上游项目的归档里取一份到
|
||||
# assets/firmware/，并打印它的哈希。
|
||||
python3 tools/fetch_firmware.py
|
||||
|
||||
# 3. 固件保存设置所依赖的外部 flash 镜像。
|
||||
python3 tools/make_flash.py
|
||||
|
||||
# 4. 跑起来。
|
||||
python3 tools/webui.py --qemu ~/src/qemu-7.2/build/qemu-system-arm \
|
||||
--elf assets/firmware/f4hwn.fieldops.v6.0.0.bin # 然后打开 http://127.0.0.1:8080/
|
||||
|
||||
`--frame-addr` 与 `--status-addr` 默认值对应一份已知构建，换构建会变——怎么找见
|
||||
[网页远控](#网页远控)。Windows 上 `work/run-webui.ps1` 把第 4 步按本机路径包好了。
|
||||
|
||||
把任意 `.bin` 拖到页面上即可启动。页面上的 **Firmware slots** 表可以读写 flash 镜像里
|
||||
多系统固件的四个槽位，**Multiboot** 会按住 MENU 重启以进入多系统菜单——前提是那份构建
|
||||
确实带菜单：页面会标注不带菜单的构建，因为那时这个按钮做不了任何事。
|
||||
|
||||
再确认一下没坏：
|
||||
|
||||
bash tools/run_tests.sh -q # 约 15 秒，不需要模拟器
|
||||
bash tools/run_tests.sh # 全部；需要第 1 步那棵树
|
||||
|
||||
## 构建
|
||||
|
||||
需要 QEMU 7.2 源码树，以及 `meson`、`ninja`、`libfdt-dev`、`libglib2.0-dev`、
|
||||
@@ -142,6 +197,7 @@ VFO 重算了状态。在真机上你只看到"什么都没发生"，在这里
|
||||
python3 tools/test_flash_persist.py
|
||||
python3 tools/test_freq_entry.py
|
||||
python3 tools/test_serial_rx.py
|
||||
python3 tools/test_slot_serial.py
|
||||
python3 tools/test_bk4819.py
|
||||
bash tools/test_bk4819_readback.sh
|
||||
python3 tools/test_smeter.py
|
||||
@@ -218,9 +274,21 @@ VFO 重算了状态。在真机上你只看到"什么都没发生"，在这里
|
||||
| `POST /api/key` | `{"key": "MENU", "action": "down"}` — 也可以是 `up` 或 `tap` |
|
||||
| `POST /api/ptt` | `{"held": true}` — 按住 PTT，`false` 释放 |
|
||||
| `POST /api/release-all` | 释放所有按键，万一有键卡住 |
|
||||
| `GET /api/status` | QMP `query-status`，另含 `speaker` 字段 |
|
||||
| `GET /api/status` | QMP `query-status`，另含 `speaker`、`panel` 与 `firmware` |
|
||||
| `GET /api/firmware` | 当前镜像，以及它将按什么形态加载 |
|
||||
| `POST /api/firmware` | 请求体就是 `.bin` 或 `.elf`；启动它并重启模拟器 |
|
||||
| `GET /api/slots` | 当前 flash 镜像里的固件槽 |
|
||||
| `POST /api/slots/<n>` | 请求体是一个 `.bin`；写进槽 `n` 并重启 |
|
||||
| `POST /api/slots/<n>/erase` | 擦除槽 `n` |
|
||||
| `POST /api/flash` | 请求体是一份 flash 镜像；之后就用它 |
|
||||
|
||||
画面用 QMP `memsave` 读取，每帧约 1.35 ms，且 guest 全程继续运行。这里有两个细节很容易搞错：
|
||||
画面现在取自显示控制器自己的内存：对面板的 `gram` 属性做一次 QMP `qom-get`。
|
||||
这样无论固件是谁写的、把缓冲区放在哪里，画面都是对的 —— 同一祖先改出来的各个版本
|
||||
显示逻辑也各不相同，多系统那版干脆把图像放在完全不同的位置。
|
||||
|
||||
下面这段是**旧的取帧路径**（用 QMP `memsave` 直接读 guest RAM 里的
|
||||
`gFrameBuffer` 与 `gStatusLine`，每帧约 1.35 ms），它保留为"没有面板模型的模拟器"
|
||||
的回退路径；接下来的两条注意事项针对的正是它：
|
||||
|
||||
- **必须用 `memsave`，不能用 `pmemsave`。** 帧缓冲符号是 CPU 虚拟地址。`pmemsave` 会把参数
|
||||
当成物理地址，返回一整块零 —— 于是画面渲染成全空白，而且哪里都不报错。
|
||||
@@ -233,6 +301,55 @@ VFO 重算了状态。在真机上你只看到"什么都没发生"，在这里
|
||||
- **没有任何认证。** 任何能访问到这个端口的人都能完全控制这台模拟电台。正因如此，
|
||||
它默认只绑定 loopback。
|
||||
|
||||
### 从页面上传固件
|
||||
|
||||
把 `.bin` 直接拖到页面上，或用 "Firmware" 选择文件，服务器就会存下它并启动它。
|
||||
不需要包成 ELF，也不需要你去查地址。
|
||||
|
||||
镜像有两种形态，加载地址不同：
|
||||
|
||||
| 形态 | 怎么认出来 | 加载地址 |
|
||||
| --- | --- | --- |
|
||||
| 应用镜像 | 复位向量在 `0x08002800` 之后 | `0x08002800` |
|
||||
| 整片镜像 | 复位向量落在引导区（`0x08000000`..`0x080027ff`） | `0x08000000` |
|
||||
|
||||
`.elf` 自带程序头，两者都不需要。形态是**从镜像自己的头两个字读出来的**
|
||||
（主机侧 `tools/uvk5_image.py`，机器侧 `uvk5_sniff_app_offset()`），不是靠标志位或
|
||||
文件名，因为判断错的症状是**静默**的：镜像整体偏 `0x2800` 字节，第一次取指读到的是
|
||||
随便什么数据。不是可启动镜像的文件会以 400 拒绝，电台继续跑原来的固件。
|
||||
|
||||
上传文件放在 `work/firmware/`（`UVK5_UPLOAD_DIR` 可改）。模拟器正在运行时上传会
|
||||
让它重启一次 —— 镜像是 QEMU 启动时选定的。
|
||||
|
||||
这里两种形态都端到端验证过：应用 `.bin`、`.elf`，以及一个"引导入口跳到
|
||||
`0x08002800` 处应用"的整片镜像，最终都到达同一幅画面。
|
||||
|
||||
### 固件槽，与多系统版固件
|
||||
|
||||
v6.0.0 版把开机菜单和四个固件槽放在外部 flash 里：开机按住 MENU 就会列出它们，选中一个
|
||||
会用该槽的镜像重刷内部 flash 并复位。两部分都能从网页上操作。
|
||||
|
||||
- **固件槽**一栏每个槽一行，显示名字、版本、大小，以及头部 CRC-32 与镜像是否一致；可以
|
||||
往某个槽写入一个 `.bin`，或擦除该槽。改动只写**工作副本**
|
||||
（`work/firmware/flash-current.img`），绝不改服务器启动时指定的那份文件，改完自动重启
|
||||
模拟器来生效。
|
||||
- **Multiboot**（或 Shift+M）会**从复位起按住 MENU** 重启模拟器。网页的按键事件做不到
|
||||
这件事，因为固件在复位后的头几毫秒就采样键盘。机器侧对应
|
||||
`-M uv-k5-v3,boot-key=MENU` 或 `UVK5_BOOT_KEY`，保持时间由 `UVK5_BOOT_KEY_MS`
|
||||
决定（默认 8 秒：开机路径可能花 20 秒把当前固件"采纳"进槽 0，之后才会去采样键盘）。
|
||||
- `tools/uvk5_slots.py` 做同样的事但离线：把槽写进 flash 镜像，并打印每个槽的内容。
|
||||
|
||||
布局来自固件源码 `App/driver/mb_flash.h`：槽 0 在 `0x020000`，是内部镜像的备份；槽 1..4
|
||||
从 `0x040000` 起、每 128 KiB 一个；镜像从槽内偏移 4 KiB 开始；64 字节头部含魔数
|
||||
`FMB1`、镜像大小和 CRC-32。固件自带的 `0x0720`..`0x0727` 串口命令也按同样方式写槽，
|
||||
Windows 上的工具走的就是那条路。
|
||||
|
||||
有两种行为看起来像模拟器出错，其实不是：当**状态标记损坏**而槽 0 有效时，固件会停在
|
||||
`STATE ERROR` 画面上以保护 Main；当状态标记**缺失**时，它会在菜单出现前把当前固件"采纳"
|
||||
进槽 0（重写外部 flash）。写槽时会顺手擦掉那两个标记扇区，让它能重新判断。模型里的内部
|
||||
flash 是可编程的（`0x40022000`：解锁、页擦除、编程、EOP、永不 BSY），所以恢复槽位是真的
|
||||
替换了复位后 CPU 执行的镜像。
|
||||
|
||||
### 从别处访问
|
||||
|
||||
这里的部署方式是服务只监听 loopback，前面放 nginx 做 TLS，对外是
|
||||
@@ -299,7 +416,8 @@ VFO 重算了状态。在真机上你只看到"什么都没发生"，在这里
|
||||
SPI2 0x40003800 flash
|
||||
ADC1 0x40012400
|
||||
|
||||
已建模：RCC、GPIO、ADC、两个 SPI 控制器、DMA1、TIM2，以及 PY25Q16 flash。
|
||||
已建模：RCC、GPIO、ADC、两个 SPI 控制器、DMA1、TIM2、PY25Q16 flash，以及
|
||||
ST7565 显示控制器自身的设置（对比度、反显、开屏/关屏）。
|
||||
其余全部由一个带日志的兜底模块响应 —— **那份日志正是判断下一个值得建模的东西的依据。**
|
||||
|
||||
固件能启动之前，有七件事必须做对，每一件都是靠观察它停在哪里发现的：
|
||||
@@ -352,6 +470,39 @@ VFO 重算了状态。在真机上你只看到"什么都没发生"，在这里
|
||||
代码逐渐脱节。`CW_ReadKeys` 里的防抖是**照抄**而不是打桩的，因为它的不对称性
|
||||
（要连续三次读取才登记按下，而释放是立即的）本身就是被测时序行为的一部分。
|
||||
|
||||
## 显示控制器自己的设置
|
||||
|
||||
对比度（`SetCtr`）和反显（`SetInv`）是发给 ST7565 的命令，不是帧缓冲内容 ——
|
||||
`0x81 <值>` 与 `0xA6`/`0xA7` —— 所以 `gFrameBuffer` 一个字节都不变，任何渲染这份缓冲的
|
||||
界面都看不出效果。这就是 SPI1 后面挂了一个小型 `TYPE_ST7565` 的原因（A0 接 PA6、CS 接 PB2，
|
||||
即 `App/driver/st7565.c` 用的引脚）。它解析命令流并暴露三个**只读**属性：
|
||||
|
||||
qom-get /machine/panel invert # 0xA6 / 0xA7 之后的值
|
||||
qom-get /machine/panel contrast # 0x81 后面那个值
|
||||
qom-get /machine/panel display-on # 0xAE / 0xAF 之后的值
|
||||
|
||||
`tools/uvk5_lcd.py` 在渲染时应用反显，因为这个效果是完全确定的，于是那个菜单项在网页里
|
||||
看得见了。对比度是模拟量（玻璃有多黑），只报告不渲染。三者都在 `/api/status` 里，
|
||||
页面上显示在喇叭图标旁边。`display-on` 只报告不动作：软复位（`0xE2`）是否清掉那个锁存位
|
||||
无法确证，拿不确定的语义去把画面变黑，比不动它更糟。
|
||||
|
||||
## Windows 上
|
||||
|
||||
模拟器、模型和工具本身是可移植的，不可移植的是外围包装。有四处不同，现在都在仓库内处理了：
|
||||
|
||||
- **QMP 走 TCP。** Windows 版 QEMU 无法创建 unix socket，所以端点除了路径还可以是
|
||||
`host:port` —— `tools/uvk5_qmp.py`、`tools/key.py`、`tools/uvk5_supervisor.py` 两者都收。
|
||||
- **一处编译修正。** MSYS2 的 mingw-w64 能原样编译 QEMU 7.2，只有 `qemu/py32f071.c` 需要
|
||||
`#include "qapi/visitor.h"`（`visit_type_uint64`），原版源码树不会间接带入。
|
||||
- **发行版 `.bin` 不是内核镜像。** `armv7m_load_kernel()` 会把裸二进制加载到给它的地址上，
|
||||
而这里那个地址是 flash 的**别名区**，于是 `.bin` 会整体高 0x2800 字节、永远起不来。
|
||||
`tools/bin2elf.py` 给它套一个带正确程序头的 ELF32/ARM，这才是 `-kernel` 要的东西。
|
||||
- **中文字体包在 SPI flash 里**，不在固件里：`tools/make_flash.py --blob 0:pack.uf2`
|
||||
把每个 UF2 块放到它自己的目标地址。不做这一步，字体区读出来就是 0xFF。
|
||||
|
||||
`work/` 里留着一次 Windows 移植的记录：启动脚本、用固件源码验证过的取帧地址，
|
||||
以及那些花掉时间的失败。
|
||||
|
||||
## 许可
|
||||
|
||||
Apache 2.0，见 [LICENSE](LICENSE)。
|
||||
|
||||
+891
-14
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,84 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Wrap a raw V3 application .bin in an ELF32/ARM header so QEMU can load it.
|
||||
|
||||
Firmware releases for the UV-K5 V3 / UV-K1 ship as a flat application image: the
|
||||
file starts at its own vector table, which the linker places at 0x08002800, past
|
||||
the 10 KB bootloader region. An .elf from the same build carries that address in
|
||||
its program headers; a .bin has nowhere to put it.
|
||||
|
||||
This adds the missing program header, so a release .bin loads exactly where the
|
||||
.elf would:
|
||||
|
||||
read the initial SP and reset vector from the image's own vector table
|
||||
emit one PT_LOAD at p_paddr = 0x08002800, entry = the reset vector
|
||||
|
||||
Usage:
|
||||
bin2elf.py FIRMWARE.bin [-o FIRMWARE.elf] [--base 0x08002800]
|
||||
|
||||
Then, as with any other image:
|
||||
|
||||
qemu-system-arm -M "uv-k5-v3,flash-image=assets/flash.img" -kernel FIRMWARE.elf ...
|
||||
|
||||
Guessing is not needed and is not done: the base defaults to the model's
|
||||
PY32_APP_OFFSET and is checked against the reset vector, so an image linked for a
|
||||
different base is refused rather than silently loaded in the wrong place.
|
||||
"""
|
||||
import argparse
|
||||
import struct
|
||||
import sys
|
||||
|
||||
APP_BASE_DEFAULT = 0x08002800 # qemu/py32f071.c: PY32_FLASH_BASE | PY32_APP_OFFSET
|
||||
VECTOR_TABLE_WORDS = 48 # 0xc0 bytes, per the linker script's .isr_vector
|
||||
|
||||
EM_ARM = 40
|
||||
ET_EXEC = 2
|
||||
PT_LOAD = 1
|
||||
EF_ARM_EABI_VER5 = 0x05000000
|
||||
|
||||
|
||||
def build_elf(image: bytes, base: int) -> bytes:
|
||||
if len(image) < 8:
|
||||
raise SystemExit("image is too short to hold a vector table")
|
||||
sp, reset = struct.unpack_from("<II", image, 0)
|
||||
if not (0x20000000 <= sp <= 0x20004000):
|
||||
raise SystemExit(
|
||||
f"initial SP 0x{sp:08x} is not in SRAM -- is this an application image?")
|
||||
if (reset & ~1) < base or (reset & ~1) >= base + len(image):
|
||||
raise SystemExit(
|
||||
f"reset vector 0x{reset:08x} does not land inside the image loaded at "
|
||||
f"0x{base:08x} -- wrong --base?")
|
||||
|
||||
p_offset = base & 0xFFFF # keeps p_offset congruent with p_vaddr
|
||||
ehdr = struct.pack(
|
||||
"<16sHHIIIIIHHHHHH",
|
||||
b"\x7fELF\x01\x01\x01" + b"\x00" * 9, # 32-bit LE, System V ABI
|
||||
ET_EXEC, EM_ARM, 1, reset, 52, 0, EF_ARM_EABI_VER5,
|
||||
52, 32, 1, 40, 0, 0)
|
||||
phdr = struct.pack(
|
||||
"<IIIIIIII",
|
||||
PT_LOAD, p_offset, base, base, len(image), len(image), 5, 0x1000)
|
||||
pad = b"\x00" * (p_offset - len(ehdr) - len(phdr))
|
||||
return ehdr + phdr + pad + image
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument("image")
|
||||
ap.add_argument("-o", "--out")
|
||||
ap.add_argument("--base", type=lambda s: int(s, 0), default=APP_BASE_DEFAULT)
|
||||
args = ap.parse_args()
|
||||
|
||||
data = open(args.image, "rb").read()
|
||||
out = args.out or (args.image.rsplit(".", 1)[0] + ".elf")
|
||||
elf = build_elf(data, args.base)
|
||||
open(out, "wb").write(elf)
|
||||
|
||||
sp, reset = struct.unpack_from("<II", data, 0)
|
||||
print(f"{args.image} -> {out}")
|
||||
print(f" {len(data)} bytes at 0x{args.base:08x}")
|
||||
print(f" initial SP 0x{sp:08x}, entry 0x{reset:08x}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+15
-11
@@ -27,12 +27,16 @@ and the checker was broken. A checker that cries wolf gets ignored, so anything
|
||||
cannot verify unambiguously is left out rather than guessed at.
|
||||
"""
|
||||
|
||||
import os
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
|
||||
SIM = pathlib.Path(__file__).resolve().parent.parent
|
||||
FW = pathlib.Path("/root/uvk5-port/uvk5-sat/App")
|
||||
# Where the firmware sources are. Override when the tree is not at the default
|
||||
# path -- without that the checker cannot run anywhere but the machine it was
|
||||
# written on, and the file:line checks are the ones that catch drifting prose.
|
||||
FW = pathlib.Path(os.environ.get("UVK5_FW_DIR", "/root/uvk5-port/uvk5-sat/App"))
|
||||
|
||||
PAIRS = [
|
||||
("README.md", "README.zh-CN.md"),
|
||||
@@ -95,7 +99,7 @@ def resolve_fw(name):
|
||||
def check_tools_exist():
|
||||
print("tools named in a README must exist")
|
||||
for doc in ("README.md", "README.zh-CN.md"):
|
||||
text = (SIM / doc).read_text()
|
||||
text = (SIM / doc).read_text(encoding="utf-8")
|
||||
for tool in sorted(set(re.findall(r"tools/([a-z0-9_]+\.(?:py|sh))", text))):
|
||||
if not (SIM / "tools" / tool).exists():
|
||||
fail(f"{doc} names tools/{tool}, which does not exist")
|
||||
@@ -103,10 +107,10 @@ def check_tools_exist():
|
||||
|
||||
def check_tests_documented():
|
||||
print("every test in run_tests.sh must be documented")
|
||||
runner = (SIM / "tools" / "run_tests.sh").read_text()
|
||||
runner = (SIM / "tools" / "run_tests.sh").read_text(encoding="utf-8")
|
||||
in_runner = set(re.findall(r"tools/([a-z0-9_]+\.(?:py|sh))", runner))
|
||||
for doc in ("README.md", "README.zh-CN.md"):
|
||||
text = (SIM / doc).read_text()
|
||||
text = (SIM / doc).read_text(encoding="utf-8")
|
||||
for tool in sorted(in_runner):
|
||||
if tool not in text:
|
||||
fail(f"{doc} does not mention {tool}, which run_tests.sh runs")
|
||||
@@ -116,7 +120,7 @@ def check_links():
|
||||
print("internal .md links must resolve")
|
||||
for doc in [d for pair in PAIRS for d in pair]:
|
||||
path = SIM / doc
|
||||
for target in re.findall(r"\]\(([^)]+\.md)\)", path.read_text()):
|
||||
for target in re.findall(r"\]\(([^)]+\.md)\)", path.read_text(encoding="utf-8")):
|
||||
if target.startswith("http"):
|
||||
continue
|
||||
if not (path.parent / target).exists():
|
||||
@@ -126,8 +130,8 @@ def check_links():
|
||||
def check_pairs():
|
||||
print("translation pairs must have matching structure")
|
||||
for en_name, zh_name in PAIRS:
|
||||
en = re.findall(r"^(#+) (.+)$", (SIM / en_name).read_text(), re.M)
|
||||
zh = re.findall(r"^(#+) (.+)$", (SIM / zh_name).read_text(), re.M)
|
||||
en = re.findall(r"^(#+) (.+)$", (SIM / en_name).read_text(encoding="utf-8"), re.M)
|
||||
zh = re.findall(r"^(#+) (.+)$", (SIM / zh_name).read_text(encoding="utf-8"), re.M)
|
||||
if len(en) != len(zh):
|
||||
fail(f"{en_name} has {len(en)} headings, {zh_name} has {len(zh)}")
|
||||
continue
|
||||
@@ -139,7 +143,7 @@ def check_pairs():
|
||||
|
||||
def check_memory_map():
|
||||
print("memory-map addresses must match the model")
|
||||
model = (SIM / "qemu" / "py32f071.c").read_text()
|
||||
model = (SIM / "qemu" / "py32f071.c").read_text(encoding="utf-8")
|
||||
for sym, documented in MEMORY_MAP.items():
|
||||
m = re.search(rf"#define {sym}\s+(\S+)", model)
|
||||
if not m:
|
||||
@@ -161,7 +165,7 @@ def check_documented_flags():
|
||||
"""
|
||||
print("documented tool flags must exist")
|
||||
text = "\n".join(
|
||||
(SIM / doc).read_text() for pair in PAIRS for doc in pair)
|
||||
(SIM / doc).read_text(encoding="utf-8") for pair in PAIRS for doc in pair)
|
||||
joined = re.sub(r"\\\s*\n\s*", " ", text)
|
||||
|
||||
claims = {}
|
||||
@@ -173,7 +177,7 @@ def check_documented_flags():
|
||||
path = SIM / "tools" / tool
|
||||
if not path.exists():
|
||||
continue # already reported by check_tools_exist
|
||||
src = path.read_text()
|
||||
src = path.read_text(encoding="utf-8")
|
||||
for flag in sorted(flags):
|
||||
if flag not in src:
|
||||
fail(f"docs pass {flag} to {tool}, which does not accept it")
|
||||
@@ -186,7 +190,7 @@ def check_line_refs():
|
||||
if path is None:
|
||||
fail(f"{name} is referenced but not found in the firmware tree")
|
||||
continue
|
||||
lines = path.read_text().splitlines()
|
||||
lines = path.read_text(encoding="utf-8").splitlines()
|
||||
if line > len(lines):
|
||||
fail(f"{name}:{line} is past the end of the file ({len(lines)} lines)")
|
||||
continue
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fetch a released firmware build, so the tests are not anchored to one machine.
|
||||
|
||||
Several emulator tests need a real image to run against: the multi-system slot tests
|
||||
need a build with ENABLE_FEAT_F4HWN_MULTIBOOT (the 0x0720..0x0727 commands are inside
|
||||
that ifdef), and a plain application image is useful for the rest. Neither is in this
|
||||
repository -- firmware is not ours to redistribute -- and pointing tests at one
|
||||
developer's build directory is what makes a suite unreproducible.
|
||||
|
||||
So the tests read a documented directory, `assets/firmware/`, and skip with a message
|
||||
naming this tool when it is empty. Run it once:
|
||||
|
||||
python3 tools/fetch_firmware.py # fieldops-v6.0.0, the multiboot one
|
||||
python3 tools/fetch_firmware.py --list
|
||||
python3 tools/fetch_firmware.py --all
|
||||
|
||||
Files come from the upstream project's own archive, through jsDelivr, and each one
|
||||
prints its SHA-256 so a fetch can be compared with someone else's.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import os
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
ROOT = os.path.dirname(HERE)
|
||||
DEST_DIR = os.path.join(ROOT, "assets", "firmware")
|
||||
|
||||
# armel/uv-k1-k5v3-firmware-custom, the firmware the emulator is built for. The v6.0.0
|
||||
# builds are the ones with the multi-system boot menu; the localised releases floating
|
||||
# around are often built without it.
|
||||
BASE = "https://cdn.jsdelivr.net/gh/armel/uv-k1-k5v3-firmware-custom@main/"
|
||||
|
||||
RELEASES = {
|
||||
"fieldops-v6.0.0": ("archive/f4hwn.fieldops.v6.0.0.bin",
|
||||
"application image with the multi-system boot menu"),
|
||||
"fusion-v6.0.0": ("archive/f4hwn.fusion.v6.0.0.bin",
|
||||
"application image with the multi-system boot menu"),
|
||||
"fusion-v5.9.0": ("archive/f4hwn.fusion.v5.9.0.bin", "application image, no boot menu"),
|
||||
}
|
||||
|
||||
|
||||
def fetch(name: str, dest_dir: str = DEST_DIR) -> str:
|
||||
if name not in RELEASES:
|
||||
raise SystemExit("unknown release %r; --list shows what there is" % name)
|
||||
path, _why = RELEASES[name]
|
||||
os.makedirs(dest_dir, exist_ok=True)
|
||||
out = os.path.join(dest_dir, os.path.basename(path))
|
||||
url = BASE + path
|
||||
print("fetching %s" % url)
|
||||
with urllib.request.urlopen(url, timeout=120) as response:
|
||||
blob = response.read()
|
||||
if not blob:
|
||||
raise SystemExit("empty response from %s" % url)
|
||||
with open(out, "wb") as fh:
|
||||
fh.write(blob)
|
||||
print("wrote %s (%d bytes)" % (out, len(blob)))
|
||||
print("sha256 %s" % hashlib.sha256(blob).hexdigest())
|
||||
return out
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument("--list", action="store_true", help="show the known releases")
|
||||
ap.add_argument("--all", action="store_true", help="fetch everything")
|
||||
ap.add_argument("names", nargs="*", help="release names (default: fieldops-v6.0.0)")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if args.list:
|
||||
for name, (path, why) in sorted(RELEASES.items()):
|
||||
print("%-18s %-40s %s" % (name, path, why))
|
||||
return 0
|
||||
names = sorted(RELEASES) if args.all else (args.names or ["fieldops-v6.0.0"])
|
||||
for name in names:
|
||||
fetch(name)
|
||||
print("")
|
||||
print("tests look in %s now; UVK5_MULTIBOOT_IMAGE overrides the path"
|
||||
% DEST_DIR)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+9
-3
@@ -47,10 +47,16 @@ class Qmp:
|
||||
"""Minimal QMP client: connect, negotiate, send commands."""
|
||||
|
||||
def __init__(self, path: str):
|
||||
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
# A unix socket path on Linux, or "host:port": a Windows build of QEMU
|
||||
# has no unix sockets, so the TCP form is the only way in there.
|
||||
host, _, port = path.rpartition(":")
|
||||
try:
|
||||
self.sock.connect(path)
|
||||
except (FileNotFoundError, ConnectionRefusedError) as exc:
|
||||
if host and port.isdigit():
|
||||
self.sock = socket.create_connection((host, int(port)), timeout=10)
|
||||
else:
|
||||
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
self.sock.connect(path)
|
||||
except (FileNotFoundError, ConnectionRefusedError, OSError) as exc:
|
||||
raise SystemExit(
|
||||
f"cannot reach the emulator at {path}: {exc}\n"
|
||||
"Start it with sim/tools/run.sh first."
|
||||
|
||||
+24
-15
@@ -28,21 +28,21 @@ Usage:
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
import re
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
HOME = os.path.expanduser("~")
|
||||
QEMU = os.environ.get(
|
||||
"QEMU_BIN", f"{HOME}/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm")
|
||||
ELF = os.environ.get(
|
||||
"ELF", f"{HOME}/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf")
|
||||
QEMU = uvk5_testenv.qemu()
|
||||
ELF = uvk5_testenv.firmware()
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
FLASH = os.path.join(os.path.dirname(HERE), "assets", "flash.img")
|
||||
|
||||
QMP = "/tmp/uvk5-keypad-test-qmp.sock"
|
||||
QMP = uvk5_socket.server_endpoint("qmp")
|
||||
GDB_PORT = "1239"
|
||||
|
||||
# App/misc.c: key_debounce_10ms = 2 (20 ms), key_repeat_delay_10ms = 40 (400 ms).
|
||||
@@ -64,28 +64,33 @@ class Emu:
|
||||
for path in (QMP,):
|
||||
if os.path.exists(path):
|
||||
os.unlink(path)
|
||||
for path, what in ((QEMU, "QEMU binary"), (ELF, "firmware ELF"),
|
||||
(FLASH, "flash image")):
|
||||
if not os.path.exists(path):
|
||||
sys.exit(f"missing {what}: {path}")
|
||||
_missing = uvk5_testenv.missing([
|
||||
(QEMU, "QEMU binary", "set QEMU=/path/to/qemu-system-arm or put it on PATH"),
|
||||
(ELF, "firmware ELF", "run tools/fetch_firmware.py or set ELF=..."),
|
||||
(FLASH, "flash image", "run tools/make_flash.py"),
|
||||
])
|
||||
if _missing:
|
||||
print("SKIP: %s" % _missing)
|
||||
sys.exit(0)
|
||||
|
||||
self.proc = subprocess.Popen(
|
||||
[QEMU, "-M", f"uv-k5-v3,flash-image={FLASH}", "-nographic",
|
||||
"-monitor", "none", "-qmp", f"unix:{QMP},server=on,wait=off",
|
||||
"-monitor", "none", "-qmp", QMP,
|
||||
"-kernel", ELF, "-gdb", f"tcp::{GDB_PORT}"],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
|
||||
for _ in range(150):
|
||||
try:
|
||||
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
self.sock.connect(QMP)
|
||||
# Connecting is the test: a unix path can be waited for as a file, a
|
||||
# TCP endpoint cannot, and this works for both.
|
||||
self.sock = uvk5_socket.connect(QMP, timeout=2)
|
||||
break
|
||||
except OSError:
|
||||
if self.proc.poll() is not None:
|
||||
sys.exit("QEMU exited during startup")
|
||||
time.sleep(0.1)
|
||||
else:
|
||||
sys.exit(f"QMP socket never appeared at {QMP}")
|
||||
sys.exit(f"QMP never accepted a connection at {QMP}")
|
||||
|
||||
self.buf = b""
|
||||
self._read() # greeting
|
||||
@@ -134,7 +139,7 @@ class Emu:
|
||||
("kr0", "*(char*)&gKeyReading0"),
|
||||
("cursor", f"*(unsigned char*){GMENUCURSOR_ADDR}"),
|
||||
]
|
||||
args = ["gdb-multiarch", "-batch", "-ex", "set confirm off",
|
||||
args = [str(uvk5_testenv.gdb()), "-batch", "-ex", "set confirm off",
|
||||
"-ex", "set pagination off",
|
||||
"-ex", f"target remote :{GDB_PORT}"]
|
||||
for name, expr in exprs:
|
||||
@@ -157,6 +162,10 @@ class Emu:
|
||||
|
||||
|
||||
def main():
|
||||
if uvk5_testenv.gdb() is None:
|
||||
return uvk5_testenv.skip("gdb-multiarch is missing; this test reads the guest over "
|
||||
"a gdb attach, which has not been ported to the QMP memsave "
|
||||
"route the page uses")
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("-v", "--verbose", action="store_true")
|
||||
args = ap.parse_args()
|
||||
|
||||
+65
-1
@@ -9,26 +9,77 @@ paths, so the emulated radio would not represent a real one.
|
||||
The image itself is not committed: it is 2 MB and fully derived from
|
||||
assets/calibration.bin.
|
||||
|
||||
Usage: make_flash.py [--calibration FILE] [--out FILE]
|
||||
Firmware that keeps data outside its own 128 KB -- the f4hwn/Chinese builds put
|
||||
their font packs there -- needs that data placed too, at the same offsets the
|
||||
real radio uses:
|
||||
|
||||
--blob 0xA0000:pack.uf2 # raw file, or a .uf2 written by address
|
||||
--blob 0xA0000:font.bin # plain blob at an explicit offset
|
||||
|
||||
A .uf2 is decoded block by block and each 256-byte payload is written to its own
|
||||
target address, so the offsets come from the file rather than from a guess.
|
||||
|
||||
Usage: make_flash.py [--calibration FILE] [--out FILE] [--blob ADDR:FILE ...]
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import pathlib
|
||||
import struct
|
||||
import sys
|
||||
|
||||
FLASH_SIZE = 2 * 1024 * 1024
|
||||
CALIBRATION_ADDR = 0x010000
|
||||
CALIBRATION_SIZE = 512
|
||||
|
||||
UF2_MAGIC0 = 0x0A324655
|
||||
UF2_MAGIC1 = 0x9E5D5157
|
||||
UF2_MAGIC_END = 0x0AB16F30
|
||||
|
||||
HERE = pathlib.Path(__file__).resolve().parent
|
||||
ASSETS = HERE.parent / "assets"
|
||||
|
||||
|
||||
def apply_uf2(image: bytearray, data: bytes, label: str) -> None:
|
||||
"""Write every UF2 block to the address recorded in that block."""
|
||||
if len(data) % 512:
|
||||
raise SystemExit(f"{label}: not a UF2 image (size is not a multiple of 512)")
|
||||
written = 0
|
||||
for i in range(0, len(data), 512):
|
||||
m0, m1, _flags, addr, plen, _no, _num, _family = struct.unpack_from(
|
||||
"<IIIIIIII", data, i)
|
||||
# The closing magic sits at offset 508; the 476 bytes between the header
|
||||
# and it are the payload, of which payloadSize is meaningful.
|
||||
(mend,) = struct.unpack_from("<I", data, i + 508)
|
||||
if m0 != UF2_MAGIC0 or m1 != UF2_MAGIC1 or mend != UF2_MAGIC_END:
|
||||
raise SystemExit(f"{label}: bad UF2 magic at block {i // 512}")
|
||||
if plen > 476:
|
||||
raise SystemExit(f"{label}: block {i // 512} claims {plen} payload bytes")
|
||||
end = addr + plen
|
||||
if end > len(image):
|
||||
raise SystemExit(
|
||||
f"{label}: block {i // 512} writes 0x{addr:x}..0x{end:x}, "
|
||||
f"past the end of the {len(image)}-byte flash")
|
||||
image[addr:end] = data[i + 32:i + 32 + plen]
|
||||
written += plen
|
||||
print(f" {label}: {written} bytes from UF2")
|
||||
|
||||
|
||||
def apply_blob(image: bytearray, addr: int, data: bytes, label: str) -> None:
|
||||
end = addr + len(data)
|
||||
if end > len(image):
|
||||
raise SystemExit(f"{label}: 0x{addr:x}..0x{end:x} does not fit in flash")
|
||||
image[addr:end] = data
|
||||
print(f" {label}: {len(data)} bytes at 0x{addr:08x}")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--calibration", type=pathlib.Path,
|
||||
default=ASSETS / "calibration.bin")
|
||||
ap.add_argument("--out", type=pathlib.Path, default=ASSETS / "flash.img")
|
||||
ap.add_argument("--blob", action="append", default=[], metavar="ADDR:FILE",
|
||||
help="extra data to place; a .uf2 is decoded by its own "
|
||||
"block addresses, anything else lands at ADDR")
|
||||
args = ap.parse_args()
|
||||
|
||||
if not args.calibration.is_file():
|
||||
@@ -42,6 +93,19 @@ def main() -> int:
|
||||
image = bytearray(b"\xff" * FLASH_SIZE)
|
||||
image[CALIBRATION_ADDR:CALIBRATION_ADDR + len(cal)] = cal
|
||||
|
||||
for spec in args.blob:
|
||||
if ":" not in spec:
|
||||
raise SystemExit(f"--blob wants ADDR:FILE, got {spec!r}")
|
||||
addr_text, name = spec.split(":", 1)
|
||||
path = pathlib.Path(name)
|
||||
if not path.is_file():
|
||||
raise SystemExit(f"--blob file not found: {path}")
|
||||
data = path.read_bytes()
|
||||
if path.suffix.lower() == ".uf2":
|
||||
apply_uf2(image, data, path.name)
|
||||
else:
|
||||
apply_blob(image, int(addr_text, 0), data, path.name)
|
||||
|
||||
args.out.write_bytes(image)
|
||||
print(f"wrote {args.out} ({len(image)} bytes)")
|
||||
print(f" calibration at {CALIBRATION_ADDR:#08x}: "
|
||||
|
||||
+28
-15
@@ -20,6 +20,18 @@ HERE=$(cd "$(dirname "$0")" && pwd)
|
||||
SIM=$(dirname "$HERE")
|
||||
QEMU_SRC=${QEMU_SRC:-/root/qemu-build/qemu-7.2+dfsg}
|
||||
|
||||
# One interpreter name, resolved once. This script used to spell $PY on every
|
||||
# line, which is a Windows problem (there is a python, not a python3) and a
|
||||
# virtualenv problem alike -- and a runner that cannot start is indistinguishable
|
||||
# from a runner that passes, which is the failure mode this script exists to avoid.
|
||||
PY=${PYTHON:-}
|
||||
if [ -z "$PY" ]; then
|
||||
for candidate in $PY python; do
|
||||
if command -v "$candidate" >/dev/null 2>&1; then PY=$candidate; break; fi
|
||||
done
|
||||
fi
|
||||
[ -n "$PY" ] || { echo "no $PY or python on PATH; set PYTHON=/path/to/python" >&2; exit 1; }
|
||||
|
||||
QUICK=0
|
||||
[ "${1:-}" = "-q" ] && QUICK=1
|
||||
|
||||
@@ -66,9 +78,9 @@ cd "$HERE"
|
||||
# First, that this script itself reports failures. A runner that silently counts every
|
||||
# test as passing is worse than no runner, because it gets trusted.
|
||||
run "runner self-check" bash "$HERE/test_run_tests.sh"
|
||||
run "docs match code" python3 "$HERE/check_docs.py"
|
||||
run "unit: model helpers" python3 -m unittest discover -p 'test_uvk5*.py' -q
|
||||
run "unit: web UI" python3 -m unittest test_webui -q
|
||||
run "docs match code" $PY "$HERE/check_docs.py"
|
||||
run "unit: model helpers" $PY -m unittest discover -p 'test_uvk5*.py' -q
|
||||
run "unit: web UI" $PY -m unittest test_webui -q
|
||||
|
||||
if [ "$QUICK" = "1" ]; then
|
||||
printf '\n%d passed, %d failed (unit tests only)\n' "$pass" "$fail"
|
||||
@@ -80,19 +92,20 @@ fi
|
||||
# Ordered cheapest first, so an obvious breakage surfaces without waiting for the
|
||||
# whole run.
|
||||
cd "$SIM"
|
||||
run "keypad" python3 tools/keypad_test.py
|
||||
run "BK4819 registers" python3 tools/test_bk4819.py
|
||||
run "keypad" $PY tools/keypad_test.py
|
||||
run "BK4819 registers" $PY tools/test_bk4819.py
|
||||
run "register readback" bash tools/test_bk4819_readback.sh
|
||||
run "S-meter" python3 tools/test_smeter.py
|
||||
run "PTT" python3 tools/test_ptt.py
|
||||
run "scan" python3 tools/test_scan.py
|
||||
run "audio path" python3 tools/test_audio_path.py
|
||||
run "battery" python3 tools/test_battery.py
|
||||
run "millis" python3 tools/test_millis.py
|
||||
run "spectrum" python3 tools/test_spectrum.py
|
||||
run "serial receive" python3 tools/test_serial_rx.py
|
||||
run "flash persistence" python3 tools/test_flash_persist.py
|
||||
run "frequency entry" python3 tools/test_freq_entry.py
|
||||
run "S-meter" $PY tools/test_smeter.py
|
||||
run "PTT" $PY tools/test_ptt.py
|
||||
run "scan" $PY tools/test_scan.py
|
||||
run "audio path" $PY tools/test_audio_path.py
|
||||
run "battery" $PY tools/test_battery.py
|
||||
run "millis" $PY tools/test_millis.py
|
||||
run "spectrum" $PY tools/test_spectrum.py
|
||||
run "serial receive" $PY tools/test_serial_rx.py
|
||||
run "slot over serial" $PY tools/test_slot_serial.py
|
||||
run "flash persistence" $PY tools/test_flash_persist.py
|
||||
run "frequency entry" $PY tools/test_freq_entry.py
|
||||
|
||||
printf '\n%d passed, %d failed\n' "$pass" "$fail"
|
||||
if [ "$fail" != "0" ]; then
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
#!/usr/bin/env bash
|
||||
# Put this machine into a QEMU 7.2 source tree, then build it.
|
||||
#
|
||||
# The same steps README.md documents, in the order that works, made idempotent and
|
||||
# checked: doing them by hand is where "I could not get it to run" begins, and a
|
||||
# hand-copied file with no record of what it replaced is what makes the next QEMU
|
||||
# release a manual job.
|
||||
#
|
||||
# QEMU_SRC=/path/to/qemu-7.2 bash tools/setup_qemu.sh
|
||||
#
|
||||
# Nothing here is machine-specific. QEMU_SRC defaults to a sibling of this checkout.
|
||||
set -euo pipefail
|
||||
|
||||
HERE=$(cd "$(dirname "$0")" && pwd)
|
||||
ROOT=$(dirname "$HERE")
|
||||
QEMU_SRC=${QEMU_SRC:-$ROOT/../qemu-7.2}
|
||||
|
||||
say() { printf '%s\n' "$*"; }
|
||||
die() { printf '%s\n' "$*" >&2; exit 1; }
|
||||
|
||||
[ -d "$QEMU_SRC" ] || die "no QEMU source tree at $QEMU_SRC -- set QEMU_SRC"
|
||||
[ -f "$QEMU_SRC/hw/arm/meson.build" ] || die "$QEMU_SRC does not look like a QEMU tree"
|
||||
for f in qemu/py32f071.c qemu/armv7m_systick.c.patched qemu/armv7m_systick.h.patched; do
|
||||
[ -f "$ROOT/$f" ] || die "missing $f"
|
||||
done
|
||||
|
||||
say "== 1. machine and timer sources"
|
||||
# py32f071.c is ours outright. The two systick files are whole upstream files with
|
||||
# this machine's changes in them; they are copied over and the originals are kept as
|
||||
# .orig so a diff against a new QEMU release is a diff, not an archaeology exercise.
|
||||
cp "$ROOT/qemu/py32f071.c" "$QEMU_SRC/hw/arm/py32f071.c"
|
||||
for pair in "hw/timer/armv7m_systick.c:qemu/armv7m_systick.c.patched" \
|
||||
"include/hw/timer/armv7m_systick.h:qemu/armv7m_systick.h.patched"; do
|
||||
dst="${pair%%:*}"; src="${pair##*:}"
|
||||
if [ -f "$QEMU_SRC/$dst" ] && [ ! -f "$QEMU_SRC/$dst.orig" ]; then
|
||||
cp "$QEMU_SRC/$dst" "$QEMU_SRC/$dst.orig"
|
||||
fi
|
||||
cp "$ROOT/$src" "$QEMU_SRC/$dst"
|
||||
done
|
||||
|
||||
say "== 2. register the machine"
|
||||
if ! grep -q 'config UVK5_V3' "$QEMU_SRC/hw/arm/Kconfig"; then
|
||||
cat >> "$QEMU_SRC/hw/arm/Kconfig" <<'KCONFIG'
|
||||
|
||||
config UVK5_V3
|
||||
bool
|
||||
default y
|
||||
depends on TCG && ARM
|
||||
select PY32F071_SOC
|
||||
|
||||
config PY32F071_SOC
|
||||
bool
|
||||
select ARM_V7M
|
||||
select UNIMP
|
||||
KCONFIG
|
||||
say " appended UVK5_V3 / PY32F071_SOC to hw/arm/Kconfig"
|
||||
else
|
||||
say " hw/arm/Kconfig already has UVK5_V3"
|
||||
fi
|
||||
if ! grep -q "py32f071.c" "$QEMU_SRC/hw/arm/meson.build"; then
|
||||
cat >> "$QEMU_SRC/hw/arm/meson.build" <<'MESON'
|
||||
|
||||
arm_ss.add(when: 'CONFIG_UVK5_V3', if_true: files('py32f071.c'))
|
||||
MESON
|
||||
say " added py32f071.c to hw/arm/meson.build"
|
||||
else
|
||||
say " hw/arm/meson.build already lists py32f071.c"
|
||||
fi
|
||||
|
||||
say "== 3. configure and build"
|
||||
cd "$QEMU_SRC"
|
||||
if [ ! -f build/build.ninja ]; then
|
||||
./configure --target-list=arm-softmmu --disable-docs --disable-tools
|
||||
fi
|
||||
ninja -C build qemu-system-arm
|
||||
|
||||
say ""
|
||||
say "built: $QEMU_SRC/build/qemu-system-arm"
|
||||
say "point the tools at it with QEMU=$QEMU_SRC/build/qemu-system-arm, or run"
|
||||
say " bash tools/run_tests.sh -q"
|
||||
+15
-20
@@ -32,11 +32,12 @@ import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
|
||||
SIM = pathlib.Path(__file__).resolve().parent.parent
|
||||
QEMU = pathlib.Path(os.environ.get(
|
||||
"QEMU", "/root/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm"))
|
||||
ELF = pathlib.Path(os.environ.get(
|
||||
"ELF", "/root/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf"))
|
||||
QEMU = uvk5_testenv.qemu()
|
||||
ELF = uvk5_testenv.firmware()
|
||||
PRISTINE = SIM / "assets/pristine/flash-pristine.img.gz"
|
||||
|
||||
BOOT_SECONDS = 24
|
||||
@@ -44,9 +45,7 @@ BOOT_SECONDS = 24
|
||||
|
||||
class Qmp:
|
||||
def __init__(self, path):
|
||||
self.s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
self.s.settimeout(25)
|
||||
self.s.connect(path)
|
||||
self.s = uvk5_socket.connect(path, timeout=25)
|
||||
self.buf = b""
|
||||
self._read()
|
||||
self.cmd("qmp_capabilities")
|
||||
@@ -81,30 +80,26 @@ class Qmp:
|
||||
|
||||
|
||||
def main():
|
||||
for tool in (QEMU, ELF, PRISTINE):
|
||||
if not tool.exists():
|
||||
print(f"SKIP missing {tool}")
|
||||
return 0
|
||||
for tool, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine flash image")):
|
||||
if tool is None or not tool.exists():
|
||||
return uvk5_testenv.skip("%s is missing (%s); see the README Quick start"
|
||||
% (what, tool or "not found"))
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
img = pathlib.Path(tmp) / "flash.img"
|
||||
img.write_bytes(gzip.decompress(PRISTINE.read_bytes()))
|
||||
sock = pathlib.Path(tmp) / "qmp.sock"
|
||||
sock = uvk5_socket.server_endpoint("qmp", directory=str(tmp))
|
||||
|
||||
proc = subprocess.Popen(
|
||||
[str(QEMU), "-M", f"uv-k5-v3,flash-image={img}",
|
||||
"-nographic", "-monitor", "none",
|
||||
"-qmp", f"unix:{sock},server=on,wait=off",
|
||||
"-qmp", sock,
|
||||
"-kernel", str(ELF)],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
try:
|
||||
for _ in range(BOOT_SECONDS * 4):
|
||||
if sock.exists():
|
||||
break
|
||||
time.sleep(0.25)
|
||||
else:
|
||||
print("FAIL QMP socket never appeared")
|
||||
return 1
|
||||
# Qmp() below connects, and uvk5_socket retries until QEMU's QMP answers, so
|
||||
# there is no socket path to wait for -- on Windows there would not be one.
|
||||
time.sleep(BOOT_SECONDS)
|
||||
time.sleep(BOOT_SECONDS)
|
||||
|
||||
qmp = Qmp(str(sock))
|
||||
|
||||
+20
-21
@@ -25,11 +25,12 @@ import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
|
||||
SIM = pathlib.Path(__file__).resolve().parent.parent
|
||||
QEMU = pathlib.Path(os.environ.get(
|
||||
"QEMU", "/root/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm"))
|
||||
ELF = pathlib.Path(os.environ.get(
|
||||
"ELF", "/root/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf"))
|
||||
QEMU = uvk5_testenv.qemu()
|
||||
ELF = uvk5_testenv.firmware()
|
||||
PRISTINE = SIM / "assets/pristine/flash-pristine.img.gz"
|
||||
|
||||
BOOT_SECONDS = 24
|
||||
@@ -42,9 +43,7 @@ SETTLE = 6
|
||||
|
||||
class Qmp:
|
||||
def __init__(self, path):
|
||||
self.s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
self.s.settimeout(25)
|
||||
self.s.connect(path)
|
||||
self.s = uvk5_socket.connect(path, timeout=25)
|
||||
self.buf = b""
|
||||
self._read()
|
||||
self.cmd("qmp_capabilities")
|
||||
@@ -80,7 +79,7 @@ def firmware_state(port):
|
||||
anything timing-dependent.
|
||||
"""
|
||||
out = subprocess.run(
|
||||
["gdb-multiarch", "-batch",
|
||||
[str(uvk5_testenv.gdb()), "-batch",
|
||||
"-ex", "set confirm off", "-ex", "set pagination off",
|
||||
"-ex", f"target remote :{port}",
|
||||
"-ex", 'printf "LEVEL=%d LOW=%d\\n",'
|
||||
@@ -96,31 +95,31 @@ def firmware_state(port):
|
||||
|
||||
|
||||
def main():
|
||||
for tool in (QEMU, ELF, PRISTINE):
|
||||
if not tool.exists():
|
||||
print(f"SKIP missing {tool}")
|
||||
return 0
|
||||
if uvk5_testenv.gdb() is None:
|
||||
return uvk5_testenv.skip("gdb-multiarch is missing; this test reads firmware "
|
||||
"globals over a gdb attach, which has not been ported to "
|
||||
"the QMP memsave route the page uses")
|
||||
for tool, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine flash image")):
|
||||
if tool is None or not tool.exists():
|
||||
return uvk5_testenv.skip("%s is missing (%s); see the README Quick start"
|
||||
% (what, tool or "not found"))
|
||||
|
||||
port = 1262
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
img = pathlib.Path(tmp) / "flash.img"
|
||||
img.write_bytes(gzip.decompress(PRISTINE.read_bytes()))
|
||||
sock = pathlib.Path(tmp) / "qmp.sock"
|
||||
sock = uvk5_socket.server_endpoint("qmp", directory=str(tmp))
|
||||
|
||||
proc = subprocess.Popen(
|
||||
[str(QEMU), "-M", f"uv-k5-v3,flash-image={img}",
|
||||
"-nographic", "-monitor", "none",
|
||||
"-qmp", f"unix:{sock},server=on,wait=off",
|
||||
"-qmp", sock,
|
||||
"-kernel", str(ELF), "-gdb", f"tcp::{port}"],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
try:
|
||||
for _ in range(BOOT_SECONDS * 4):
|
||||
if sock.exists():
|
||||
break
|
||||
time.sleep(0.25)
|
||||
else:
|
||||
print("FAIL QMP socket never appeared")
|
||||
return 1
|
||||
# Qmp() below connects, and uvk5_socket retries until QEMU's QMP answers, so
|
||||
# there is no socket path to wait for -- on Windows there would not be one.
|
||||
time.sleep(BOOT_SECONDS)
|
||||
time.sleep(BOOT_SECONDS)
|
||||
|
||||
qmp = Qmp(str(sock))
|
||||
|
||||
+45
-19
@@ -27,10 +27,13 @@ import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
ROOT = os.path.dirname(HERE)
|
||||
QEMU = os.path.expanduser("~/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm")
|
||||
ELF = os.path.expanduser("~/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf")
|
||||
QEMU = uvk5_testenv.qemu() # env QEMU/UVK5_QEMU, else PATH
|
||||
ELF = uvk5_testenv.firmware() # env ELF/UVK5_FIRMWARE, else assets/firmware
|
||||
PRISTINE = os.path.join(ROOT, "assets", "pristine", "flash-pristine.img.gz")
|
||||
|
||||
BOOT_SECONDS = 20
|
||||
@@ -41,10 +44,14 @@ REG_RSSI = 0x67
|
||||
|
||||
|
||||
class Qmp:
|
||||
def __init__(self, path):
|
||||
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
self.sock.settimeout(30)
|
||||
self.sock.connect(path)
|
||||
def __init__(self, endpoint):
|
||||
# A socket or an endpoint. QEMU's QMP accepts a single client, so whoever
|
||||
# waited for it to appear hands its connection in rather than connecting a
|
||||
# second time -- which hangs.
|
||||
if hasattr(endpoint, "recv"):
|
||||
self.sock = endpoint
|
||||
else:
|
||||
self.sock = uvk5_socket.connect(endpoint, timeout=30)
|
||||
self.buf = b""
|
||||
self._read()
|
||||
self.cmd("qmp_capabilities")
|
||||
@@ -76,32 +83,51 @@ class Qmp:
|
||||
|
||||
|
||||
def main():
|
||||
for path, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine image")):
|
||||
if not os.path.exists(path):
|
||||
sys.exit(f"missing {what}: {path}")
|
||||
absent = uvk5_testenv.missing([
|
||||
(QEMU, "QEMU", "set QEMU=/path/to/qemu-system-arm, or put it on PATH"),
|
||||
(ELF, "firmware", "run tools/fetch_firmware.py, or set ELF=/path/to/image"),
|
||||
(PRISTINE, "pristine flash image", "it ships in assets/pristine/"),
|
||||
])
|
||||
if absent:
|
||||
return uvk5_testenv.skip(absent)
|
||||
|
||||
workdir = tempfile.mkdtemp(prefix="uvk5-bk4819-")
|
||||
image = os.path.join(workdir, "flash.img")
|
||||
sock_path = os.path.join(workdir, "qmp.sock")
|
||||
sock_path = uvk5_socket.server_endpoint("qmp", directory=workdir)
|
||||
with gzip.open(PRISTINE, "rb") as src, open(image, "wb") as dst:
|
||||
shutil.copyfileobj(src, dst)
|
||||
|
||||
child_env = dict(os.environ)
|
||||
child_env["UVK5_FLASH_IMAGE"] = image
|
||||
# stderr to a FILE, not a pipe: the model writes the firmware's serial output
|
||||
# there, and a pipe nobody drains fills up, blocks the guest, and then QMP stops
|
||||
# answering -- which reads as "the emulator never started".
|
||||
qemu_log = os.path.join(workdir, "qemu.log")
|
||||
log_fh = open(qemu_log, "w+b")
|
||||
proc = subprocess.Popen(
|
||||
[QEMU, "-M", f"uv-k5-v3,flash-image={image}", "-nographic", "-monitor", "none",
|
||||
"-qmp", f"unix:{sock_path},server=on,wait=off", "-kernel", ELF],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
|
||||
[QEMU, "-M", "uv-k5-v3", "-nographic", "-monitor", "none",
|
||||
"-qmp", sock_path, "-kernel", ELF],
|
||||
stdout=subprocess.DEVNULL, stderr=log_fh, env=child_env)
|
||||
|
||||
failures = []
|
||||
try:
|
||||
for _ in range(300):
|
||||
if os.path.exists(sock_path):
|
||||
qmp_sock = None
|
||||
deadline = time.monotonic() + 60
|
||||
while time.monotonic() < deadline:
|
||||
if proc.poll() is not None:
|
||||
raise RuntimeError("QEMU exited during startup")
|
||||
try:
|
||||
# Connecting is the test: a unix path can be waited for as a file,
|
||||
# a TCP endpoint cannot, and this works for both.
|
||||
qmp_sock = uvk5_socket.connect(sock_path, timeout=2)
|
||||
break
|
||||
time.sleep(0.1)
|
||||
else:
|
||||
raise RuntimeError("QMP socket never appeared")
|
||||
except OSError:
|
||||
time.sleep(0.1)
|
||||
if qmp_sock is None:
|
||||
raise RuntimeError("QMP never accepted a connection at %s" % sock_path)
|
||||
|
||||
time.sleep(BOOT_SECONDS)
|
||||
qmp = Qmp(sock_path)
|
||||
qmp = Qmp(qmp_sock)
|
||||
|
||||
# 1. Still running means the untimed REG_0C spin terminated.
|
||||
status = qmp.cmd("query-status").get("return", {})
|
||||
|
||||
@@ -35,11 +35,11 @@ trap 'cp /tmp/bk-readback-orig.c "$SRC"; cp "$SRC" "$QSRC" 2>/dev/null || true;
|
||||
python3 - "$SRC" "$SEED" <<'PY'
|
||||
import sys
|
||||
src, seed = sys.argv[1], sys.argv[2]
|
||||
s = open(src).read()
|
||||
s = open(src, encoding="utf-8").read()
|
||||
needle = " s->regs[BK4819_REG_NOISE] = 0x0010;"
|
||||
if needle not in s:
|
||||
sys.exit("seed point not found; has bk4819_seed_measurements changed?")
|
||||
open(src, "w").write(s.replace(needle, f"{needle}\n s->regs[0x0C] = {seed};", 1))
|
||||
open(src, "w", encoding="utf-8").write(s.replace(needle, f"{needle}\n s->regs[0x0C] = {seed};", 1))
|
||||
PY
|
||||
|
||||
cp "$SRC" "$QSRC"
|
||||
|
||||
+20
-19
@@ -31,13 +31,16 @@ import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
SIM = os.path.dirname(HERE)
|
||||
QEMU = os.path.expanduser("~/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm")
|
||||
ELF = os.path.expanduser("~/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf")
|
||||
QEMU = uvk5_testenv.qemu()
|
||||
ELF = uvk5_testenv.firmware()
|
||||
SOURCE_IMAGE = os.path.join(SIM, "assets", "flash.img")
|
||||
|
||||
QMP = "/tmp/uvk5-persist-test.sock"
|
||||
QMP = uvk5_socket.server_endpoint("qmp")
|
||||
|
||||
# Flash offsets the firmware demonstrably writes during a boot, measured rather than
|
||||
# guessed. The mapping is in App/driver/eeprom_compat.c: these are *flash* addresses,
|
||||
@@ -59,10 +62,14 @@ MUST_NOT_CHANGE = [("vfo frequencies", 0x009000, 0xD6)]
|
||||
|
||||
|
||||
class Qmp:
|
||||
def __init__(self, path):
|
||||
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
def __init__(self, endpoint):
|
||||
# A socket or an endpoint: QEMU's QMP takes one client, so the caller that
|
||||
# waited for it hands its connection in.
|
||||
if hasattr(endpoint, "recv"):
|
||||
self.sock = endpoint
|
||||
else:
|
||||
self.sock = uvk5_socket.connect(endpoint, timeout=30)
|
||||
self.sock.settimeout(25)
|
||||
self.sock.connect(path)
|
||||
self.buf = b""
|
||||
self._readline()
|
||||
self.command("qmp_capabilities")
|
||||
@@ -100,17 +107,12 @@ def boot(image):
|
||||
os.unlink(QMP)
|
||||
proc = subprocess.Popen(
|
||||
[QEMU, "-M", f"uv-k5-v3,flash-image={image}", "-nographic",
|
||||
"-monitor", "none", "-qmp", f"unix:{QMP},server=on,wait=off",
|
||||
"-monitor", "none", "-qmp", QMP,
|
||||
"-kernel", ELF],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
deadline = time.time() + 25
|
||||
while time.time() < deadline:
|
||||
if os.path.exists(QMP):
|
||||
return proc, Qmp(QMP)
|
||||
time.sleep(0.1)
|
||||
proc.kill()
|
||||
raise RuntimeError("QMP socket never appeared")
|
||||
|
||||
# Qmp() connects, and uvk5_socket retries until QEMU's QMP answers, so there is no
|
||||
# socket path to wait for -- and on Windows there would not be one.
|
||||
return proc, Qmp(QMP)
|
||||
|
||||
def shutdown(proc, qmp):
|
||||
"""Quit through QMP, which is exactly what the web UI's power off does."""
|
||||
@@ -142,10 +144,9 @@ def snapshot(path):
|
||||
|
||||
|
||||
def main():
|
||||
for path, what in ((QEMU, "QEMU"), (ELF, "firmware"),
|
||||
(SOURCE_IMAGE, "flash image")):
|
||||
if not os.path.exists(path):
|
||||
sys.exit(f"missing {what}: {path}")
|
||||
_missing = uvk5_testenv.missing([(p, w, "see the README Quick start") for p, w in ((QEMU, "QEMU"), (ELF, "firmware"))])
|
||||
if _missing:
|
||||
return uvk5_testenv.skip(_missing)
|
||||
|
||||
workdir = tempfile.mkdtemp(prefix="uvk5-persist-")
|
||||
image = os.path.join(workdir, "flash.img")
|
||||
|
||||
+20
-17
@@ -32,10 +32,13 @@ import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
ROOT = os.path.dirname(HERE)
|
||||
QEMU = os.path.expanduser("~/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm")
|
||||
ELF = os.path.expanduser("~/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf")
|
||||
QEMU = uvk5_testenv.qemu()
|
||||
ELF = uvk5_testenv.firmware()
|
||||
PRISTINE = os.path.join(ROOT, "assets", "pristine", "flash-pristine.img.gz")
|
||||
|
||||
BOOT_SECONDS = 20
|
||||
@@ -50,10 +53,14 @@ WANT_BAND = 5 # 400-470 MHz contains 435
|
||||
|
||||
|
||||
class Qmp:
|
||||
def __init__(self, path):
|
||||
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
def __init__(self, endpoint):
|
||||
# A socket or an endpoint: QEMU's QMP takes one client, so the caller that
|
||||
# waited for it hands its connection in.
|
||||
if hasattr(endpoint, "recv"):
|
||||
self.sock = endpoint
|
||||
else:
|
||||
self.sock = uvk5_socket.connect(endpoint, timeout=30)
|
||||
self.sock.settimeout(30)
|
||||
self.sock.connect(path)
|
||||
self.buf = b""
|
||||
self._read() # greeting
|
||||
self.cmd("qmp_capabilities")
|
||||
@@ -95,16 +102,10 @@ def boot(image, sock_path):
|
||||
os.unlink(sock_path)
|
||||
proc = subprocess.Popen(
|
||||
[QEMU, "-M", f"uv-k5-v3,flash-image={image}", "-nographic",
|
||||
"-monitor", "none", "-qmp", f"unix:{sock_path},server=on,wait=off",
|
||||
"-monitor", "none", "-qmp", sock_path,
|
||||
"-kernel", ELF],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
for _ in range(300):
|
||||
if os.path.exists(sock_path):
|
||||
break
|
||||
time.sleep(0.1)
|
||||
else:
|
||||
proc.kill()
|
||||
raise RuntimeError("QMP socket never appeared")
|
||||
# Qmp() below connects; uvk5_socket retries until QEMU answers.
|
||||
time.sleep(BOOT_SECONDS)
|
||||
return proc
|
||||
|
||||
@@ -129,13 +130,15 @@ def stored_frequency(image, band, vfo=0):
|
||||
|
||||
|
||||
def main():
|
||||
for path, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine image")):
|
||||
if not os.path.exists(path):
|
||||
sys.exit(f"missing {what}: {path}")
|
||||
for path, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine flash image")):
|
||||
if path is None or not os.path.exists(path):
|
||||
print("SKIP: %s is missing (%s); see the README Quick start"
|
||||
% (what, path or "not found"))
|
||||
return 0
|
||||
|
||||
workdir = tempfile.mkdtemp(prefix="uvk5-freq-")
|
||||
image = os.path.join(workdir, "flash.img")
|
||||
sock_path = os.path.join(workdir, "qmp.sock")
|
||||
sock_path = uvk5_socket.server_endpoint("qmp", directory=workdir)
|
||||
with gzip.open(PRISTINE, "rb") as src, open(image, "wb") as dst:
|
||||
shutil.copyfileobj(src, dst)
|
||||
|
||||
|
||||
+20
-21
@@ -28,11 +28,12 @@ import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
|
||||
SIM = pathlib.Path(__file__).resolve().parent.parent
|
||||
QEMU = pathlib.Path(os.environ.get(
|
||||
"QEMU", "/root/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm"))
|
||||
ELF = pathlib.Path(os.environ.get(
|
||||
"ELF", "/root/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf"))
|
||||
QEMU = uvk5_testenv.qemu()
|
||||
ELF = uvk5_testenv.firmware()
|
||||
PRISTINE = SIM / "assets/pristine/flash-pristine.img.gz"
|
||||
|
||||
BOOT_SECONDS = 24
|
||||
@@ -42,9 +43,7 @@ GAP = 5.0
|
||||
|
||||
class Qmp:
|
||||
def __init__(self, path):
|
||||
self.s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
self.s.settimeout(25)
|
||||
self.s.connect(path)
|
||||
self.s = uvk5_socket.connect(path, timeout=25)
|
||||
self.buf = b""
|
||||
self._read()
|
||||
self.cmd("qmp_capabilities")
|
||||
@@ -76,7 +75,7 @@ def read_counter(port):
|
||||
on access, not stored, so dumping memory another way would miss it.
|
||||
"""
|
||||
out = subprocess.run(
|
||||
["gdb-multiarch", "-batch",
|
||||
[str(uvk5_testenv.gdb()), "-batch",
|
||||
"-ex", "set confirm off", "-ex", "set pagination off",
|
||||
"-ex", f"target remote :{port}",
|
||||
"-ex", f'printf "CNT=%u\\n", *(unsigned int*){TIM2_CNT}',
|
||||
@@ -89,31 +88,31 @@ def read_counter(port):
|
||||
|
||||
|
||||
def main():
|
||||
for tool in (QEMU, ELF, PRISTINE):
|
||||
if not tool.exists():
|
||||
print(f"SKIP missing {tool}")
|
||||
return 0
|
||||
if uvk5_testenv.gdb() is None:
|
||||
return uvk5_testenv.skip("gdb-multiarch is missing; this test reads firmware "
|
||||
"globals over a gdb attach, which has not been ported to "
|
||||
"the QMP memsave route the page uses")
|
||||
for tool, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine flash image")):
|
||||
if tool is None or not tool.exists():
|
||||
return uvk5_testenv.skip("%s is missing (%s); see the README Quick start"
|
||||
% (what, tool or "not found"))
|
||||
|
||||
port = 1263
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
img = pathlib.Path(tmp) / "flash.img"
|
||||
img.write_bytes(gzip.decompress(PRISTINE.read_bytes()))
|
||||
sock = pathlib.Path(tmp) / "qmp.sock"
|
||||
sock = uvk5_socket.server_endpoint("qmp", directory=str(tmp))
|
||||
|
||||
proc = subprocess.Popen(
|
||||
[str(QEMU), "-M", f"uv-k5-v3,flash-image={img}",
|
||||
"-nographic", "-monitor", "none",
|
||||
"-qmp", f"unix:{sock},server=on,wait=off",
|
||||
"-qmp", sock,
|
||||
"-kernel", str(ELF), "-gdb", f"tcp::{port}"],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
try:
|
||||
for _ in range(BOOT_SECONDS * 4):
|
||||
if sock.exists():
|
||||
break
|
||||
time.sleep(0.25)
|
||||
else:
|
||||
print("FAIL QMP socket never appeared")
|
||||
return 1
|
||||
# Qmp() below connects, and uvk5_socket retries until QEMU's QMP answers, so
|
||||
# there is no socket path to wait for -- on Windows there would not be one.
|
||||
time.sleep(BOOT_SECONDS)
|
||||
time.sleep(BOOT_SECONDS)
|
||||
|
||||
failures = 0
|
||||
|
||||
+26
-21
@@ -30,11 +30,12 @@ import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
|
||||
SIM = pathlib.Path(__file__).resolve().parent.parent
|
||||
QEMU = pathlib.Path(os.environ.get(
|
||||
"QEMU", "/root/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm"))
|
||||
ELF = pathlib.Path(os.environ.get(
|
||||
"ELF", "/root/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf"))
|
||||
QEMU = uvk5_testenv.qemu()
|
||||
ELF = uvk5_testenv.firmware()
|
||||
PRISTINE = SIM / "assets/pristine/flash-pristine.img.gz"
|
||||
|
||||
FRAME_ADDR = 0x200013DC
|
||||
@@ -45,10 +46,14 @@ FUNCTION_TRANSMIT = 1
|
||||
|
||||
|
||||
class Qmp:
|
||||
def __init__(self, path):
|
||||
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
def __init__(self, endpoint):
|
||||
# A socket or an endpoint: QEMU's QMP takes one client, so the caller that
|
||||
# waited for it hands its connection in.
|
||||
if hasattr(endpoint, "recv"):
|
||||
self.sock = endpoint
|
||||
else:
|
||||
self.sock = uvk5_socket.connect(endpoint, timeout=30)
|
||||
self.sock.settimeout(25)
|
||||
self.sock.connect(path)
|
||||
self.buf = b""
|
||||
self._read()
|
||||
self.cmd("qmp_capabilities")
|
||||
@@ -98,7 +103,7 @@ def function_value(elf, port):
|
||||
not anything timing-dependent. Key injection would be a different matter.
|
||||
"""
|
||||
out = subprocess.run(
|
||||
["gdb-multiarch", "-batch",
|
||||
[str(uvk5_testenv.gdb()), "-batch",
|
||||
"-ex", "set confirm off", "-ex", "set pagination off",
|
||||
"-ex", f"target remote :{port}",
|
||||
"-ex", 'printf "FN=%d\\n", *(unsigned char*)&gCurrentFunction',
|
||||
@@ -111,31 +116,31 @@ def function_value(elf, port):
|
||||
|
||||
|
||||
def main():
|
||||
for tool in (QEMU, ELF, PRISTINE):
|
||||
if not tool.exists():
|
||||
print(f"SKIP missing {tool}")
|
||||
return 0
|
||||
if uvk5_testenv.gdb() is None:
|
||||
return uvk5_testenv.skip("gdb-multiarch is missing; this test reads firmware "
|
||||
"globals over a gdb attach, which has not been ported to "
|
||||
"the QMP memsave route the page uses")
|
||||
for tool, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine flash image")):
|
||||
if tool is None or not tool.exists():
|
||||
return uvk5_testenv.skip("%s is missing (%s); see the README Quick start"
|
||||
% (what, tool or "not found"))
|
||||
|
||||
port = 1261
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
img = pathlib.Path(tmp) / "flash.img"
|
||||
img.write_bytes(gzip.decompress(PRISTINE.read_bytes()))
|
||||
sock = pathlib.Path(tmp) / "qmp.sock"
|
||||
sock = uvk5_socket.server_endpoint("qmp", directory=str(tmp))
|
||||
|
||||
proc = subprocess.Popen(
|
||||
[str(QEMU), "-M", f"uv-k5-v3,flash-image={img}",
|
||||
"-nographic", "-monitor", "none",
|
||||
"-qmp", f"unix:{sock},server=on,wait=off",
|
||||
"-qmp", sock,
|
||||
"-kernel", str(ELF), "-gdb", f"tcp::{port}"],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
try:
|
||||
for _ in range(BOOT_SECONDS * 4):
|
||||
if sock.exists():
|
||||
break
|
||||
time.sleep(0.25)
|
||||
else:
|
||||
print("FAIL QMP socket never appeared")
|
||||
return 1
|
||||
# Qmp() below connects, and uvk5_socket retries until QEMU's QMP answers, so
|
||||
# there is no socket path to wait for -- on Windows there would not be one.
|
||||
time.sleep(BOOT_SECONDS)
|
||||
time.sleep(BOOT_SECONDS)
|
||||
|
||||
qmp = Qmp(str(sock))
|
||||
|
||||
+15
-20
@@ -25,11 +25,12 @@ import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
|
||||
SIM = pathlib.Path(__file__).resolve().parent.parent
|
||||
QEMU = pathlib.Path(os.environ.get(
|
||||
"QEMU", "/root/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm"))
|
||||
ELF = pathlib.Path(os.environ.get(
|
||||
"ELF", "/root/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf"))
|
||||
QEMU = uvk5_testenv.qemu()
|
||||
ELF = uvk5_testenv.firmware()
|
||||
PRISTINE = SIM / "assets/pristine/flash-pristine.img.gz"
|
||||
|
||||
FRAME_ADDR = 0x200013DC
|
||||
@@ -44,9 +45,7 @@ SAMPLE_GAP = 1.5
|
||||
|
||||
class Qmp:
|
||||
def __init__(self, path):
|
||||
self.s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
self.s.settimeout(25)
|
||||
self.s.connect(path)
|
||||
self.s = uvk5_socket.connect(path, timeout=25)
|
||||
self.buf = b""
|
||||
self._read()
|
||||
self.cmd("qmp_capabilities")
|
||||
@@ -84,30 +83,26 @@ class Qmp:
|
||||
|
||||
|
||||
def main():
|
||||
for tool in (QEMU, ELF, PRISTINE):
|
||||
if not tool.exists():
|
||||
print(f"SKIP missing {tool}")
|
||||
return 0
|
||||
for tool, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine flash image")):
|
||||
if tool is None or not tool.exists():
|
||||
return uvk5_testenv.skip("%s is missing (%s); see the README Quick start"
|
||||
% (what, tool or "not found"))
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
img = pathlib.Path(tmp) / "flash.img"
|
||||
img.write_bytes(gzip.decompress(PRISTINE.read_bytes()))
|
||||
sock = pathlib.Path(tmp) / "qmp.sock"
|
||||
sock = uvk5_socket.server_endpoint("qmp", directory=str(tmp))
|
||||
|
||||
proc = subprocess.Popen(
|
||||
[str(QEMU), "-M", f"uv-k5-v3,flash-image={img}",
|
||||
"-nographic", "-monitor", "none",
|
||||
"-qmp", f"unix:{sock},server=on,wait=off",
|
||||
"-qmp", sock,
|
||||
"-kernel", str(ELF)],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
try:
|
||||
for _ in range(BOOT_SECONDS * 4):
|
||||
if sock.exists():
|
||||
break
|
||||
time.sleep(0.25)
|
||||
else:
|
||||
print("FAIL QMP socket never appeared")
|
||||
return 1
|
||||
# Qmp() below connects, and uvk5_socket retries until QEMU's QMP answers, so
|
||||
# there is no socket path to wait for -- on Windows there would not be one.
|
||||
time.sleep(BOOT_SECONDS)
|
||||
time.sleep(BOOT_SECONDS)
|
||||
|
||||
qmp = Qmp(str(sock))
|
||||
|
||||
@@ -50,7 +50,9 @@ def main():
|
||||
if os.path.exists(QMP):
|
||||
os.unlink(QMP)
|
||||
|
||||
text = open(LOG, errors="replace").read()
|
||||
# Explicit UTF-8: the firmware's own output is not always ASCII, and the locale
|
||||
# codec on Windows would mangle it (it cannot decode Chinese at all).
|
||||
text = open(LOG, encoding="utf-8", errors="replace").read()
|
||||
lines = [l for l in text.splitlines() if l.startswith("SERIAL")]
|
||||
print(f"captured {len(lines)} SERIAL line(s)")
|
||||
for line in lines[:10]:
|
||||
|
||||
+16
-10
@@ -28,10 +28,13 @@ import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
ROOT = os.path.dirname(HERE)
|
||||
QEMU = os.path.expanduser("~/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm")
|
||||
ELF = os.path.expanduser("~/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf")
|
||||
QEMU = uvk5_testenv.qemu()
|
||||
ELF = uvk5_testenv.firmware()
|
||||
PRISTINE = os.path.join(ROOT, "assets", "pristine", "flash-pristine.img.gz")
|
||||
|
||||
BOOT_SECONDS = 20
|
||||
@@ -81,9 +84,11 @@ def parse_frames(buf: bytes):
|
||||
|
||||
|
||||
def main():
|
||||
for path, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine image")):
|
||||
if not os.path.exists(path):
|
||||
sys.exit(f"missing {what}: {path}")
|
||||
for path, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine flash image")):
|
||||
if path is None or not os.path.exists(path):
|
||||
print("SKIP: %s is missing (%s); see the README Quick start"
|
||||
% (what, path or "not found"))
|
||||
return 0
|
||||
|
||||
workdir = tempfile.mkdtemp(prefix="uvk5-serial-")
|
||||
image = os.path.join(workdir, "flash.img")
|
||||
@@ -91,15 +96,16 @@ def main():
|
||||
with gzip.open(PRISTINE, "rb") as src, open(image, "wb") as dst:
|
||||
shutil.copyfileobj(src, dst)
|
||||
|
||||
# A listening socket for QEMU's serial chardev to connect back to.
|
||||
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
srv.bind(sock_path)
|
||||
srv.listen(1)
|
||||
# A listening socket for QEMU's serial chardev to connect back to. Through
|
||||
# uvk5_socket, so this is a unix socket where the platform has them and TCP where
|
||||
# it does not -- a Windows QEMU cannot create a unix one, and this test is part of
|
||||
# the verification path that has to work wherever the emulator does.
|
||||
srv, serial_endpoint = uvk5_socket.listen("serial")
|
||||
srv.settimeout(40)
|
||||
|
||||
proc = subprocess.Popen(
|
||||
[QEMU, "-M", f"uv-k5-v3,flash-image={image}", "-nographic", "-monitor", "none",
|
||||
"-serial", f"unix:{sock_path}", "-kernel", ELF],
|
||||
"-serial", serial_endpoint, "-kernel", ELF],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
|
||||
failures = []
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
#!/usr/bin/env python3
|
||||
"""A host tool can write a firmware slot through the firmware's own commands.
|
||||
|
||||
The firmware exposes slot management over its programming port (App/app/uart.c, the
|
||||
"Firmware Slots" block): 0x0720 reads a header, 0x0722 erases a slot, 0x0724 programs
|
||||
bytes at an offset, 0x0726 returns the image CRC. tools/uvk5_slots_serial.py speaks
|
||||
them, and this boots a real instance to check that a slot written that way lands in
|
||||
the external flash with a header the firmware itself validates.
|
||||
|
||||
Four things have to line up, and each failed silently on its own before it did:
|
||||
|
||||
- The first 0x0514 is often swallowed, so the handshake is retried.
|
||||
- A frame carries 8 bytes of framing, 4 of header and 12 of payload before the data,
|
||||
and the receive buffer is 256 bytes (App/driver/uart.c: UART_DMA_Buffer[256]), so
|
||||
chunks are 200 bytes.
|
||||
- The serial session times out after ~6 s without a 0x0514
|
||||
(gSerialConfigCountDown_500ms = 12), which a transfer of any size crosses, so the
|
||||
session is renewed as it goes.
|
||||
- K5Viewer streams the screen down the same link, so the port has to be drained
|
||||
continuously; a client that reads only while waiting for a reply blocks the guest.
|
||||
|
||||
Needs a build with ENABLE_FEAT_F4HWN_MULTIBOOT -- the slot commands simply are not
|
||||
there without it -- so it skips unless one is pointed at with UVK5_MULTIBOOT_IMAGE,
|
||||
and unless a QEMU is known (UVK5_QEMU, or one on PATH).
|
||||
"""
|
||||
import gzip
|
||||
import os
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import unittest
|
||||
import zlib
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
ROOT = os.path.dirname(HERE)
|
||||
sys.path.insert(0, HERE)
|
||||
|
||||
import uvk5_slots as slots # noqa: E402
|
||||
import uvk5_slots_serial as serial # noqa: E402
|
||||
|
||||
PRISTINE = os.path.join(ROOT, "assets", "pristine", "flash-pristine.img.gz")
|
||||
BOOT_SECONDS = 20
|
||||
SLOT = 3
|
||||
IMAGE = bytes((i * 7 + 3) & 0xFF for i in range(4096)) # small, recognisable
|
||||
|
||||
|
||||
def qemu_path():
|
||||
"""QEMU, from QEMU (the runner's variable) or UVK5_QEMU, else from PATH."""
|
||||
for var in ("QEMU", "UVK5_QEMU"):
|
||||
env = os.environ.get(var)
|
||||
if env and os.path.exists(env):
|
||||
return env
|
||||
return shutil.which("qemu-system-arm")
|
||||
|
||||
|
||||
def multiboot_image():
|
||||
"""Where a build with ENABLE_FEAT_F4HWN_MULTIBOOT might be.
|
||||
|
||||
assets/firmware/ is what tools/fetch_firmware.py fills; work/ is where hand-kept
|
||||
images live. Neither is in the repository, so the test skips rather than pretends.
|
||||
"""
|
||||
env = os.environ.get("UVK5_MULTIBOOT_IMAGE")
|
||||
if env and os.path.exists(env):
|
||||
return env
|
||||
for candidate in (os.path.join(ROOT, "assets", "firmware", "f4hwn.fieldops.v6.0.0.bin"),
|
||||
os.path.join(ROOT, "assets", "firmware", "f4hwn.fusion.v6.0.0.bin"),
|
||||
os.path.join(ROOT, "work", "multiboot.bin")):
|
||||
if os.path.exists(candidate):
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
def free_port():
|
||||
"""A port nothing is listening on, for QEMU to listen on instead.
|
||||
|
||||
TCP, not a unix socket: a Windows QEMU cannot create one, and the point of this
|
||||
test is that it runs wherever the emulator does.
|
||||
"""
|
||||
probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
probe.bind(("127.0.0.1", 0))
|
||||
port = probe.getsockname()[1]
|
||||
probe.close()
|
||||
return port
|
||||
|
||||
|
||||
class TestSlotOverSerial(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.qemu = qemu_path()
|
||||
if not self.qemu:
|
||||
self.skipTest("no qemu-system-arm (set UVK5_QEMU)")
|
||||
self.firmware = multiboot_image()
|
||||
if not self.firmware:
|
||||
self.skipTest("no multi-system build: run tools/fetch_firmware.py, or set "
|
||||
"UVK5_MULTIBOOT_IMAGE. The slot commands do not exist "
|
||||
"without ENABLE_FEAT_F4HWN_MULTIBOOT")
|
||||
if not os.path.exists(PRISTINE):
|
||||
self.skipTest("assets/pristine/flash-pristine.img.gz is missing")
|
||||
|
||||
self.tmp = tempfile.mkdtemp(prefix="uvk5-slotserial-")
|
||||
self.log = open(os.path.join(self.tmp, "qemu.log"), "w+b")
|
||||
self.image = os.path.join(self.tmp, "flash.img")
|
||||
with gzip.open(PRISTINE, "rb") as src, open(self.image, "wb") as dst:
|
||||
shutil.copyfileobj(src, dst)
|
||||
|
||||
port = free_port()
|
||||
env = dict(os.environ)
|
||||
env["UVK5_FLASH_IMAGE"] = self.image
|
||||
self.proc = subprocess.Popen(
|
||||
[self.qemu, "-M", "uv-k5-v3", "-nographic", "-monitor", "none",
|
||||
"-serial", "tcp:127.0.0.1:%d,server=on,wait=off" % port,
|
||||
"-kernel", self.firmware],
|
||||
stdout=subprocess.DEVNULL, stderr=self.log, env=env)
|
||||
# QEMU listens; we connect, retrying while it comes up.
|
||||
deadline = time.monotonic() + BOOT_SECONDS + 40
|
||||
self.conn = None
|
||||
while time.monotonic() < deadline:
|
||||
if self.proc.poll() is not None:
|
||||
break
|
||||
try:
|
||||
self.conn = socket.create_connection(("127.0.0.1", port), timeout=2)
|
||||
break
|
||||
except OSError:
|
||||
time.sleep(0.2)
|
||||
if self.conn is None:
|
||||
# Say why, and keep QEMU's own output: a native Windows QEMU dies at load
|
||||
# unless its DLL directory is on PATH, and with stderr discarded that looks
|
||||
# exactly like a machine that never listened.
|
||||
self.fail("no connection on 127.0.0.1:%d (qemu exit %r)\n%s\n"
|
||||
"%s" % (port, self.proc.poll(), self._qemu_output(),
|
||||
"on Windows the MSYS2 mingw64 bin directory has to be "
|
||||
"on PATH, or QEMU cannot load its DLLs"))
|
||||
time.sleep(BOOT_SECONDS)
|
||||
|
||||
def _qemu_output(self):
|
||||
try:
|
||||
self.log.flush()
|
||||
with open(self.log.name, "rb") as fh:
|
||||
return fh.read()[-2000:].decode("utf-8", "replace")
|
||||
except OSError:
|
||||
return "(no QEMU output)"
|
||||
|
||||
def tearDown(self):
|
||||
try:
|
||||
if self.conn is not None:
|
||||
self.conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
self.proc.terminate()
|
||||
try:
|
||||
self.proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
self.proc.kill()
|
||||
try:
|
||||
self.log.close()
|
||||
except Exception:
|
||||
pass
|
||||
shutil.rmtree(self.tmp, ignore_errors=True)
|
||||
|
||||
def test_an_erased_and_written_slot_validates(self):
|
||||
radio = serial.Radio(self.conn, log=lambda *a: None)
|
||||
try:
|
||||
radio.session()
|
||||
self.assertEqual(radio.erase(SLOT), 0, "the firmware refused to erase the slot")
|
||||
serial.install(radio, SLOT, IMAGE, "test-image", "1.2.3", log=lambda *a: None)
|
||||
status, crc = radio.validate(SLOT)
|
||||
self.assertEqual(status, 0)
|
||||
self.assertEqual(crc, zlib.crc32(IMAGE) & 0xFFFFFFFF)
|
||||
finally:
|
||||
radio.close()
|
||||
|
||||
def test_the_slot_lands_in_the_flash_image_on_disk(self):
|
||||
radio = serial.Radio(self.conn, log=lambda *a: None)
|
||||
try:
|
||||
radio.session()
|
||||
radio.erase(SLOT)
|
||||
serial.install(radio, SLOT, IMAGE, "test-image", "1.2.3", log=lambda *a: None)
|
||||
finally:
|
||||
radio.close()
|
||||
# The model writes the changed range back as it goes, so the file is current
|
||||
# without waiting for the emulator to exit.
|
||||
with open(self.image, "rb") as fh:
|
||||
buf = fh.read()
|
||||
header, image = slots.read_slot(buf, SLOT)
|
||||
self.assertIsNotNone(header, "slot %d is not committed in the image" % SLOT)
|
||||
self.assertEqual(header["name"], "test-image")
|
||||
self.assertEqual(header["image_size"], len(IMAGE))
|
||||
self.assertTrue(header["crc_ok"], "the header CRC does not describe the image")
|
||||
self.assertEqual(image, IMAGE)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
+21
-20
@@ -37,12 +37,13 @@ import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
|
||||
HERE = pathlib.Path(__file__).resolve().parent
|
||||
SIM = HERE.parent
|
||||
QEMU = pathlib.Path(os.environ.get(
|
||||
"QEMU", "/root/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm"))
|
||||
ELF = pathlib.Path(os.environ.get(
|
||||
"ELF", "/root/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf"))
|
||||
QEMU = uvk5_testenv.qemu()
|
||||
ELF = uvk5_testenv.firmware()
|
||||
PRISTINE = SIM / "assets/pristine/flash-pristine.img.gz"
|
||||
|
||||
FRAME_ADDR = 0x200013DC
|
||||
@@ -51,10 +52,14 @@ BOOT_SECONDS = 24
|
||||
|
||||
|
||||
class Qmp:
|
||||
def __init__(self, path):
|
||||
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
def __init__(self, endpoint):
|
||||
# A socket or an endpoint: QEMU's QMP takes one client, so the caller that
|
||||
# waited for it hands its connection in.
|
||||
if hasattr(endpoint, "recv"):
|
||||
self.sock = endpoint
|
||||
else:
|
||||
self.sock = uvk5_socket.connect(endpoint, timeout=30)
|
||||
self.sock.settimeout(25)
|
||||
self.sock.connect(path)
|
||||
self.buf = b""
|
||||
self._read()
|
||||
self.cmd("qmp_capabilities")
|
||||
@@ -97,30 +102,26 @@ class Qmp:
|
||||
|
||||
|
||||
def main():
|
||||
for tool in (QEMU, ELF, PRISTINE):
|
||||
if not tool.exists():
|
||||
print(f"SKIP missing {tool}")
|
||||
return 0
|
||||
for tool, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine flash image")):
|
||||
if tool is None or not tool.exists():
|
||||
return uvk5_testenv.skip("%s is missing (%s); see the README Quick start"
|
||||
% (what, tool or "not found"))
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
img = pathlib.Path(tmp) / "flash.img"
|
||||
img.write_bytes(gzip.decompress(PRISTINE.read_bytes()))
|
||||
sock = pathlib.Path(tmp) / "qmp.sock"
|
||||
sock = uvk5_socket.server_endpoint("qmp", directory=str(tmp))
|
||||
|
||||
proc = subprocess.Popen(
|
||||
[str(QEMU), "-M", f"uv-k5-v3,flash-image={img}",
|
||||
"-nographic", "-monitor", "none",
|
||||
"-qmp", f"unix:{sock},server=on,wait=off",
|
||||
"-qmp", sock,
|
||||
"-kernel", str(ELF)],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
try:
|
||||
for _ in range(BOOT_SECONDS * 4):
|
||||
if sock.exists():
|
||||
break
|
||||
time.sleep(0.25)
|
||||
else:
|
||||
print("FAIL QMP socket never appeared")
|
||||
return 1
|
||||
# Qmp() below connects, and uvk5_socket retries until QEMU's QMP answers, so
|
||||
# there is no socket path to wait for -- on Windows there would not be one.
|
||||
time.sleep(BOOT_SECONDS)
|
||||
time.sleep(BOOT_SECONDS)
|
||||
|
||||
qmp = Qmp(str(sock))
|
||||
|
||||
+15
-20
@@ -29,11 +29,12 @@ import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
import uvk5_socket
|
||||
import uvk5_testenv
|
||||
|
||||
SIM = pathlib.Path(__file__).resolve().parent.parent
|
||||
QEMU = pathlib.Path(os.environ.get(
|
||||
"QEMU", "/root/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm"))
|
||||
ELF = pathlib.Path(os.environ.get(
|
||||
"ELF", "/root/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf"))
|
||||
QEMU = uvk5_testenv.qemu()
|
||||
ELF = uvk5_testenv.firmware()
|
||||
PRISTINE = SIM / "assets/pristine/flash-pristine.img.gz"
|
||||
|
||||
BOOT_SECONDS = 24
|
||||
@@ -46,9 +47,7 @@ OFF_STATION_HZ10 = 41000000 # 410.000 MHz, several MHz clear of anything
|
||||
|
||||
class Qmp:
|
||||
def __init__(self, path):
|
||||
self.s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
self.s.settimeout(25)
|
||||
self.s.connect(path)
|
||||
self.s = uvk5_socket.connect(path, timeout=25)
|
||||
self.buf = b""
|
||||
self._read()
|
||||
self.cmd("qmp_capabilities")
|
||||
@@ -104,30 +103,26 @@ def rssi_after_tuning(qmp, digits, settle=4):
|
||||
|
||||
|
||||
def main():
|
||||
for tool in (QEMU, ELF, PRISTINE):
|
||||
if not tool.exists():
|
||||
print(f"SKIP missing {tool}")
|
||||
return 0
|
||||
for tool, what in ((QEMU, "QEMU"), (ELF, "firmware"), (PRISTINE, "pristine flash image")):
|
||||
if tool is None or not tool.exists():
|
||||
return uvk5_testenv.skip("%s is missing (%s); see the README Quick start"
|
||||
% (what, tool or "not found"))
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
img = pathlib.Path(tmp) / "flash.img"
|
||||
img.write_bytes(gzip.decompress(PRISTINE.read_bytes()))
|
||||
sock = pathlib.Path(tmp) / "qmp.sock"
|
||||
sock = uvk5_socket.server_endpoint("qmp", directory=str(tmp))
|
||||
|
||||
proc = subprocess.Popen(
|
||||
[str(QEMU), "-M", f"uv-k5-v3,flash-image={img}",
|
||||
"-nographic", "-monitor", "none",
|
||||
"-qmp", f"unix:{sock},server=on,wait=off",
|
||||
"-qmp", sock,
|
||||
"-kernel", str(ELF)],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
try:
|
||||
for _ in range(BOOT_SECONDS * 4):
|
||||
if sock.exists():
|
||||
break
|
||||
time.sleep(0.25)
|
||||
else:
|
||||
print("FAIL QMP socket never appeared")
|
||||
return 1
|
||||
# Qmp() below connects, and uvk5_socket retries until QEMU's QMP answers, so
|
||||
# there is no socket path to wait for -- on Windows there would not be one.
|
||||
time.sleep(BOOT_SECONDS)
|
||||
time.sleep(BOOT_SECONDS)
|
||||
|
||||
qmp = Qmp(str(sock))
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Unit tests for firmware image detection. No emulator needed.
|
||||
|
||||
The distinction matters because getting it wrong is silent: an image loaded at the
|
||||
wrong offset runs 0x2800 bytes off and the first fetch reads whatever data is there.
|
||||
"""
|
||||
import os
|
||||
import struct
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
import uvk5_image as ui
|
||||
|
||||
|
||||
def write(tmp, name, data):
|
||||
path = os.path.join(tmp, name)
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(data)
|
||||
return path
|
||||
|
||||
|
||||
def image(tmp, name, sp, reset, size=0x4000, extra_at_2800=None):
|
||||
"""A file whose first two words are a vector table."""
|
||||
buf = bytearray(size)
|
||||
struct.pack_into("<II", buf, 0, sp, reset)
|
||||
if extra_at_2800 is not None:
|
||||
struct.pack_into("<II", buf, 0x2800, *extra_at_2800)
|
||||
return write(tmp, name, bytes(buf))
|
||||
|
||||
|
||||
class TestDetect(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.mkdtemp()
|
||||
|
||||
def test_application_image(self):
|
||||
"""Reset handler past the application offset: linked for 0x08002800."""
|
||||
path = image(self.tmp, "app.bin", 0x20004000, 0x08002d49)
|
||||
info = ui.detect(path)
|
||||
self.assertEqual(info.kind, "application")
|
||||
self.assertEqual(info.app_offset, ui.APP_OFFSET)
|
||||
|
||||
def test_full_flash_image(self):
|
||||
"""Entry point inside the bootloader region: address 0 aliases the base."""
|
||||
path = image(self.tmp, "bl.bin", 0x200032c0, 0x08000901,
|
||||
extra_at_2800=(0x20004000, 0x08002d49))
|
||||
info = ui.detect(path)
|
||||
self.assertEqual(info.kind, "full-flash")
|
||||
self.assertEqual(info.app_offset, 0)
|
||||
|
||||
def test_elf_is_passed_through(self):
|
||||
path = write(self.tmp, "app.elf", b"\x7fELF" + bytes(64))
|
||||
info = ui.detect(path)
|
||||
self.assertEqual(info.kind, "elf")
|
||||
self.assertEqual(info.app_offset, ui.APP_OFFSET)
|
||||
|
||||
def test_rejects_a_file_with_no_vector_table(self):
|
||||
path = write(self.tmp, "junk.bin", b"not a firmware at all" * 8)
|
||||
with self.assertRaises(ui.ImageError):
|
||||
ui.detect(path)
|
||||
|
||||
def test_rejects_an_sp_outside_sram(self):
|
||||
"""A plausible reset handler is not enough; both words have to hold."""
|
||||
path = image(self.tmp, "bad.bin", 0x12345678, 0x08002d49)
|
||||
with self.assertRaises(ui.ImageError):
|
||||
ui.detect(path)
|
||||
|
||||
def test_rejects_an_empty_file(self):
|
||||
path = write(self.tmp, "empty.bin", b"")
|
||||
with self.assertRaises(ui.ImageError):
|
||||
ui.detect(path)
|
||||
|
||||
def test_rejects_a_missing_file(self):
|
||||
with self.assertRaises(ui.ImageError):
|
||||
ui.detect(os.path.join(self.tmp, "nope.bin"))
|
||||
|
||||
def test_slot_can_be_swapped(self):
|
||||
slot = ui.ImageSlot()
|
||||
self.assertIsNone(slot.path)
|
||||
slot.set(image(self.tmp, "app.bin", 0x20004000, 0x08002d49))
|
||||
self.assertEqual(slot.app_offset, ui.APP_OFFSET)
|
||||
slot.set(image(self.tmp, "bl.bin", 0x200032c0, 0x08000901))
|
||||
self.assertEqual(slot.app_offset, 0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -24,7 +24,9 @@ class TestKeys(unittest.TestCase):
|
||||
"""
|
||||
src = os.path.join(os.path.dirname(os.path.abspath(__file__)),
|
||||
os.pardir, "qemu", "py32f071.c")
|
||||
text = open(src).read()
|
||||
# Explicit UTF-8: the default is the locale codec, and on Windows that is
|
||||
# GBK, which cannot decode this file once it contains any non-ASCII byte.
|
||||
text = open(src, encoding="utf-8").read()
|
||||
block = re.search(
|
||||
r"keypad_key_names\[[^\]]*\]\s*=\s*\{(.*?)\};", text, re.S)
|
||||
self.assertIsNotNone(block, "could not find keypad_key_names in the model")
|
||||
|
||||
+33
-2
@@ -6,6 +6,7 @@ import tempfile
|
||||
import unittest
|
||||
import zlib
|
||||
|
||||
import uvk5_lcd
|
||||
from uvk5_lcd import (FRAME_BYTES, LCD_HEIGHT, LCD_WIDTH, STATUS_BYTES,
|
||||
FrameGrabber, encode_png, unpack)
|
||||
|
||||
@@ -89,8 +90,10 @@ class StubClient:
|
||||
reported anywhere, which is exactly the bug this stub is here to catch.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
def __init__(self, invert=False, contrast=31, display_on=True):
|
||||
self.calls = []
|
||||
self.panel = {"invert": invert, "contrast": contrast,
|
||||
"display-on": display_on}
|
||||
|
||||
def command(self, name, **args):
|
||||
self.calls.append((name, args))
|
||||
@@ -98,6 +101,11 @@ class StubClient:
|
||||
raise AssertionError(
|
||||
"pmemsave reads physical addresses and silently returns zeros "
|
||||
"for gFrameBuffer; use memsave")
|
||||
if name == "qom-get":
|
||||
# The panel's own settings, which are not in the framebuffer at all.
|
||||
if args.get("path") != uvk5_lcd.PANEL_PATH:
|
||||
raise AssertionError(f"unexpected qom-get path {args.get('path')}")
|
||||
return self.panel[args["property"]]
|
||||
if name != "memsave":
|
||||
raise AssertionError(f"unexpected command {name}")
|
||||
with open(args["filename"], "wb") as fh:
|
||||
@@ -114,7 +122,30 @@ class TestFrameGrabber(unittest.TestCase):
|
||||
png = grabber.png(scale=2)
|
||||
|
||||
self.assertTrue(png.startswith(b"\x89PNG\r\n\x1a\n"))
|
||||
self.assertEqual([c[0] for c in client.calls], ["memsave", "memsave"])
|
||||
# Two reads, in frame-then-status order, followed by the panel's own
|
||||
# settings -- inversion and contrast live in the controller, not in RAM.
|
||||
self.assertEqual([c[0] for c in client.calls],
|
||||
["memsave", "memsave"] + ["qom-get"] * 3)
|
||||
|
||||
def test_panel_inversion_flips_the_picture(self):
|
||||
"""SetInv (0xA7) changes no byte of gFrameBuffer, so the render has to follow
|
||||
the controller or the menu entry looks like it did nothing."""
|
||||
tmp = tempfile.mkdtemp()
|
||||
normal = FrameGrabber(StubClient(invert=False), 0x1000, 0x2000, spool_dir=tmp)
|
||||
flipped = FrameGrabber(StubClient(invert=True), 0x1000, 0x2000,
|
||||
spool_dir=tempfile.mkdtemp())
|
||||
self.assertNotEqual(normal.png(scale=1), flipped.png(scale=1))
|
||||
|
||||
def test_display_off_is_reported_but_does_not_blank_the_frame(self):
|
||||
"""Whether a software reset (0xE2) clears the display-on latch is not certain,
|
||||
so the flag is reported rather than acted on: blanking the screen on a guess
|
||||
would be worse than leaving the image alone."""
|
||||
off = FrameGrabber(StubClient(display_on=False), 0x1000, 0x2000,
|
||||
spool_dir=tempfile.mkdtemp())
|
||||
on = FrameGrabber(StubClient(display_on=True), 0x1000, 0x2000,
|
||||
spool_dir=tempfile.mkdtemp())
|
||||
self.assertFalse(off.panel_state()[2])
|
||||
self.assertEqual(off.png(scale=1), on.png(scale=1))
|
||||
|
||||
def test_reads_the_right_addresses_and_sizes(self):
|
||||
client = StubClient()
|
||||
|
||||
@@ -72,6 +72,21 @@ class TestLogBuffer(unittest.TestCase):
|
||||
log.pump_stream(io.BytesIO(b"\xff\xfe bad\ngood\n"), default_source="qemu")
|
||||
self.assertEqual(len(log.entries()), 2)
|
||||
|
||||
def test_pump_stream_summarises_binary_serial(self):
|
||||
"""The CPS protocol shares the serial wire with the firmware's own output.
|
||||
|
||||
Decoded as text it filled the pane with control characters and buried the
|
||||
readable line; a mostly-binary line is reported as a size and a hex prefix,
|
||||
and it keeps its "serial" attribution.
|
||||
"""
|
||||
log = LogBuffer(capacity=20)
|
||||
log.pump_stream(io.BytesIO(b"SERIAL hello\nSERIAL \x02\x10\xff\xfe\x03\n"),
|
||||
default_source="qemu")
|
||||
got = [(e["source"], e["text"]) for e in log.entries()]
|
||||
self.assertEqual(got[0], ("serial", "hello"))
|
||||
self.assertEqual(got[1][0], "serial")
|
||||
self.assertTrue(got[1][1].startswith("<binary 5 bytes>"), got[1][1])
|
||||
|
||||
def test_add_is_thread_safe(self):
|
||||
log = LogBuffer(capacity=500)
|
||||
|
||||
|
||||
+13
-14
@@ -7,14 +7,18 @@ import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
|
||||
import uvk5_socket
|
||||
|
||||
from uvk5_qmp import QmpClient
|
||||
|
||||
|
||||
def fake_server(path, script):
|
||||
"""Minimal QMP server: greets, then replies to each command from `script`."""
|
||||
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
srv.bind(path)
|
||||
srv.listen(1)
|
||||
def fake_server(script):
|
||||
"""Minimal QMP server: greets, then replies to each command from script.
|
||||
|
||||
Returns (endpoint, server). The listener comes from uvk5_socket because a
|
||||
Windows QEMU cannot create a unix socket, and this file used to insist on one.
|
||||
"""
|
||||
srv, endpoint = uvk5_socket.listen("qmp")
|
||||
|
||||
def run():
|
||||
conn, _ = srv.accept()
|
||||
@@ -32,22 +36,20 @@ def fake_server(path, script):
|
||||
srv.close()
|
||||
|
||||
threading.Thread(target=run, daemon=True).start()
|
||||
return srv
|
||||
return endpoint, srv
|
||||
|
||||
|
||||
class TestQmpClient(unittest.TestCase):
|
||||
def test_negotiates_and_returns_command_result(self):
|
||||
path = os.path.join(tempfile.mkdtemp(), "qmp.sock")
|
||||
# reply 1 = qmp_capabilities, reply 2 = our command
|
||||
fake_server(path, [{"return": {}}, {"return": {"status": "running"}}])
|
||||
path, _srv = fake_server([{"return": {}}, {"return": {"status": "running"}}])
|
||||
|
||||
client = QmpClient(path)
|
||||
self.addCleanup(client.close)
|
||||
self.assertEqual(client.command("query-status"), {"status": "running"})
|
||||
|
||||
def test_raises_on_qmp_error(self):
|
||||
path = os.path.join(tempfile.mkdtemp(), "qmp.sock")
|
||||
fake_server(path, [{"return": {}},
|
||||
path, _srv = fake_server([{"return": {}},
|
||||
{"error": {"class": "GenericError", "desc": "nope"}}])
|
||||
client = QmpClient(path)
|
||||
self.addCleanup(client.close)
|
||||
@@ -62,10 +64,7 @@ class TestQmpClient(unittest.TestCase):
|
||||
return, so a client that stopped at the first message would hand back the
|
||||
event instead.
|
||||
"""
|
||||
path = os.path.join(tempfile.mkdtemp(), "qmp.sock")
|
||||
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
srv.bind(path)
|
||||
srv.listen(1)
|
||||
srv, path = uvk5_socket.listen("qmp") # (socket, endpoint)
|
||||
|
||||
def run():
|
||||
conn, _ = srv.accept()
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Unit tests for the endpoint helper. No emulator needed."""
|
||||
import socket
|
||||
import unittest
|
||||
|
||||
import uvk5_socket
|
||||
|
||||
|
||||
class TestEndpoints(unittest.TestCase):
|
||||
def test_the_endpoint_is_a_listening_tcp_form(self):
|
||||
"QEMU is the one listening, so the returned endpoint says so."
|
||||
srv, endpoint = uvk5_socket.listen("probe")
|
||||
self.addCleanup(srv.close)
|
||||
self.assertTrue(endpoint.startswith("unix:") or endpoint.startswith("tcp:"))
|
||||
|
||||
def test_every_accepted_endpoint_form_connects(self):
|
||||
"""The forms that reach connect() must all work.
|
||||
|
||||
A bare "host:port" did not once: the parser read it as a scheme, left the host
|
||||
empty, and hung until the deadline -- so the page could not power the emulator on
|
||||
while a QEMU started by hand answered instantly.
|
||||
|
||||
A fresh listener per form, because a backlog of one means the second client waits
|
||||
for an accept() nothing is calling -- which looks exactly like a broken parser.
|
||||
"""
|
||||
for make in (lambda port: "127.0.0.1:%d" % port,
|
||||
lambda port: "tcp:127.0.0.1:%d" % port,
|
||||
lambda port: "tcp:127.0.0.1:%d,server=on,wait=off" % port):
|
||||
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
srv.bind(("127.0.0.1", 0))
|
||||
srv.listen(4)
|
||||
port = srv.getsockname()[1]
|
||||
self.addCleanup(srv.close)
|
||||
endpoint = make(port)
|
||||
with self.subTest(endpoint=endpoint):
|
||||
client = uvk5_socket.connect(endpoint, timeout=5)
|
||||
client.close()
|
||||
|
||||
def test_the_two_directions_are_not_confused(self):
|
||||
srv_listen, endpoint = uvk5_socket.listen("x")
|
||||
self.addCleanup(srv_listen.close)
|
||||
self.assertNotIn("server=on", endpoint,
|
||||
"listen() means QEMU connects out; asking it to serve is a deadlock")
|
||||
served = uvk5_socket.server_endpoint("x")
|
||||
self.assertIn("server=on", served)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -3,6 +3,8 @@
|
||||
import os
|
||||
import socket
|
||||
import unittest
|
||||
|
||||
import uvk5_socket
|
||||
import unittest.mock
|
||||
|
||||
from uvk5_supervisor import Supervisor
|
||||
@@ -216,6 +218,8 @@ class TestWaitForSocket(unittest.TestCase):
|
||||
import shutil
|
||||
shutil.rmtree(self.dir, ignore_errors=True)
|
||||
|
||||
@unittest.skipUnless(uvk5_socket.can_use_unix(),
|
||||
"needs unix sockets, which a Windows QEMU cannot create")
|
||||
def test_returns_false_for_a_stale_socket_file(self):
|
||||
from uvk5_supervisor import wait_for_socket
|
||||
# A socket file with nothing listening: bind then close.
|
||||
@@ -225,6 +229,8 @@ class TestWaitForSocket(unittest.TestCase):
|
||||
self.assertTrue(os.path.exists(self.path), "need a leftover file")
|
||||
self.assertFalse(wait_for_socket(self.path, timeout=0.5))
|
||||
|
||||
@unittest.skipUnless(uvk5_socket.can_use_unix(),
|
||||
"needs unix sockets, which a Windows QEMU cannot create")
|
||||
def test_returns_true_when_something_is_listening(self):
|
||||
from uvk5_supervisor import wait_for_socket
|
||||
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
|
||||
@@ -1,9 +1,17 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Unit tests for the web UI. Stubs the QMP client, so no emulator needed."""
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
import unittest
|
||||
|
||||
import uvk5_image
|
||||
import webui
|
||||
from uvk5_supervisor import FlashSlot
|
||||
|
||||
|
||||
class StubClient:
|
||||
@@ -725,6 +733,13 @@ class TestClientIpInLogs(unittest.TestCase):
|
||||
def test_entries_without_a_request_have_no_ip(self):
|
||||
"""Firmware serial and qemu output come from no client at all."""
|
||||
from uvk5_logs import LogBuffer
|
||||
|
||||
try:
|
||||
from uvk5_supervisor import FlashSlot as flash_holder
|
||||
except Exception: # pragma: no cover - the supervisor is optional here
|
||||
class flash_holder:
|
||||
def __init__(self, path):
|
||||
self.path = path
|
||||
log = LogBuffer()
|
||||
log.add("serial", "boot banner")
|
||||
self.assertIsNone(log.entries()[-1]["ip"])
|
||||
@@ -740,6 +755,113 @@ class TestClientIpInLogs(unittest.TestCase):
|
||||
self.assertLess(tmpl.index("ip"), tmpl.index("e.source"))
|
||||
|
||||
|
||||
class TestFirmwareEndpoints(unittest.TestCase):
|
||||
"""Uploading firmware. The body is the image; its shape comes from the image."""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.mkdtemp()
|
||||
self._old_upload_dir = os.environ.get("UVK5_UPLOAD_DIR")
|
||||
os.environ["UVK5_UPLOAD_DIR"] = self.tmp
|
||||
self.slot = uvk5_image.ImageSlot()
|
||||
self.client = StubClient()
|
||||
self.sup = FakeSupervisor(self.client)
|
||||
app = webui.create_app(self.client, frame_addr=0x1000, status_addr=0x2000,
|
||||
supervisor=self.sup, image=self.slot)
|
||||
app.config.update(TESTING=True)
|
||||
self.http = app.test_client()
|
||||
|
||||
def tearDown(self):
|
||||
if self._old_upload_dir is None:
|
||||
os.environ.pop("UVK5_UPLOAD_DIR", None)
|
||||
else:
|
||||
os.environ["UVK5_UPLOAD_DIR"] = self._old_upload_dir
|
||||
|
||||
@staticmethod
|
||||
def app_image(size=0x4000):
|
||||
"""An application image: vector table first, entry past the app offset."""
|
||||
buf = bytearray(size)
|
||||
struct.pack_into("<II", buf, 0, 0x20004000, 0x08002d49)
|
||||
return bytes(buf)
|
||||
|
||||
@staticmethod
|
||||
def full_flash_image(size=0x4000):
|
||||
buf = bytearray(size)
|
||||
struct.pack_into("<II", buf, 0, 0x200032c0, 0x08000901) # entry in the BL region
|
||||
return bytes(buf)
|
||||
|
||||
def upload(self, data, name="fw.bin", http=None):
|
||||
return (http or self.http).post(
|
||||
"/api/firmware?name=" + name, data=data,
|
||||
content_type="application/octet-stream")
|
||||
|
||||
def test_reports_nothing_loaded_at_first(self):
|
||||
body = self.http.get("/api/firmware").get_json()
|
||||
self.assertFalse(body["loaded"])
|
||||
self.assertIsNone(body["firmware"])
|
||||
|
||||
def test_upload_adopts_the_image_and_restarts(self):
|
||||
res = self.upload(self.app_image())
|
||||
self.assertEqual(res.status_code, 200)
|
||||
info = res.get_json()
|
||||
self.assertEqual(info["firmware"]["kind"], "application")
|
||||
self.assertEqual(info["firmware"]["app_offset"], uvk5_image.APP_OFFSET)
|
||||
self.assertTrue(info["restarted"])
|
||||
self.assertIn("power_off", self.sup.calls)
|
||||
self.assertIn("power_on", self.sup.calls)
|
||||
self.assertEqual(self.slot.current.kind, "application")
|
||||
self.assertTrue(os.path.isfile(os.path.join(self.tmp, "fw.bin")))
|
||||
|
||||
def test_upload_recognises_a_full_flash_image(self):
|
||||
res = self.upload(self.full_flash_image(), name="bl.bin")
|
||||
self.assertEqual(res.status_code, 200)
|
||||
info = res.get_json()["firmware"]
|
||||
self.assertEqual(info["kind"], "full-flash")
|
||||
self.assertEqual(info["app_offset"], 0)
|
||||
|
||||
def test_upload_while_off_does_not_try_to_restart(self):
|
||||
sup = FakeSupervisor(None)
|
||||
app = webui.create_app(None, 0x1000, 0x2000, supervisor=sup, image=self.slot)
|
||||
app.config.update(TESTING=True)
|
||||
res = self.upload(self.app_image(), http=app.test_client())
|
||||
self.assertFalse(res.get_json()["restarted"])
|
||||
self.assertNotIn("power_on", sup.calls)
|
||||
self.assertNotIn("power_off", sup.calls)
|
||||
|
||||
def test_a_file_that_is_not_an_image_changes_nothing(self):
|
||||
self.slot.set(os.path.join(self.tmp, "good.bin")) if False else None
|
||||
first = self.upload(self.app_image(), name="good.bin")
|
||||
self.assertEqual(first.status_code, 200)
|
||||
kept = self.slot.current.path
|
||||
self.sup.calls.clear()
|
||||
res = self.upload(b"this is not a firmware image" * 40, name="junk.bin")
|
||||
self.assertEqual(res.status_code, 400)
|
||||
self.assertIn("vector table", res.get_json()["error"])
|
||||
# The radio is left exactly as it was, still running, still the old image.
|
||||
self.assertEqual(self.slot.current.path, kept)
|
||||
self.assertNotIn("power_off", self.sup.calls)
|
||||
self.assertNotIn("power_on", self.sup.calls)
|
||||
|
||||
def test_empty_body_is_rejected(self):
|
||||
res = self.upload(b"", name="empty.bin")
|
||||
self.assertEqual(res.status_code, 400)
|
||||
self.assertIn("no image", res.get_json()["error"])
|
||||
|
||||
def test_oversized_body_is_rejected(self):
|
||||
res = self.upload(b"\x00" * (webui.MAX_UPLOAD_BYTES + 1), name="big.bin")
|
||||
self.assertEqual(res.status_code, 413)
|
||||
|
||||
def test_upload_without_an_image_slot_is_refused(self):
|
||||
app = webui.create_app(self.client, 0x1000, 0x2000, supervisor=self.sup, image=None)
|
||||
app.config.update(TESTING=True)
|
||||
res = self.upload(self.app_image(), http=app.test_client())
|
||||
self.assertEqual(res.status_code, 409)
|
||||
|
||||
def test_the_name_cannot_escape_the_upload_directory(self):
|
||||
res = self.upload(self.app_image(), name="..%2F..%2Fevil.bin")
|
||||
self.assertEqual(res.status_code, 200)
|
||||
self.assertEqual(os.path.dirname(self.slot.current.path), self.tmp)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -791,3 +913,127 @@ class TestSpeakerIndicator(unittest.TestCase):
|
||||
for forbidden in ("getUserMedia", "AudioContext", "navigator.mediaDevices",
|
||||
"new Audio", "<audio"):
|
||||
self.assertNotIn(forbidden, body)
|
||||
|
||||
|
||||
class TestFirmwareSlots(unittest.TestCase):
|
||||
"""The firmware slot endpoints behind the page's slot table.
|
||||
|
||||
These edit the external-flash image the emulator boots from, so most of what is
|
||||
worth asserting is not the bytes but the ordering: power off first (the emulator
|
||||
writes its own copy back while it runs), edit a working copy, power on again.
|
||||
"""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.mkdtemp()
|
||||
self._old = os.environ.get("UVK5_UPLOAD_DIR")
|
||||
os.environ["UVK5_UPLOAD_DIR"] = self.tmp
|
||||
self.flash = os.path.join(self.tmp, "base.img")
|
||||
with open(self.flash, "wb") as fh:
|
||||
fh.write(b"\xff" * (2 * 1024 * 1024))
|
||||
self.holder = FlashSlot(self.flash)
|
||||
self.client = StubClient()
|
||||
self.sup = FakeSupervisor(self.client)
|
||||
self.app = webui.create_app(self.client, frame_addr=0x1000, status_addr=0x2000,
|
||||
supervisor=self.sup, flash=self.holder)
|
||||
self.app.config.update(TESTING=True)
|
||||
self.web = self.app.test_client()
|
||||
|
||||
def tearDown(self):
|
||||
if self._old is None:
|
||||
os.environ.pop("UVK5_UPLOAD_DIR", None)
|
||||
else:
|
||||
os.environ["UVK5_UPLOAD_DIR"] = self._old
|
||||
shutil.rmtree(self.tmp, ignore_errors=True)
|
||||
|
||||
def test_lists_five_empty_slots(self):
|
||||
body = self.web.get("/api/slots").get_json()
|
||||
self.assertEqual(len(body["slots"]), 5)
|
||||
self.assertTrue(all(s["empty"] for s in body["slots"]))
|
||||
self.assertEqual(body["slots"][0]["base"], 0x020000)
|
||||
self.assertEqual(body["slots"][4]["base"], 0x0A0000)
|
||||
|
||||
def test_writing_a_slot_programs_a_valid_header(self):
|
||||
image = bytes(range(256)) * 4
|
||||
r = self.web.post("/api/slots/1?name=testfw&version=9.9",
|
||||
data=image, content_type="application/octet-stream")
|
||||
self.assertEqual(r.status_code, 200)
|
||||
slot = r.get_json()["slot"]
|
||||
self.assertEqual(slot["image_size"], len(image))
|
||||
self.assertTrue(slot["crc_ok"], "the header CRC must describe the image written")
|
||||
self.assertNotEqual(os.path.abspath(self.holder.path),
|
||||
os.path.abspath(self.flash))
|
||||
with open(self.holder.path, "rb") as fh:
|
||||
self.assertEqual(fh.read()[0x040000:0x040004], b"FMB1")
|
||||
|
||||
def test_writing_powers_the_emulator_off_before_editing_and_back_on(self):
|
||||
self.web.post("/api/slots/2", data=b"x" * 512,
|
||||
content_type="application/octet-stream")
|
||||
self.assertEqual(self.sup.calls[:2], ["power_off", "power_on"])
|
||||
|
||||
def test_erasing_clears_the_slot(self):
|
||||
self.web.post("/api/slots/3", data=b"y" * 512,
|
||||
content_type="application/octet-stream")
|
||||
self.assertFalse(self.web.get("/api/slots").get_json()["slots"][3]["empty"])
|
||||
self.web.post("/api/slots/3/erase")
|
||||
self.assertTrue(self.web.get("/api/slots").get_json()["slots"][3]["empty"])
|
||||
|
||||
def test_rejects_an_out_of_range_slot(self):
|
||||
r = self.web.post("/api/slots/9", data=b"z", content_type="application/octet-stream")
|
||||
self.assertEqual(r.status_code, 400)
|
||||
|
||||
def test_rejects_an_empty_body(self):
|
||||
r = self.web.post("/api/slots/1", data=b"", content_type="application/octet-stream")
|
||||
self.assertEqual(r.status_code, 400)
|
||||
|
||||
def test_rejects_an_image_larger_than_a_slot(self):
|
||||
r = self.web.post("/api/slots/1", data=b"a" * (128 * 1024),
|
||||
content_type="application/octet-stream")
|
||||
self.assertEqual(r.status_code, 400)
|
||||
|
||||
def test_uploading_a_flash_image_switches_the_emulator_to_it(self):
|
||||
r = self.web.post("/api/flash?name=mine.img", data=b"\xff" * (2 * 1024 * 1024),
|
||||
content_type="application/octet-stream")
|
||||
self.assertEqual(r.status_code, 200)
|
||||
self.assertEqual(os.path.basename(self.holder.path), "mine.img")
|
||||
|
||||
def test_page_carries_the_slot_table(self):
|
||||
html = self.web.get("/").get_data(as_text=True)
|
||||
self.assertIn("slottable", html)
|
||||
self.assertIn("/api/slots", html)
|
||||
|
||||
class TestPageScriptParses(unittest.TestCase):
|
||||
"""The page's JavaScript has to be syntactically valid.
|
||||
|
||||
It is assembled by an f-string, so escaping mistakes are easy and invisible: one
|
||||
bad backslash in a string literal makes the whole script fail to parse, and the
|
||||
page then sits on "connecting..." forever while every endpoint still answers. That
|
||||
is exactly what happened, and only a browser would have shown it -- so this checks
|
||||
it here.
|
||||
"""
|
||||
|
||||
def setUp(self):
|
||||
app = webui.create_app(None, frame_addr=0x1000, status_addr=0x2000)
|
||||
app.config.update(TESTING=True)
|
||||
self.html = app.test_client().get("/").get_data(as_text=True)
|
||||
|
||||
def test_the_script_is_one_block(self):
|
||||
self.assertIn("<script>", self.html)
|
||||
self.assertIn("</script>", self.html)
|
||||
|
||||
def test_the_script_parses_with_node_when_it_is_available(self):
|
||||
node = shutil.which("node")
|
||||
if node is None:
|
||||
self.skipTest("node is not installed")
|
||||
bodies = re.findall(r"<script>(.*?)</script>", self.html, re.S)
|
||||
self.assertTrue(bodies, "the page has no script block")
|
||||
with tempfile.NamedTemporaryFile("w", suffix=".js", delete=False,
|
||||
encoding="utf-8") as fh:
|
||||
fh.write("\n;\n".join(bodies))
|
||||
path = fh.name
|
||||
try:
|
||||
done = subprocess.run([node, "--check", path], capture_output=True,
|
||||
text=True, timeout=60)
|
||||
self.assertEqual(done.returncode, 0,
|
||||
"the page's script does not parse:\n" + done.stderr)
|
||||
finally:
|
||||
os.unlink(path)
|
||||
@@ -0,0 +1,139 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Which image to boot, and where the loader has to put it.
|
||||
|
||||
Firmware arrives in two shapes and the difference is not cosmetic:
|
||||
|
||||
* an **application** image is linked for 0x08002800. Its first bytes *are* its vector
|
||||
table, and address 0 aliases the application region so the reset fetch finds it.
|
||||
* a **full-flash** image starts at 0x08000000 -- a bootloader, whose entry point
|
||||
lives in the bootloader region (0x08000000..0x080027FF), with the application
|
||||
embedded behind it.
|
||||
|
||||
Get that wrong and nothing complains: the image lands 0x2800 bytes off and the first
|
||||
fetch reads whatever data is there (0xFF, usually, so it faults). The shape is
|
||||
readable from the image itself, so this reads the vector table rather than asking the
|
||||
user or guessing from a file name.
|
||||
"""
|
||||
import os
|
||||
import struct
|
||||
|
||||
FLASH_BASE = 0x08000000
|
||||
APP_OFFSET = 0x2800
|
||||
SRAM_BASE = 0x20000000
|
||||
SRAM_TOP = 0x20004000 # 16 KB of SRAM, so SP starts at the top
|
||||
ELF_MAGIC = b"\x7fELF"
|
||||
MAX_IMAGE = 4 * 1024 * 1024 # a full 2 MB flash dump plus slack
|
||||
|
||||
|
||||
class ImageError(ValueError):
|
||||
"""The file is not an image this machine can boot."""
|
||||
|
||||
|
||||
class ImageInfo:
|
||||
def __init__(self, path, kind, app_offset, size, sp=None, reset=None):
|
||||
self.path = path
|
||||
self.kind = kind # "application" | "full-flash" | "elf"
|
||||
self.app_offset = app_offset # flash offset address 0 must alias
|
||||
self.size = size
|
||||
self.sp = sp
|
||||
self.reset = reset
|
||||
|
||||
def as_dict(self):
|
||||
return {
|
||||
"name": os.path.basename(self.path),
|
||||
"path": self.path,
|
||||
"kind": self.kind,
|
||||
"app_offset": self.app_offset,
|
||||
"size": self.size,
|
||||
"sp": None if self.sp is None else "0x%08x" % self.sp,
|
||||
"reset": None if self.reset is None else "0x%08x" % self.reset,
|
||||
}
|
||||
|
||||
def __repr__(self):
|
||||
return "<ImageInfo %s %s %dB app_offset=0x%x>" % (
|
||||
self.kind, os.path.basename(self.path), self.size, self.app_offset)
|
||||
|
||||
|
||||
def _vector_at(data, offset):
|
||||
"""(SP, reset) if a plausible vector table starts at @offset, else None.
|
||||
|
||||
Both words are checked against what the hardware can accept: SP inside SRAM, and
|
||||
a reset handler inside the image itself. A single plausible word is a
|
||||
coincidence in code, so a half-match is treated as no match.
|
||||
"""
|
||||
if offset + 8 > len(data):
|
||||
return None
|
||||
sp, reset = struct.unpack_from("<II", data, offset)
|
||||
if not (SRAM_BASE < sp <= SRAM_TOP):
|
||||
return None
|
||||
if not (FLASH_BASE <= reset < FLASH_BASE + len(data)):
|
||||
return None
|
||||
return sp, reset
|
||||
|
||||
|
||||
def detect(path):
|
||||
"""Work out how to load @path. Raises ImageError if it cannot be booted."""
|
||||
if not os.path.isfile(path):
|
||||
raise ImageError("no such file: %s" % path)
|
||||
size = os.path.getsize(path)
|
||||
if size == 0:
|
||||
raise ImageError("empty file")
|
||||
if size > MAX_IMAGE:
|
||||
raise ImageError("%d bytes is larger than any UV-K5 image" % size)
|
||||
with open(path, "rb") as fh:
|
||||
data = fh.read()
|
||||
|
||||
if data[:4] == ELF_MAGIC:
|
||||
# An ELF carries its own program headers, so the loader base is irrelevant;
|
||||
# the app offset only decides what address 0 aliases, which for an
|
||||
# application ELF is the application region.
|
||||
return ImageInfo(path, "elf", APP_OFFSET, size)
|
||||
|
||||
vector = _vector_at(data, 0)
|
||||
if vector is None:
|
||||
raise ImageError(
|
||||
"not a bootable image: no vector table at offset 0 (the first word must "
|
||||
"be an SRAM address and the second a handler inside the file)")
|
||||
sp, reset = vector
|
||||
|
||||
# A full-flash image's entry point lives in the bootloader region, ahead of where
|
||||
# the application starts. That is the whole distinction -- not the size, and not
|
||||
# a second vector table, because an application's code can contain anything at
|
||||
# offset 0x2800.
|
||||
if reset < FLASH_BASE + APP_OFFSET:
|
||||
return ImageInfo(path, "full-flash", 0, size, sp, reset)
|
||||
return ImageInfo(path, "application", APP_OFFSET, size, sp, reset)
|
||||
|
||||
|
||||
class ImageSlot:
|
||||
"""The image the next launch boots.
|
||||
|
||||
Mutable on purpose: an upload has to take effect at the next power-on without
|
||||
restarting the server, and the launcher reads this at spawn time rather than
|
||||
closing over a path chosen when the server started.
|
||||
"""
|
||||
|
||||
def __init__(self, path=None):
|
||||
self.current = detect(path) if path else None
|
||||
|
||||
def set(self, path_or_info):
|
||||
"""Load an image, or adopt one that has already been detected.
|
||||
|
||||
Accepting an ImageInfo lets a caller validate a file *before* disturbing
|
||||
anything: an upload that turns out not to be an image must not leave the
|
||||
radio powered off.
|
||||
"""
|
||||
self.current = (path_or_info if isinstance(path_or_info, ImageInfo)
|
||||
else detect(path_or_info))
|
||||
return self.current
|
||||
|
||||
def clear(self):
|
||||
self.current = None
|
||||
|
||||
@property
|
||||
def path(self):
|
||||
return self.current.path if self.current else None
|
||||
|
||||
@property
|
||||
def app_offset(self):
|
||||
return self.current.app_offset if self.current else None
|
||||
+96
-5
@@ -8,6 +8,8 @@ and the CLI screenshotter cannot drift apart.
|
||||
"""
|
||||
import os
|
||||
import struct
|
||||
import sys
|
||||
import tempfile
|
||||
import zlib
|
||||
|
||||
LCD_WIDTH = 128
|
||||
@@ -59,6 +61,19 @@ def encode_png(pixels, scale: int = 4) -> bytes:
|
||||
+ chunk(b"IEND", b""))
|
||||
|
||||
|
||||
# The display controller's own settings. Inversion and display-on are panel state,
|
||||
# not framebuffer content, so nothing in guest RAM reflects them -- which is exactly
|
||||
# why a menu entry that changes them looks like it did nothing.
|
||||
PANEL_PATH = "/machine/panel"
|
||||
|
||||
|
||||
def default_spool_dir() -> str:
|
||||
"""A tmpfs when the host has one, the system temp directory otherwise."""
|
||||
if os.path.isdir("/dev/shm"):
|
||||
return "/dev/shm"
|
||||
return tempfile.gettempdir()
|
||||
|
||||
|
||||
class FrameGrabber:
|
||||
"""Reads the LCD out of guest memory over QMP.
|
||||
|
||||
@@ -75,13 +90,18 @@ class FrameGrabber:
|
||||
"""
|
||||
|
||||
def __init__(self, client, frame_addr: int, status_addr: int,
|
||||
spool_dir: str = "/dev/shm"):
|
||||
spool_dir: str = None):
|
||||
self._client = client
|
||||
self._frame_addr = frame_addr
|
||||
self._status_addr = status_addr
|
||||
# pmemsave writes to a path, so a tmpfs avoids disk I/O every frame.
|
||||
self._frame_path = os.path.join(spool_dir, "uvk5-frame.bin")
|
||||
self._status_path = os.path.join(spool_dir, "uvk5-status.bin")
|
||||
# memsave writes to a path, so a tmpfs avoids disk I/O every frame --
|
||||
# where there is one. /dev/shm does not exist on Windows, and naming it
|
||||
# there makes every frame grab fail, which shows up only as a blank
|
||||
# screen with "no frame available" from the web UI.
|
||||
self._panel_warned = False
|
||||
self._spool_dir = spool_dir or default_spool_dir()
|
||||
self._frame_path = os.path.join(self._spool_dir, "uvk5-frame.bin")
|
||||
self._status_path = os.path.join(self._spool_dir, "uvk5-status.bin")
|
||||
|
||||
def raw(self) -> tuple[bytes, bytes]:
|
||||
"""Return (status, frame) exactly as the firmware holds them."""
|
||||
@@ -95,6 +115,77 @@ class FrameGrabber:
|
||||
status = fh.read(STATUS_BYTES)
|
||||
return status, frame
|
||||
|
||||
def panel_state(self):
|
||||
"""(invert, contrast, display_on) as the display controller holds them.
|
||||
|
||||
Read from the panel model rather than the framebuffer: 0xA6/0xA7, 0x81 and
|
||||
0xAE/0xAF live in the controller. If the model is not there (an older
|
||||
emulator build) the defaults describe an ordinary, unobstructed panel.
|
||||
"""
|
||||
try:
|
||||
invert = bool(self._client.command("qom-get", path=PANEL_PATH,
|
||||
property="invert"))
|
||||
contrast = int(self._client.command("qom-get", path=PANEL_PATH,
|
||||
property="contrast"))
|
||||
display_on = bool(self._client.command("qom-get", path=PANEL_PATH,
|
||||
property="display-on"))
|
||||
except Exception as exc:
|
||||
# Not fatal -- an emulator built without the panel model, or one that
|
||||
# just went away, still has a framebuffer worth showing. But say so
|
||||
# once: swallowing this silently is how a wrong render looks like a
|
||||
# firmware that ignores the setting. (It hid a stub bug in the test
|
||||
# for this very method.)
|
||||
if not self._panel_warned:
|
||||
self._panel_warned = True
|
||||
print(f"panel state unavailable, rendering as-is: {exc}",
|
||||
file=sys.stderr)
|
||||
return (False, 0, True)
|
||||
return (invert, contrast, display_on)
|
||||
|
||||
def panel_gram(self) -> bytes:
|
||||
"""The display controller's own display RAM: the screen as it is shown.
|
||||
|
||||
Preferred over raw() when the caller does not know where *this* firmware
|
||||
keeps its buffers. Builds that share an ancestor still differ in their
|
||||
display logic, and the multi-system release keeps its image somewhere else
|
||||
entirely -- but every one of them pushes pixels through the same controller.
|
||||
"""
|
||||
hexed = self._client.command("qom-get", path=PANEL_PATH, property="gram")
|
||||
return bytes.fromhex(hexed)
|
||||
|
||||
def panel_pixels(self):
|
||||
"""The screen as pixels, from the controller's own memory.
|
||||
|
||||
No hardware mirroring is applied. The driver programs 0xA1 (segment reverse)
|
||||
and 0xC0, but the columns arrive in the order the glass needs, so mirroring
|
||||
on top of the data flips the picture: against the guest's own framebuffer at
|
||||
the same instant, 8153 of 8192 pixels agree with no mirror and 6557 with
|
||||
one. The flags stay reported, not acted on.
|
||||
|
||||
Raises if the panel model is absent, which is how the caller knows to fall
|
||||
back to guest RAM -- see uvk5_stream.FramePump.
|
||||
"""
|
||||
gram = self.panel_gram()
|
||||
if len(gram) != TOTAL_ROWS * LCD_WIDTH:
|
||||
raise ValueError("panel GRAM is %d bytes, expected %d"
|
||||
% (len(gram), TOTAL_ROWS * LCD_WIDTH))
|
||||
return self._apply_panel(unpack(gram[:STATUS_BYTES], gram[STATUS_BYTES:]))
|
||||
|
||||
def panel_png(self, scale: int = 4) -> bytes:
|
||||
"""The panel's own memory, encoded as a PNG."""
|
||||
return encode_png(self.panel_pixels(), scale)
|
||||
def _apply_panel(self, pixels):
|
||||
"""Apply the panel settings that change the picture (inversion only)."""
|
||||
invert, _contrast, display_on = self.panel_state()
|
||||
if invert:
|
||||
# 0xA7: the panel inverts the whole image, which is a visible, fully
|
||||
# determined effect -- so the render follows it. Contrast is analogue
|
||||
# and cannot be rendered; display-on is deliberately *not* acted on
|
||||
# here: whether a software reset (0xE2) clears the display-on latch is
|
||||
# not certain, and blanking the screen on a guess would be worse than
|
||||
# reporting the flag and leaving the image alone.
|
||||
pixels = [[1 - value for value in row] for row in pixels]
|
||||
return pixels
|
||||
def png(self, scale: int = 4) -> bytes:
|
||||
status, frame = self.raw()
|
||||
return encode_png(unpack(status, frame), scale)
|
||||
return encode_png(self._apply_panel(unpack(status, frame)), scale)
|
||||
+35
-8
@@ -15,6 +15,29 @@ import collections
|
||||
import threading
|
||||
import time
|
||||
|
||||
# Bytes that are safe to show as text. Everything else in a serial line means the
|
||||
# wire is carrying binary, not output meant to be read.
|
||||
_TEXT_BYTES = frozenset(range(0x20, 0x7f)) | {0x09}
|
||||
|
||||
|
||||
def describe_line(raw: bytes) -> str:
|
||||
"""A line of serial, or a summary when the bytes are not text.
|
||||
|
||||
Serial carries two very different things over one wire: the firmware's readable
|
||||
output, and the CPS programming protocol, which is binary. Decoding the second
|
||||
as text filled the pane with control characters and buried the first, so a line
|
||||
that is mostly non-printable becomes its size plus a hex prefix instead.
|
||||
"""
|
||||
body = raw.rstrip(b"\r\n")
|
||||
if not body:
|
||||
return ""
|
||||
unprintable = sum(1 for b in body if b not in _TEXT_BYTES)
|
||||
if unprintable * 4 <= len(body):
|
||||
return body.decode("utf-8", "replace")
|
||||
head = body[:24].hex(" ")
|
||||
tail = "" if len(body) <= 24 else f" … +{len(body) - 24} bytes"
|
||||
return f"<binary {len(body)} bytes> {head}{tail}"
|
||||
|
||||
|
||||
class LogBuffer:
|
||||
def __init__(self, capacity: int = 500):
|
||||
@@ -56,13 +79,17 @@ class LogBuffer:
|
||||
|
||||
Decoding is lenient: serial bytes can be garbage before the firmware has
|
||||
configured the port, and losing the whole stream to one bad byte would be
|
||||
worse than showing a replacement character.
|
||||
worse than showing it. A line that is mostly binary is summarised rather
|
||||
than decoded -- see describe_line().
|
||||
"""
|
||||
for raw in iter(stream.readline, b""):
|
||||
line = raw.decode("utf-8", "replace").rstrip("\r\n")
|
||||
if not line:
|
||||
continue
|
||||
if line.startswith("SERIAL "):
|
||||
self.add("serial", line[len("SERIAL "):])
|
||||
else:
|
||||
self.add(default_source, line)
|
||||
# Strip the model's tag before looking at the bytes: on a binary line the
|
||||
# hex summary would otherwise hide the prefix and the line would lose its
|
||||
# "serial" attribution.
|
||||
source = default_source
|
||||
if raw.startswith(b"SERIAL "):
|
||||
source = "serial"
|
||||
raw = raw[len(b"SERIAL "):]
|
||||
line = describe_line(raw)
|
||||
if line:
|
||||
self.add(source, line)
|
||||
+20
-6
@@ -13,19 +13,33 @@ import socket
|
||||
import threading
|
||||
|
||||
|
||||
def connect(endpoint: str, timeout: float):
|
||||
"""Open the QMP connection.
|
||||
|
||||
Accepts everything QEMU accepts: a bare unix path, host:port, or the full forms
|
||||
tcp:host:port[,options] and unix:path -- which is what uvk5_socket.listen() hands
|
||||
back, so the listening and connecting sides cannot drift apart. One parser, in
|
||||
uvk5_socket, because two of them is how the Windows path broke: this one only
|
||||
understood host:port, so a tcp:... endpoint fell through to the unix branch and
|
||||
failed with a missing AF_UNIX.
|
||||
"""
|
||||
import uvk5_socket
|
||||
|
||||
return uvk5_socket.connect(endpoint, timeout=timeout)
|
||||
|
||||
|
||||
class QmpClient:
|
||||
def __init__(self, path: str, timeout: float = 5.0):
|
||||
self._lock = threading.Lock()
|
||||
self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
self._sock.settimeout(timeout)
|
||||
try:
|
||||
self._sock.connect(path)
|
||||
self._sock = connect(path, timeout)
|
||||
except OSError as exc:
|
||||
raise RuntimeError(
|
||||
f"cannot reach the emulator at {path}: {exc}\n"
|
||||
"Start it with tools/run.sh first. Note the QMP socket takes a "
|
||||
"single client, so tools/key.py cannot be connected at the same "
|
||||
"time."
|
||||
"Start it with tools/run.sh first (on Windows pass --qmp "
|
||||
"127.0.0.1:4444 and start QEMU with -qmp tcp:...). Note the QMP "
|
||||
"socket takes a single client, so tools/key.py cannot be "
|
||||
"connected at the same time."
|
||||
) from exc
|
||||
self._buf = b""
|
||||
self._read_json() # greeting
|
||||
|
||||
@@ -0,0 +1,244 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Program a UV-K5 over its serial bootloader protocol.
|
||||
|
||||
Not the same protocol as the EEPROM read/write commands (0x0514/0x051B). This one is
|
||||
the firmware-update flow, and its message set and framing come from the firmware's own
|
||||
host tool (`tools/serialtool` in armel/uv-k1-k5v3-firmware-custom, MIT licensed):
|
||||
|
||||
0x0518 device -> host UID and bootloader version, announced repeatedly
|
||||
0x0530 host -> device the bootloader version we expect (handshake)
|
||||
0x0519 host -> device timestamp, page index, page count, then up to 256 bytes
|
||||
0x051A device -> host timestamp, page index, error code
|
||||
|
||||
Framing is `AB CD | len | payload | crc | DC BA` with the payload XOR-ed by a fixed
|
||||
16-byte table. The CRC is only checked on the host side: the device does not send a
|
||||
useful one, which is why the reference client ignores it.
|
||||
|
||||
The transport here is a socket, because that is what the emulator offers through
|
||||
`-serial tcp:host:port`. A real radio needs a serial port; pass one as *endpoint* and
|
||||
pyserial is used instead (`pip install pyserial`).
|
||||
"""
|
||||
import argparse
|
||||
import socket
|
||||
import struct
|
||||
import sys
|
||||
import time
|
||||
|
||||
MSG_NOTIFY_DEV_INFO = 0x0518
|
||||
MSG_NOTIFY_BL_VER = 0x0530
|
||||
MSG_PROG_FW = 0x0519
|
||||
MSG_PROG_FW_RESP = 0x051A
|
||||
|
||||
PAGE_SIZE = 256
|
||||
MAGIC = b"\xab\xcd"
|
||||
END = b"\xdc\xba"
|
||||
# The obfuscation table, copied from tools/serialtool/msg.py.
|
||||
OBFUS = bytes([0x16, 0x6C, 0x14, 0xE6, 0x2E, 0x91, 0x0D, 0x40,
|
||||
0x21, 0x35, 0xD5, 0x40, 0x13, 0x03, 0xE9, 0x80])
|
||||
|
||||
|
||||
def obfuscate(data: bytes) -> bytes:
|
||||
return bytes(b ^ OBFUS[i % len(OBFUS)] for i, b in enumerate(data))
|
||||
|
||||
|
||||
def crc16_xmodem(data: bytes) -> int:
|
||||
crc = 0
|
||||
for byte in data:
|
||||
crc ^= byte << 8
|
||||
for _ in range(8):
|
||||
crc = ((crc << 1) ^ 0x1021) & 0xFFFF if crc & 0x8000 else (crc << 1) & 0xFFFF
|
||||
return crc
|
||||
|
||||
|
||||
def build(msg_type: int, data: bytes = b"") -> bytes:
|
||||
"""One frame: header, payload, CRC, footer, obfuscated in one pass.
|
||||
|
||||
The length field counts the *message* -- type, length and data -- and excludes the
|
||||
CRC. A device announcement reads `ab cd 24 00` for a 36-byte message (a 32-byte
|
||||
UID+version body), which is what pins this down. Counting the CRC as well makes
|
||||
every frame two bytes too long, and the device then drops all of them without a
|
||||
word: the handshake is ignored and no page is ever acknowledged.
|
||||
"""
|
||||
payload = struct.pack("<HH", msg_type, len(data)) + data
|
||||
body = payload + struct.pack("<H", crc16_xmodem(payload))
|
||||
return MAGIC + struct.pack("<H", len(payload)) + obfuscate(body) + END
|
||||
|
||||
|
||||
class Frames:
|
||||
"""Reassembles frames from a byte stream and de-obfuscates them."""
|
||||
|
||||
def __init__(self):
|
||||
self._buf = bytearray()
|
||||
|
||||
def feed(self, chunk: bytes):
|
||||
self._buf.extend(chunk)
|
||||
|
||||
def take(self):
|
||||
"""The next (type, data) pair, or None if one is not complete yet."""
|
||||
while True:
|
||||
start = self._buf.find(MAGIC)
|
||||
if start < 0:
|
||||
del self._buf[:] # nothing usable
|
||||
return None
|
||||
if len(self._buf) < start + 8:
|
||||
del self._buf[:start]
|
||||
return None
|
||||
length = struct.unpack_from("<H", self._buf, start + 2)[0]
|
||||
end = start + 6 + length
|
||||
if len(self._buf) < end + 2:
|
||||
del self._buf[:start]
|
||||
return None
|
||||
if bytes(self._buf[end:end + 2]) != END:
|
||||
del self._buf[:start + 2]
|
||||
continue
|
||||
body = obfuscate(bytes(self._buf[start + 4:end]))
|
||||
del self._buf[:end + 2]
|
||||
if len(body) < 4:
|
||||
continue
|
||||
msg_type, data_len = struct.unpack_from("<HH", body, 0)
|
||||
return msg_type, body[4:4 + data_len]
|
||||
|
||||
|
||||
def open_transport(endpoint: str, timeout: float = 10.0):
|
||||
"""A socket to host:port, or a serial port for anything else."""
|
||||
host, _, port = endpoint.rpartition(":")
|
||||
if host and port.isdigit():
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
sock = socket.create_connection((host, int(port)), timeout=2.0)
|
||||
sock.settimeout(0.5)
|
||||
return sock
|
||||
except OSError:
|
||||
time.sleep(0.2)
|
||||
raise SystemExit("no connection to %s" % endpoint)
|
||||
import serial # pyserial, only for a real radio
|
||||
port_obj = serial.Serial(endpoint, 38400, timeout=0.5)
|
||||
return port_obj
|
||||
|
||||
|
||||
class Flasher:
|
||||
def __init__(self, transport, log=print):
|
||||
self._t = transport
|
||||
self._frames = Frames()
|
||||
self._log = log
|
||||
|
||||
def _pump(self, seconds=1.0):
|
||||
end = time.monotonic() + seconds
|
||||
while time.monotonic() < end:
|
||||
try:
|
||||
chunk = self._t.recv(4096) if isinstance(self._t, socket.socket) \
|
||||
else self._t.read(4096)
|
||||
except (socket.timeout, OSError):
|
||||
chunk = b""
|
||||
if chunk:
|
||||
self._frames.feed(chunk)
|
||||
|
||||
def _send(self, msg_type: int, data: bytes = b""):
|
||||
self._t.sendall(build(msg_type, data)) if isinstance(self._t, socket.socket) \
|
||||
else self._t.write(build(msg_type, data))
|
||||
|
||||
def wait_for_device(self, timeout=20.0):
|
||||
"""Wait for 0x0518, which the bootloader sends about every 200 ms."""
|
||||
self._log("waiting for the device announcement (0x0518) ...")
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
self._pump(0.5)
|
||||
while True:
|
||||
msg = self._frames.take()
|
||||
if msg is None:
|
||||
break
|
||||
msg_type, data = msg
|
||||
if msg_type == MSG_NOTIFY_DEV_INFO:
|
||||
uid = data[:16].hex()
|
||||
bl = data[16:32].split(b"\x00")[0].decode("ascii", "replace")
|
||||
self._log("device: uid %s, bootloader %r" % (uid, bl))
|
||||
return bl
|
||||
raise SystemExit("no 0x0518 announcement; is the radio in flashing mode?")
|
||||
|
||||
def handshake(self, bl_ver: str, times: int = 3):
|
||||
"""0x0530 in reply to an announcement, three times, like the reference client.
|
||||
|
||||
The reply has to go out immediately: the reference client sits in a blocking
|
||||
read and answers each announcement the moment it lands. Batching (read for
|
||||
400 ms, then send) leaves the reply ~hundreds of announcements late, and the
|
||||
bootloader just keeps announcing -- which looks exactly like a device that
|
||||
never received anything.
|
||||
"""
|
||||
want = bl_ver[:4].encode("ascii").ljust(4, b"\x00")
|
||||
sent = 0
|
||||
deadline = time.monotonic() + 10.0
|
||||
while sent < times and time.monotonic() < deadline:
|
||||
self._pump(0.05)
|
||||
announced = False
|
||||
while True:
|
||||
msg = self._frames.take()
|
||||
if msg is None:
|
||||
break
|
||||
if msg[0] == MSG_NOTIFY_DEV_INFO:
|
||||
announced = True
|
||||
if announced:
|
||||
self._send(MSG_NOTIFY_BL_VER, want)
|
||||
sent += 1
|
||||
self._log("handshake sent %d time(s) (expecting %r)" % (sent, bl_ver[:4]))
|
||||
|
||||
def program(self, image: bytes, pages=None, retries: int = 3):
|
||||
"""Write *image* page by page. Returns the number of pages written."""
|
||||
total = (len(image) + PAGE_SIZE - 1) // PAGE_SIZE
|
||||
if pages is not None:
|
||||
total = min(total, pages)
|
||||
stamp = int(time.time() * 100) & 0xFFFFFFFF
|
||||
written = 0
|
||||
# Layout copied from the reference client: timestamp, page index, page count,
|
||||
# then FOUR reserved bytes before the payload -- the data starts at offset 16
|
||||
# of the message, not 12. Getting that wrong is silent: the device simply
|
||||
# never acknowledges the page.
|
||||
for index in range(total):
|
||||
page = image[index * PAGE_SIZE:(index + 1) * PAGE_SIZE]
|
||||
data = struct.pack("<IHH", stamp, index, total) + b"\x00" * 4 + page
|
||||
for attempt in range(retries):
|
||||
self._send(MSG_PROG_FW, data)
|
||||
self._pump(1.0)
|
||||
reply = None
|
||||
while True:
|
||||
msg = self._frames.take()
|
||||
if msg is None:
|
||||
break
|
||||
if msg[0] == MSG_PROG_FW_RESP:
|
||||
reply = msg[1]
|
||||
if reply is None:
|
||||
continue
|
||||
_stamp, page_index, err = struct.unpack_from("<IHH", reply, 0)
|
||||
if err == 0 and page_index == index:
|
||||
written += 1
|
||||
break
|
||||
else:
|
||||
raise SystemExit("page %d never acknowledged" % index)
|
||||
if (index + 1) % 32 == 0 or index + 1 == total:
|
||||
self._log(" programmed %d / %d pages" % (index + 1, total))
|
||||
return written
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description="program a UV-K5 over its serial bootloader")
|
||||
ap.add_argument("--endpoint", default="127.0.0.1:4568",
|
||||
help="host:port of a socket chardev, or a serial port name")
|
||||
ap.add_argument("--image", required=True, help="firmware image to program")
|
||||
ap.add_argument("--pages", type=int, default=None,
|
||||
help="program only the first N pages (for testing)")
|
||||
args = ap.parse_args()
|
||||
|
||||
with open(args.image, "rb") as fh:
|
||||
image = fh.read()
|
||||
|
||||
transport = open_transport(args.endpoint)
|
||||
flasher = Flasher(transport)
|
||||
bl_ver = flasher.wait_for_device()
|
||||
flasher.handshake(bl_ver)
|
||||
written = flasher.program(image, pages=args.pages)
|
||||
print("programmed %d page(s) from %s" % (written, args.image))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,224 @@
|
||||
"""External-flash firmware slots, in the format the multi-system firmware reads.
|
||||
|
||||
The layout and the header come from the firmware source, not from a datasheet or from
|
||||
the vendor's partition map:
|
||||
|
||||
mb_mb_flash.h MB_SLOT_STRIDE 0x20000, slot 1 base 0x040000, backup slot 0 0x020000
|
||||
mb_slot_header_t { magic "FMB1", hdr_version, flags, image_size,
|
||||
image_crc32, name[16], fw_version[16], reserved[16] }
|
||||
MB_FLAG_COMMITTED = 1
|
||||
|
||||
The image itself sits one 4 KiB sector past the slot base, so the header owns the first
|
||||
sector on its own -- MB_ValidateSlot() checks the magic and the CRC over image_size
|
||||
bytes, and the firmware reflashes itself from exactly this data when a slot is restored.
|
||||
|
||||
Why a small sector of slack: the multiboot code erases and programs in whole sectors, so
|
||||
a header sharing a sector with the image could not be updated without rewriting the
|
||||
image. The header's flags field says whether a slot is committed; an erased slot reads
|
||||
0xFF everywhere and validates as MB_ERR_MAGIC.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import struct
|
||||
import zlib
|
||||
|
||||
SLOT_MAGIC = 0x31424D46 # "FMB1"
|
||||
HDR_VERSION = 1
|
||||
MB_FLAG_COMMITTED = 1
|
||||
|
||||
SLOT_BACKUP_BASE = 0x020000 # slot 0: the backup of the internal firmware
|
||||
SLOT_STRIDE = 0x20000 # 128 KiB per slot
|
||||
SLOT_IMAGE_OFFSET = 0x1000 # image starts one sector in
|
||||
SLOT_COUNT = 5 # backup + 4
|
||||
SLOT_HEADER_SIZE = 64
|
||||
|
||||
# From mb_mb_flash.h: the two redundant active-state sectors.
|
||||
STATE_A_BASE = 0x00100000
|
||||
STATE_B_BASE = 0x00101000
|
||||
STATE_SIZE = 24 # mb_state_t, packed
|
||||
|
||||
|
||||
def slot_base(slot: int) -> int:
|
||||
"""External-flash base of @slot: 0 is the backup, 1..4 follow it."""
|
||||
if slot == 0:
|
||||
return SLOT_BACKUP_BASE
|
||||
if 1 <= slot < SLOT_COUNT:
|
||||
return SLOT_BACKUP_BASE + slot * SLOT_STRIDE
|
||||
raise ValueError("slot %r out of range (0..%d)" % (slot, SLOT_COUNT - 1))
|
||||
|
||||
|
||||
def build_header(image: bytes, name: str = "", fw_version: str = "") -> bytes:
|
||||
"""The 64-byte header for @image, committed."""
|
||||
if len(image) > SLOT_STRIDE - SLOT_IMAGE_OFFSET:
|
||||
raise ValueError("image is %d bytes; a slot holds %d"
|
||||
% (len(image), SLOT_STRIDE - SLOT_IMAGE_OFFSET))
|
||||
return struct.pack(
|
||||
"<IHHII16s16s16s",
|
||||
SLOT_MAGIC,
|
||||
HDR_VERSION,
|
||||
MB_FLAG_COMMITTED,
|
||||
len(image),
|
||||
zlib.crc32(image) & 0xFFFFFFFF,
|
||||
name.encode("ascii", "replace")[:15].ljust(16, b"\x00"),
|
||||
fw_version.encode("ascii", "replace")[:15].ljust(16, b"\x00"),
|
||||
b"\x00" * 16,
|
||||
)
|
||||
|
||||
|
||||
def write_slot(buf: bytearray, slot: int, image: bytes, name: str = "",
|
||||
fw_version: str = "") -> None:
|
||||
"""Lay @image into @slot of @buf, with a committed header."""
|
||||
base = slot_base(slot)
|
||||
header = build_header(image, name, fw_version)
|
||||
buf[base:base + SLOT_HEADER_SIZE] = header
|
||||
start = base + SLOT_IMAGE_OFFSET
|
||||
buf[start:start + len(image)] = image
|
||||
|
||||
|
||||
def erase_slot(buf: bytearray, slot: int) -> None:
|
||||
"""Leave @slot as an erased (invalid) slot."""
|
||||
base = slot_base(slot)
|
||||
buf[base:base + SLOT_STRIDE] = b"\xff" * SLOT_STRIDE
|
||||
|
||||
|
||||
def read_slot(buf: bytes, slot: int):
|
||||
"""(header dict, image bytes) for @slot, or (None, None) when it is not committed."""
|
||||
base = slot_base(slot)
|
||||
magic, ver, flags, size, crc = struct.unpack_from("<IHHII", buf, base)
|
||||
if magic != SLOT_MAGIC or not (flags & MB_FLAG_COMMITTED):
|
||||
return None, None
|
||||
name = buf[base + 16:base + 32].split(b"\x00")[0].decode("ascii", "replace")
|
||||
fw = buf[base + 32:base + 48].split(b"\x00")[0].decode("ascii", "replace")
|
||||
image = bytes(buf[base + SLOT_IMAGE_OFFSET:base + SLOT_IMAGE_OFFSET + size])
|
||||
return ({"slot": slot, "base": base, "hdr_version": ver, "flags": flags,
|
||||
"image_size": size, "image_crc32": crc, "crc_ok": (zlib.crc32(image) & 0xFFFFFFFF) == crc,
|
||||
"name": name, "fw_version": fw}, image)
|
||||
|
||||
|
||||
def erase_state(buf: bytearray) -> None:
|
||||
"""Erase both active-state sectors, so the firmware treats the flash as fresh.
|
||||
|
||||
This matters: a *corrupt* marker plus a valid slot 0 makes the boot path halt with
|
||||
"STATE ERROR" to protect Main (mb_multiboot.c), while a *missing* one lets it look
|
||||
at the slots and record what it finds.
|
||||
"""
|
||||
buf[STATE_A_BASE:STATE_A_BASE + STATE_SIZE] = b"\xff" * STATE_SIZE
|
||||
buf[STATE_B_BASE:STATE_B_BASE + STATE_SIZE] = b"\xff" * STATE_SIZE
|
||||
|
||||
|
||||
def build(base_image_path: str, slots, out_path: str, erase_state_sectors: bool = True) -> str:
|
||||
"""Copy @base_image_path and put @slots (a list of (slot, image_path, name, version)) in it.
|
||||
|
||||
@slots entries: (slot_number, path_to_bin, name, fw_version).
|
||||
"""
|
||||
with open(base_image_path, "rb") as fh:
|
||||
buf = bytearray(fh.read())
|
||||
for slot, image_path, name, version in slots:
|
||||
with open(image_path, "rb") as fh:
|
||||
image = fh.read()
|
||||
write_slot(buf, slot, image, name, version)
|
||||
if erase_state_sectors:
|
||||
erase_state(buf)
|
||||
with open(out_path, "wb") as fh:
|
||||
fh.write(buf)
|
||||
return out_path
|
||||
|
||||
|
||||
def describe(path: str) -> str:
|
||||
"""A human-readable summary of every slot in a flash image."""
|
||||
with open(path, "rb") as fh:
|
||||
buf = fh.read()
|
||||
lines = ["%s (%d bytes)" % (path, len(buf))]
|
||||
for slot in range(SLOT_COUNT):
|
||||
header, image = read_slot(buf, slot)
|
||||
if header is None:
|
||||
lines.append(" slot %d @0x%06x: empty" % (slot, slot_base(slot)))
|
||||
else:
|
||||
lines.append(" slot %d @0x%06x: %-16s %-16s %7d bytes crc %08x %s"
|
||||
% (slot, header["base"], header["name"], header["fw_version"],
|
||||
header["image_size"], header["image_crc32"],
|
||||
"ok" if header["crc_ok"] else "MISMATCH"))
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
|
||||
ap = argparse.ArgumentParser(description="inspect or build external-flash firmware slots")
|
||||
ap.add_argument("image", help="flash image to inspect")
|
||||
ap.add_argument("--base", help="base image to copy when building")
|
||||
ap.add_argument("--slot", type=int, action="append", default=[],
|
||||
help="slot number to fill (repeat; pair with --bin in order)")
|
||||
ap.add_argument("--bin", action="append", default=[], help="image for the matching --slot")
|
||||
ap.add_argument("--name", action="append", default=[], help="slot name for the matching --slot")
|
||||
ap.add_argument("--out", help="where to write the built image")
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.base and args.out:
|
||||
slots = []
|
||||
for i, slot in enumerate(args.slot):
|
||||
name = args.name[i] if i < len(args.name) else os.path.basename(args.bin[i])
|
||||
slots.append((slot, args.bin[i], name, ""))
|
||||
print("wrote", build(args.base, slots, args.out))
|
||||
print(describe(args.out or args.image))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------- editing files
|
||||
|
||||
def load_image(path: str) -> bytearray:
|
||||
with open(path, "rb") as fh:
|
||||
return bytearray(fh.read())
|
||||
|
||||
|
||||
def save_image(path: str, buf: bytes) -> None:
|
||||
"""Write @buf to @path, atomically enough that a crash cannot truncate the image.
|
||||
|
||||
The emulator writes this same file back while it runs, so a half-written image is
|
||||
a real possibility -- and a truncated flash image looks like a fresh radio.
|
||||
"""
|
||||
tmp = path + ".tmp"
|
||||
with open(tmp, "wb") as fh:
|
||||
fh.write(buf)
|
||||
fh.flush()
|
||||
os.fsync(fh.fileno())
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
def slots_json(path: str):
|
||||
"""The slot table as plain data, for a page or a JSON API."""
|
||||
buf = load_image(path)
|
||||
slots = []
|
||||
for slot in range(SLOT_COUNT):
|
||||
header, _image = read_slot(buf, slot)
|
||||
row = {"slot": slot, "base": slot_base(slot), "empty": header is None}
|
||||
if header is not None:
|
||||
row.update({k: header[k] for k in
|
||||
("name", "fw_version", "image_size", "image_crc32", "crc_ok",
|
||||
"hdr_version", "flags")})
|
||||
slots.append(row)
|
||||
return {"image": os.path.abspath(path), "name": os.path.basename(path),
|
||||
"size": len(buf), "slots": slots}
|
||||
|
||||
|
||||
def write_slot_file(path: str, slot: int, image: bytes, name: str = "",
|
||||
fw_version: str = "", erase_state_sectors: bool = True) -> dict:
|
||||
"""Put @image into @slot of the flash image at @path, in place."""
|
||||
buf = load_image(path)
|
||||
write_slot(buf, slot, image, name, fw_version)
|
||||
if erase_state_sectors:
|
||||
# The firmware's boot path treats a corrupt active-state marker next to a valid
|
||||
# slot 0 as "halt and protect Main" (mb_multiboot.c), which reads as the radio
|
||||
# refusing to boot. Erasing the marker lets it re-decide from the slots.
|
||||
erase_state(buf)
|
||||
save_image(path, bytes(buf))
|
||||
return slots_json(path)["slots"][slot]
|
||||
|
||||
|
||||
def erase_slot_file(path: str, slot: int) -> dict:
|
||||
buf = load_image(path)
|
||||
erase_slot(buf, slot)
|
||||
erase_state(buf)
|
||||
save_image(path, bytes(buf))
|
||||
return slots_json(path)["slots"][slot]
|
||||
@@ -0,0 +1,302 @@
|
||||
"""Write the firmware's own firmware slots over the serial link.
|
||||
|
||||
The multi-system release exposes its slots to a host (App/app/uart.c, "Firmware Slots"):
|
||||
|
||||
0x0720 slot info Data[0] = slot -> 0x0721 Slot, Status, Hdr[64]
|
||||
0x0722 slot erase Data[0] = slot, Data[2..5] = ts -> 0x0723 Slot, Status
|
||||
0x0724 slot write Data[0] = slot, Data[2..5] = offset, Data[6..7] = len,
|
||||
Data[8..11] = ts, Data[12..] = data -> 0x0725 Slot, Status
|
||||
0x0726 slot validate Data[0] = slot -> 0x0727 Crc32, Slot, Status
|
||||
|
||||
Every one of them is gated on the timestamp the 0x0514 session handshake latched
|
||||
(UART_Timestamp), exactly like the EEPROM write (CMD_051D): send a different one and the
|
||||
firmware answers MB_ERR_AUTH without doing anything.
|
||||
|
||||
The frames are the AB CD .. DC BA protocol the rest of the programming interface uses, so
|
||||
uvk5_serial_flash builds them and this only adds the message ids and payload layouts.
|
||||
|
||||
Nothing in the emulator's own path needs this: the page writes slots straight into the
|
||||
flash image. It exists because it is the path a real radio takes, and the only way to
|
||||
write a slot on hardware.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import socket
|
||||
import struct
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import zlib
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import uvk5_serial_flash as proto # build(), Frames
|
||||
import uvk5_slots as slots # the header layout, in one place
|
||||
|
||||
MSG_SLOT_INFO = 0x0720
|
||||
MSG_SLOT_INFO_ACK = 0x0721
|
||||
MSG_SLOT_ERASE = 0x0722
|
||||
MSG_SLOT_ERASE_ACK = 0x0723
|
||||
MSG_SLOT_WRITE = 0x0724
|
||||
MSG_SLOT_WRITE_ACK = 0x0725
|
||||
MSG_SLOT_VALIDATE = 0x0726
|
||||
MSG_SLOT_VALIDATE_ACK = 0x0727
|
||||
|
||||
STATUS = {
|
||||
0: "ok",
|
||||
1: "no/invalid slot header",
|
||||
2: "header format too new",
|
||||
3: "image not marked committed",
|
||||
4: "image size out of range",
|
||||
5: "image CRC mismatch",
|
||||
6: "external flash timed out",
|
||||
7: "slot index out of range",
|
||||
8: "timestamp mismatch",
|
||||
9: "restore stub RAM mismatch",
|
||||
}
|
||||
|
||||
|
||||
CHUNK = 200 # see Radio.write
|
||||
|
||||
|
||||
class SlotError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class Radio:
|
||||
"""A connection to the firmware's slot commands."""
|
||||
|
||||
def __init__(self, endpoint, timeout: float = 6.0, log=print):
|
||||
"""Talk over @endpoint (host:port, or a device path), or an open socket.
|
||||
|
||||
An already-connected socket is accepted because a test that wants QEMU to be
|
||||
the one connecting has to listen first, and handing the accepted socket in is
|
||||
simpler than racing on a free port.
|
||||
"""
|
||||
self.log = log
|
||||
if hasattr(endpoint, "recv"):
|
||||
self.sock = endpoint
|
||||
self.sock.settimeout(timeout)
|
||||
else:
|
||||
self.sock = proto.open_transport(endpoint)
|
||||
self.sock.settimeout(timeout)
|
||||
self.frames = proto.Frames()
|
||||
self.timestamp = (int(time.time() * 100) & 0xFFFFFFFF)
|
||||
self._lock = threading.Lock()
|
||||
self._stop = False
|
||||
# Drain the port on a thread of its own. The firmware streams its screen over
|
||||
# this same link (K5Viewer), so a client that only reads when it is waiting for
|
||||
# a reply backs the socket up and the *guest* then blocks writing to it:
|
||||
# measured, a single 64-byte slot write took six seconds that way, and longer
|
||||
# transfers lost replies and stalled. Reading continuously is what keeps the
|
||||
# radio responsive.
|
||||
self._reader = threading.Thread(target=self._read_loop, daemon=True)
|
||||
self._reader.start()
|
||||
|
||||
def close(self):
|
||||
self._stop = True
|
||||
try:
|
||||
self.sock.close()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def _read_loop(self):
|
||||
while not self._stop:
|
||||
try:
|
||||
chunk = self.sock.recv(65536)
|
||||
except (socket.timeout, OSError):
|
||||
continue
|
||||
if not chunk:
|
||||
return
|
||||
if os.environ.get("UVK5_SLOT_DEBUG"):
|
||||
self.log("rx %s" % chunk.hex()[:120])
|
||||
with self._lock:
|
||||
self.frames.feed(chunk)
|
||||
|
||||
def _pump(self, seconds: float, want=None):
|
||||
"""Wait for @want, or @seconds, whichever comes first.
|
||||
|
||||
The reading itself happens on the reader thread; this only looks at what it has
|
||||
reassembled, so waiting never stops the port being drained.
|
||||
"""
|
||||
end = time.monotonic() + seconds
|
||||
got = {}
|
||||
while time.monotonic() < end:
|
||||
with self._lock:
|
||||
while True:
|
||||
msg = self.frames.take()
|
||||
if msg is None:
|
||||
break
|
||||
got[msg[0]] = msg[1]
|
||||
if want is not None and want in got:
|
||||
return got
|
||||
time.sleep(0.002)
|
||||
return got
|
||||
|
||||
def session(self):
|
||||
"""0x0514, which latches our timestamp on the device."""
|
||||
# CMD_0514_t: the timestamp is what matters; the rest is padding.
|
||||
body = struct.pack("<I", self.timestamp) + b"\x00" * 4
|
||||
frame = proto.build(0x0514, body)
|
||||
# Retry rather than send once: the first 0x0514 is what makes the firmware enter
|
||||
# its serial mode, and how long that takes depends on what the main loop is
|
||||
# busy with, so a single handshake either arrives before it is listening or is
|
||||
# simply too early. Each retry is cheap and the reply is unambiguous.
|
||||
ack = None
|
||||
for attempt in range(6):
|
||||
if os.environ.get("UVK5_SLOT_DEBUG"):
|
||||
self.log("tx [%d] %s" % (attempt, frame.hex()))
|
||||
self.sock.sendall(frame)
|
||||
replies = self._pump(2.5, want=0x0515)
|
||||
ack = replies.get(0x0515)
|
||||
if ack is not None:
|
||||
break
|
||||
if ack is None:
|
||||
raise SlotError("no 0x0515 reply after 6 handshakes: is the firmware "
|
||||
"running, and is the serial port the programming one?")
|
||||
version = ack[:16].split(b"\x00")[0].decode("ascii", "replace")
|
||||
self.log("session timestamp %08x, firmware %s" % (self.timestamp, version))
|
||||
return version
|
||||
|
||||
def _command(self, msg_id: int, data: bytes, ack_id: int, wait: float = 4.0,
|
||||
tries: int = 4):
|
||||
"""Send @data and wait for @ack_id, resending a few times.
|
||||
|
||||
The firmware drops the odd command when it is busy elsewhere -- the first one
|
||||
after the handshake most often -- and a dropped command looks exactly like one
|
||||
the firmware does not implement. Everything here is idempotent (a repeated erase
|
||||
erases, a repeated chunk write writes the same bytes), so resending is safe.
|
||||
"""
|
||||
frame = proto.build(msg_id, data)
|
||||
for attempt in range(tries):
|
||||
self.sock.sendall(frame)
|
||||
replies = self._pump(wait, want=ack_id)
|
||||
ack = replies.get(ack_id)
|
||||
if ack is not None:
|
||||
return ack
|
||||
if os.environ.get("UVK5_SLOT_DEBUG"):
|
||||
self.log("0x%04x: no reply on attempt %d" % (msg_id, attempt + 1))
|
||||
# The firmware leaves its serial mode after ~6 s without a session
|
||||
# handshake (gSerialConfigCountDown_500ms = 12), and a slot write is slow
|
||||
# enough in host time to cross that: measured, the writes stop dead and the
|
||||
# guest spins at one address until a fresh 0x0514 restores it. Handshaking
|
||||
# again is what makes the rest of the transfer land.
|
||||
if attempt + 1 < tries:
|
||||
try:
|
||||
self.session()
|
||||
except SlotError:
|
||||
pass
|
||||
return None
|
||||
|
||||
def info(self, slot: int):
|
||||
"""(status, header bytes) for @slot, without a CRC pass."""
|
||||
ack = self._command(MSG_SLOT_INFO, bytes([slot]), MSG_SLOT_INFO_ACK)
|
||||
if ack is None:
|
||||
raise SlotError("0x0720 got no reply")
|
||||
return ack[1], ack[2:66]
|
||||
|
||||
def erase(self, slot: int):
|
||||
data = bytes([slot, 0]) + struct.pack("<I", self.timestamp)
|
||||
ack = self._command(MSG_SLOT_ERASE, data, MSG_SLOT_ERASE_ACK, wait=15.0)
|
||||
if ack is None:
|
||||
raise SlotError("0x0722 got no reply")
|
||||
return ack[1]
|
||||
|
||||
def write(self, slot: int, offset: int, blob: bytes):
|
||||
log = getattr(self, "log", None) or (lambda *a: None)
|
||||
# 200 bytes of data, not 240: the frame carries 8 bytes of framing, 4 of header
|
||||
# and 12 of payload before the data, and the firmware's receive buffer is 256
|
||||
# bytes (App/driver/uart.c: UART_DMA_Buffer[256]). A full-size chunk overran it
|
||||
# and the firmware silently dropped every one of them.
|
||||
began = time.monotonic()
|
||||
chunk_no = 0
|
||||
for start in range(0, len(blob), CHUNK):
|
||||
piece = blob[start:start + CHUNK]
|
||||
chunk_no += 1
|
||||
if chunk_no > 1 and (chunk_no - 1) % 25 == 0:
|
||||
# The firmware leaves its serial mode after ~6 s without a session
|
||||
# handshake (gSerialConfigCountDown_500ms = 12), and this transfer runs
|
||||
# longer than that. Measured: the writes stop dead at that point and only
|
||||
# a fresh 0x0514 revives them, so renew well inside the window.
|
||||
self.session()
|
||||
data = (bytes([slot, 0]) + struct.pack("<I", offset + start)
|
||||
+ struct.pack("<H", len(piece))
|
||||
+ struct.pack("<I", self.timestamp) + piece)
|
||||
ack = self._command(MSG_SLOT_WRITE, data, MSG_SLOT_WRITE_ACK)
|
||||
if ack is None:
|
||||
raise SlotError("0x0724 got no reply at offset 0x%x" % (offset + start))
|
||||
if ack[1] != 0:
|
||||
raise SlotError("0x0724 offset 0x%x: %s"
|
||||
% (offset + start, STATUS.get(ack[1], ack[1])))
|
||||
if chunk_no % 100 == 0 or start + CHUNK >= len(blob):
|
||||
log(" %6d/%d bytes, %.1fs" % (start + len(piece), len(blob),
|
||||
time.monotonic() - began))
|
||||
|
||||
def validate(self, slot: int):
|
||||
ack = self._command(MSG_SLOT_VALIDATE, bytes([slot]), MSG_SLOT_VALIDATE_ACK,
|
||||
wait=20.0)
|
||||
if ack is None:
|
||||
raise SlotError("0x0726 got no reply")
|
||||
crc = struct.unpack_from("<I", ack, 0)[0]
|
||||
return ack[5], crc
|
||||
|
||||
|
||||
def install(radio: Radio, slot: int, image: bytes, name: str = "",
|
||||
version: str = "", log=print):
|
||||
"""Erase @slot, write the header and the image, then validate the CRC."""
|
||||
header = slots.build_header(image, name, version)
|
||||
log("erase slot %d" % slot)
|
||||
status = radio.erase(slot)
|
||||
if status != 0:
|
||||
raise SlotError("erase: %s" % STATUS.get(status, status))
|
||||
log("write header (%d bytes) and image (%d bytes)" % (len(header), len(image)))
|
||||
radio.write(slot, 0, header)
|
||||
radio.write(slot, slots.SLOT_IMAGE_OFFSET, image)
|
||||
status, crc = radio.validate(slot)
|
||||
if status != 0:
|
||||
raise SlotError("validate: %s" % STATUS.get(status, status))
|
||||
if crc != (zlib.crc32(image) & 0xFFFFFFFF):
|
||||
raise SlotError("device CRC %08x does not match the host's %08x"
|
||||
% (crc, zlib.crc32(image) & 0xFFFFFFFF))
|
||||
log("slot %d validates: crc %08x" % (slot, crc))
|
||||
return crc
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description="write a firmware slot over the serial link")
|
||||
ap.add_argument("--endpoint", required=True,
|
||||
help="host:port or unix socket of the emulator's serial chardev")
|
||||
ap.add_argument("--slot", type=int, required=True)
|
||||
ap.add_argument("--image", help="firmware .bin to put in the slot (not needed for --info)")
|
||||
ap.add_argument("--name", default="", help="name to store in the header")
|
||||
ap.add_argument("--version", default="", help="version string to store")
|
||||
ap.add_argument("--info", action="store_true", help="only read the slot header")
|
||||
args = ap.parse_args(argv)
|
||||
if not args.info and not args.image:
|
||||
ap.error("--image is required unless --info is given")
|
||||
|
||||
radio = Radio(args.endpoint)
|
||||
try:
|
||||
radio.session()
|
||||
if args.info:
|
||||
status, header = radio.info(args.slot)
|
||||
print("slot %d: %s" % (args.slot, STATUS.get(status, status)))
|
||||
if status == 0:
|
||||
print(" header:", header.hex())
|
||||
return 0
|
||||
with open(args.image, "rb") as fh:
|
||||
image = fh.read()
|
||||
install(radio, args.slot, image, args.name, args.version)
|
||||
return 0
|
||||
except SlotError as exc:
|
||||
print("failed: %s" % exc, file=sys.stderr)
|
||||
return 1
|
||||
finally:
|
||||
radio.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,126 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Sockets for talking to QEMU, on platforms that may not have unix sockets.
|
||||
|
||||
The tools and tests used to hardcode socket.AF_UNIX, which a Windows QEMU cannot
|
||||
create -- so the emulator and the page worked there while the *verification* path did
|
||||
not, which is the half that matters. Everything here returns an endpoint string in the
|
||||
form QEMU expects, so a caller never has to know which kind it got:
|
||||
|
||||
srv, endpoint = uvk5_socket.listen("qmp") # QEMU connects to this
|
||||
qemu_args = ["-qmp", endpoint]
|
||||
|
||||
That endpoint is unix:/tmp/uvk5-XXXX/qmp.sock where unix sockets exist, and
|
||||
tcp:127.0.0.1:<port>,server=on,wait=off otherwise. QmpClient and the supervisor already
|
||||
accept both forms; the listener side is what was missing.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import socket
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
HAVE_UNIX = hasattr(socket, "AF_UNIX")
|
||||
|
||||
|
||||
def can_use_unix():
|
||||
"""True when a unix socket can actually be bound, not merely that the name exists."""
|
||||
if not HAVE_UNIX:
|
||||
return False
|
||||
probe_dir = tempfile.mkdtemp(prefix="uvk5-probe-")
|
||||
path = os.path.join(probe_dir, "probe.sock")
|
||||
try:
|
||||
probe = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
try:
|
||||
probe.bind(path)
|
||||
finally:
|
||||
probe.close()
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
finally:
|
||||
for cleanup in (lambda: os.unlink(path), lambda: os.rmdir(probe_dir)):
|
||||
try:
|
||||
cleanup()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def listen(name: str = "qmp", directory: str | None = None):
|
||||
"""A listening socket plus the endpoint for QEMU to connect *out* to.
|
||||
|
||||
Use server_endpoint() instead when QEMU should be the one listening.
|
||||
|
||||
Returns (socket, endpoint). The caller closes the socket and removes the directory
|
||||
it was given, if any.
|
||||
"""
|
||||
keep = directory or tempfile.mkdtemp(prefix="uvk5-%s-" % name)
|
||||
if can_use_unix():
|
||||
path = os.path.join(keep, name + ".sock")
|
||||
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
srv.bind(path)
|
||||
srv.listen(1)
|
||||
return srv, "unix:" + path
|
||||
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
srv.bind(("127.0.0.1", 0))
|
||||
srv.listen(1)
|
||||
return srv, "tcp:127.0.0.1:%d" % srv.getsockname()[1]
|
||||
|
||||
|
||||
def server_endpoint(name: str = "qmp", directory: str | None = None) -> str:
|
||||
"""An endpoint for QEMU to LISTEN on -- the caller then connects to it.
|
||||
|
||||
This is the direction the emulator tests use, and the opposite of listen():
|
||||
with server=on QEMU binds the port itself, so a listener held by the test makes
|
||||
QEMU fail to start and the test then connects to its own socket and waits for a
|
||||
greeting that can never come. That mistake cost a debugging round here.
|
||||
"""
|
||||
if can_use_unix():
|
||||
keep = directory or tempfile.mkdtemp(prefix="uvk5-%s-" % name)
|
||||
return "unix:" + os.path.join(keep, name + ".sock") + ",server=on,wait=off"
|
||||
return "tcp:127.0.0.1:%d,server=on,wait=off" % free_port()
|
||||
|
||||
|
||||
def free_port() -> int:
|
||||
"""A port nothing is listening on, for a caller that wants to name one itself."""
|
||||
probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
probe.bind(("127.0.0.1", 0))
|
||||
port = probe.getsockname()[1]
|
||||
probe.close()
|
||||
return port
|
||||
|
||||
|
||||
def connect(endpoint: str, timeout: float = 20.0):
|
||||
"""Connect to whatever listen() returned, waiting for it to appear."""
|
||||
kind, _, rest = endpoint.partition(":")
|
||||
if kind not in ("unix", "tcp"):
|
||||
# A bare "host:port", which is what the supervisor, the web UI and the README
|
||||
# have always passed. Read as a scheme, its "host" is the address itself and
|
||||
# the port is empty, so the connect goes to an empty host and hangs until the
|
||||
# deadline -- which reads as "the emulator never started" while the guest is
|
||||
# running happily. That shipped once and cost a page that would not power on.
|
||||
kind, rest = "tcp", endpoint
|
||||
deadline = time.monotonic() + timeout
|
||||
last = None
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
if kind == "unix":
|
||||
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
sock.settimeout(2.0)
|
||||
sock.connect(rest.partition(",")[0])
|
||||
|
||||
return sock
|
||||
host, _, port = rest.partition(",")[0].rpartition(":")
|
||||
return socket.create_connection((host, int(port)), timeout=2.0)
|
||||
except OSError as exc:
|
||||
last = exc
|
||||
time.sleep(0.2)
|
||||
raise OSError("could not connect to %s: %s" % (endpoint, last))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
srv, endpoint = listen("demo")
|
||||
print("unix sockets available:", can_use_unix())
|
||||
print("listening on", endpoint)
|
||||
srv.close()
|
||||
+29
-6
@@ -17,15 +17,17 @@ looks live.
|
||||
import threading
|
||||
import time
|
||||
|
||||
from uvk5_lcd import FrameGrabber, encode_png, unpack
|
||||
from uvk5_lcd import FrameGrabber, default_spool_dir, encode_png, unpack
|
||||
|
||||
|
||||
class FramePump:
|
||||
def __init__(self, client, frame_addr: int, status_addr: int,
|
||||
fps: int = 15, scale: int = 4, spool_dir: str = "/dev/shm"):
|
||||
fps: int = 15, scale: int = 4, spool_dir: str = None):
|
||||
self._frame_addr = frame_addr
|
||||
self._status_addr = status_addr
|
||||
self._spool_dir = spool_dir
|
||||
# Resolved by FrameGrabber: /dev/shm on Linux, the temp directory on
|
||||
# Windows, where /dev/shm does not exist.
|
||||
self._spool_dir = spool_dir or default_spool_dir()
|
||||
self._interval = 1.0 / fps
|
||||
self._scale = scale
|
||||
self._lock = threading.Lock()
|
||||
@@ -68,24 +70,45 @@ class FramePump:
|
||||
grabber = self._grabber
|
||||
if grabber is not None:
|
||||
try:
|
||||
status, frame = grabber.raw()
|
||||
status, frame, pixels = self._grab(grabber)
|
||||
current = (status, frame)
|
||||
with self._lock:
|
||||
# Re-check: a rebind may have landed mid-read, and its
|
||||
# blanking must not be undone by this stale frame.
|
||||
if self._grabber is grabber and current != self._raw:
|
||||
self._raw = current
|
||||
self._png = encode_png(unpack(status, frame),
|
||||
self._scale)
|
||||
self._png = encode_png(pixels, self._scale)
|
||||
self._generation += 1
|
||||
except Exception:
|
||||
# A dead emulator must not kill the pump: power may come
|
||||
# back, and latest() keeps serving the last good frame.
|
||||
pass
|
||||
# A dead emulator must not kill the pump: power may come
|
||||
# back, and latest() keeps serving the last good frame.
|
||||
pass
|
||||
slack = self._interval - (time.monotonic() - started)
|
||||
if slack > 0:
|
||||
self._stop.wait(slack)
|
||||
|
||||
|
||||
def _grab(self, grabber):
|
||||
"""One frame: (status, frame, pixels), from the panel if it is there.
|
||||
|
||||
Every firmware pushes its pixels through the display controller, so the
|
||||
controller's memory is the screen no matter where that build keeps its own
|
||||
buffers -- and builds sharing an ancestor still differ in their display
|
||||
logic, which is why guessing guest addresses does not generalise.
|
||||
|
||||
Guest RAM remains the fallback for an emulator built without the panel
|
||||
model, and is what the tests stub.
|
||||
"""
|
||||
try:
|
||||
pixels = grabber.panel_pixels()
|
||||
gram = grabber.panel_gram()
|
||||
return gram[:STATUS_BYTES], gram[STATUS_BYTES:], pixels
|
||||
except Exception:
|
||||
status, frame = grabber.raw()
|
||||
return status, frame, unpack(status, frame)
|
||||
def latest(self):
|
||||
with self._lock:
|
||||
return self._png
|
||||
|
||||
+182
-12
@@ -22,20 +22,110 @@ import time
|
||||
DEFAULT_QMP = "/tmp/uvk5-qmp.sock"
|
||||
|
||||
|
||||
def default_launcher(qemu: str, flash: str, elf: str,
|
||||
def qmp_argument(endpoint: str) -> str:
|
||||
"""The -qmp argument for an endpoint, unix or TCP.
|
||||
|
||||
Linux gets a unix socket. A Windows build of QEMU cannot create one at all, so
|
||||
"host:port" is passed through as a TCP listener instead -- the same shape the
|
||||
QMP client and wait_for_socket already accept.
|
||||
"""
|
||||
host, _, port = endpoint.rpartition(":")
|
||||
if host and port.isdigit():
|
||||
return f"tcp:{host}:{port},server=on,wait=off"
|
||||
return f"unix:{endpoint},server=on,wait=off"
|
||||
|
||||
|
||||
class FlashSlot:
|
||||
"""The external-flash image the emulator boots from.
|
||||
|
||||
Mutable and read at spawn time, like the image slot: the page writes a firmware
|
||||
into one of its slots (or swaps the whole image) and powers on, and nobody has to
|
||||
re-create the launcher or restart the server.
|
||||
"""
|
||||
|
||||
def __init__(self, path):
|
||||
self.path = path
|
||||
|
||||
|
||||
class BootKey:
|
||||
"""A key to hold from reset on the next power-on, and for how long.
|
||||
|
||||
Mutable and read at spawn time, like the image slot: the page turns it on for one
|
||||
boot and off again. Getting a boot mode by hand otherwise means pausing the VM,
|
||||
setting the keypad over QMP and continuing -- which is fine in a script and
|
||||
unusable from a browser.
|
||||
"""
|
||||
|
||||
def __init__(self, name=None, hold_ms=1500):
|
||||
self.name = name
|
||||
self.hold_ms = hold_ms
|
||||
|
||||
def clear(self):
|
||||
self.name = None
|
||||
|
||||
|
||||
def resolve_image(image):
|
||||
"""(path, app_offset) for whatever was handed to the launcher.
|
||||
|
||||
Accepts a path, an ImageInfo, or an ImageSlot. The slot is what the web UI
|
||||
passes, and it is read *here*, at spawn time, so uploading a firmware takes
|
||||
effect at the next power-on without restarting the server.
|
||||
|
||||
app_offset is None when nothing is known, in which case the machine's own
|
||||
default (an application at 0x2800) is used.
|
||||
"""
|
||||
if hasattr(image, "current"): # ImageSlot
|
||||
image = image.current
|
||||
if image is None:
|
||||
return None, None
|
||||
if isinstance(image, str):
|
||||
return image, None
|
||||
return image.path, image.app_offset
|
||||
|
||||
|
||||
def default_launcher(qemu: str, flash: str, elf, boot_key=None,
|
||||
qmp_path: str = DEFAULT_QMP, gdb_port: int = 1234,
|
||||
capture_stderr: bool = True):
|
||||
"""Reproduces the command line in tools/run.sh."""
|
||||
"""Reproduces the command line in tools/run.sh.
|
||||
|
||||
`elf` may be a path, an ImageInfo, or an ImageSlot -- the last is what the web UI
|
||||
passes so an uploaded firmware takes effect at the next power-on.
|
||||
"""
|
||||
def launch():
|
||||
path, app_offset = resolve_image(elf)
|
||||
if path is None:
|
||||
raise RuntimeError("no firmware loaded: upload a .bin first")
|
||||
|
||||
# No app-offset here on purpose: the machine works out the image's shape from
|
||||
# the image (see uvk5_sniff_app_offset). Passing it as a -machine property
|
||||
# looked equivalent and was not -- through this launcher QEMU rejected the
|
||||
# whole machine string with "unsupported machine type", while the identical
|
||||
# argv run by hand started fine. One less thing to get wrong.
|
||||
machine = "uv-k5-v3"
|
||||
|
||||
# The flash image travels in the environment rather than as a -machine
|
||||
# property. Same reasoning as above: -M with properties was rejected by QEMU
|
||||
# when this launcher spawned it (and only then), and the environment is a
|
||||
# channel that arrives intact. The model reads UVK5_FLASH_IMAGE as a fallback.
|
||||
env = dict(os.environ)
|
||||
env["UVK5_FLASH_IMAGE"] = getattr(flash, "path", flash)
|
||||
|
||||
# Same channel for the boot key: a -machine property list was rejected by
|
||||
# QEMU through this launcher, and the environment arrives intact.
|
||||
name = getattr(boot_key, "name", None) if boot_key is not None else None
|
||||
if name:
|
||||
env["UVK5_BOOT_KEY"] = name
|
||||
env["UVK5_BOOT_KEY_MS"] = str(getattr(boot_key, "hold_ms", 1500))
|
||||
# A stale socket makes QEMU fail to bind, which looks like "power on did
|
||||
# nothing". Clear it first.
|
||||
if os.path.exists(qmp_path):
|
||||
# nothing". Clear it first -- but only for a socket file there is one of.
|
||||
if ":" not in qmp_path and os.path.exists(qmp_path):
|
||||
os.unlink(qmp_path)
|
||||
return subprocess.Popen(
|
||||
[qemu, "-M", f"uv-k5-v3,flash-image={flash}",
|
||||
[qemu, "-M", machine,
|
||||
"-nographic", "-monitor", "none",
|
||||
"-qmp", f"unix:{qmp_path},server=on,wait=off",
|
||||
"-kernel", elf, "-gdb", f"tcp::{gdb_port}"],
|
||||
"-qmp", qmp_argument(qmp_path),
|
||||
"-kernel", path, "-gdb", "tcp::%d" % gdb_port],
|
||||
env=env,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.PIPE if capture_stderr else subprocess.DEVNULL)
|
||||
return launch
|
||||
@@ -50,9 +140,19 @@ def wait_for_socket(path: str, timeout: float = 15.0) -> bool:
|
||||
ECONNREFUSED -- which surfaces as power on returning 500. Probing with a real
|
||||
connect distinguishes "listening" from "leftover file".
|
||||
"""
|
||||
# "host:port" is a TCP QMP endpoint, which is all a Windows build of QEMU
|
||||
# can offer; there is no socket file to stat, so probe the port directly.
|
||||
host, _, port = path.rpartition(":")
|
||||
tcp = bool(host) and port.isdigit()
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if os.path.exists(path):
|
||||
if tcp:
|
||||
try:
|
||||
socket.create_connection((host, int(port)), timeout=1.0).close()
|
||||
return True
|
||||
except OSError:
|
||||
pass
|
||||
elif os.path.exists(path):
|
||||
probe = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
try:
|
||||
probe.settimeout(1.0)
|
||||
@@ -108,6 +208,47 @@ class Supervisor:
|
||||
self._client = client
|
||||
self._proc = None
|
||||
|
||||
def _start_stderr_pump(self, proc) -> bool:
|
||||
"""Forward QEMU's stderr to the log, and never stop draining it.
|
||||
|
||||
This has to read the pipe unconditionally, because QEMU blocks on write when it
|
||||
fills -- which stops its main loop, and then QMP never answers and the guest looks
|
||||
dead. The firmware streams its display down this same pipe (the model tags it
|
||||
SERIAL), so it fills with binary that has no line breaks in it, and 64 KB is
|
||||
reached in about a second.
|
||||
|
||||
Measured: with the pipe drained the launcher's QEMU accepts QMP in 0.5 s; with it
|
||||
left unread, the same command line never answers at all. So: read in fixed-size
|
||||
chunks (a readline() on a stream with no newlines hoards it), decode leniently, and
|
||||
swallow anything the log throws -- a logging failure must not become a stopped
|
||||
drain, which is a deadlock rather than a lost line.
|
||||
"""
|
||||
stream = getattr(proc, "stderr", None)
|
||||
if stream is None:
|
||||
return False
|
||||
|
||||
def drain():
|
||||
while True:
|
||||
try:
|
||||
chunk = stream.read(65536)
|
||||
except Exception:
|
||||
return
|
||||
if not chunk:
|
||||
return
|
||||
if self._log is None:
|
||||
continue
|
||||
try:
|
||||
text = chunk.decode("utf-8", "replace")
|
||||
for line in text.splitlines():
|
||||
if line:
|
||||
self._log.add("qemu", line[:400])
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
threading.Thread(target=drain, daemon=True).start()
|
||||
return True
|
||||
|
||||
|
||||
def power_on(self) -> bool:
|
||||
with self._lock:
|
||||
# A client object is not proof of a live guest. If the process died
|
||||
@@ -125,6 +266,14 @@ class Supervisor:
|
||||
if self._client is not None:
|
||||
return False
|
||||
self._proc = self._launch()
|
||||
# Drain QEMU's stderr from the moment it starts, not after a successful
|
||||
# connect. The firmware streams its screen down that pipe -- the model tags
|
||||
# it SERIAL -- and 64 KB of it fills the pipe while we are still waiting for
|
||||
# QMP. QEMU then blocks writing to stderr, its main loop stops, and the
|
||||
# connect times out with the guest perfectly healthy. That reads as "the
|
||||
# emulator never started", and it is why power-on worked with some firmware
|
||||
# and not others: only the ones that stream hard fill the pipe in time.
|
||||
draining = self._start_stderr_pump(self._proc)
|
||||
try:
|
||||
self._client = self._connect()
|
||||
except Exception as exc:
|
||||
@@ -139,16 +288,37 @@ class Supervisor:
|
||||
proc.wait(timeout=5)
|
||||
except Exception:
|
||||
proc.kill()
|
||||
# Why it died is almost always in its stderr, and until now that
|
||||
# was read only after a *successful* connect -- so a failed power
|
||||
# on reported nothing but "the QMP port never appeared", which is
|
||||
# a symptom. Close the pipe and show what QEMU said.
|
||||
if getattr(proc, "stderr", None) is not None:
|
||||
try:
|
||||
tail = proc.stderr.read(4096).decode("utf-8", "replace")
|
||||
except Exception:
|
||||
tail = ""
|
||||
tail = " ".join(tail.split())
|
||||
if tail:
|
||||
# The end, not the start: a bind failure or an error
|
||||
# exit is the last thing QEMU says.
|
||||
# The *whole* message, not a tail: QEMU reports the real
|
||||
# problem first and then a summary line, so keeping only the
|
||||
# end hid the cause behind "unsupported machine type" -- which
|
||||
# is what a rejected -machine property looks like from below.
|
||||
self._note("qemu said: %s" % tail[:1500])
|
||||
# What we actually ran is the first thing anyone asks, and
|
||||
# until now it was not recorded anywhere.
|
||||
if proc is not None and getattr(proc, "args", None):
|
||||
self._note("command: %s" % " ".join(str(a) for a in proc.args))
|
||||
self._note("argv repr: %r" % (list(proc.args),))
|
||||
self._note(f"power on failed: {exc}")
|
||||
raise
|
||||
proc = self._proc
|
||||
self._note("power on")
|
||||
# Forward QEMU's own stderr, which run.sh and the tests used to discard.
|
||||
# Firmware serial arrives here too, tagged SERIAL by the machine model.
|
||||
if self._log is not None and getattr(proc, "stderr", None) is not None:
|
||||
threading.Thread(
|
||||
target=self._log.pump_stream, args=(proc.stderr,),
|
||||
kwargs={"default_source": "qemu"}, daemon=True).start()
|
||||
if not draining:
|
||||
self._start_stderr_pump(proc)
|
||||
return True
|
||||
|
||||
def power_off(self) -> bool:
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Where the emulator tests find a QEMU and a firmware, and what to say when they cannot.
|
||||
|
||||
The emulator tests used to name one developer's build directory:
|
||||
|
||||
QEMU = os.path.expanduser("~/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm")
|
||||
ELF = os.path.expanduser("~/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf")
|
||||
|
||||
which is fine on the machine they were written on and useless anywhere else -- and a
|
||||
missing file used to be a hard failure, so a checkout without that build could never
|
||||
see a green run and nobody could tell "broken" from "not set up here".
|
||||
|
||||
Resolution order, for both:
|
||||
|
||||
qemu() env QEMU, env UVK5_QEMU, qemu-system-arm on PATH
|
||||
firmware() env ELF, env UVK5_FIRMWARE, assets/firmware/*, work/*
|
||||
|
||||
firmware() prefers assets/firmware, which tools/fetch_firmware.py fills from the upstream
|
||||
project's release archive; that is what makes these tests reproducible rather than
|
||||
anchored to one private build.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import glob
|
||||
import os
|
||||
import pathlib
|
||||
import shutil
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
ROOT = os.path.dirname(HERE)
|
||||
FIRMWARE_DIR = os.path.join(ROOT, "assets", "firmware")
|
||||
|
||||
|
||||
def qemu():
|
||||
"""Path to a qemu-system-arm, or None."""
|
||||
for var in ("QEMU", "UVK5_QEMU"):
|
||||
value = os.environ.get(var)
|
||||
if value and os.path.exists(value):
|
||||
return pathlib.Path(value)
|
||||
found = shutil.which("qemu-system-arm")
|
||||
return pathlib.Path(found) if found else None
|
||||
|
||||
|
||||
def gdb():
|
||||
"""Path to a gdb that speaks ARM, or None.
|
||||
|
||||
A few tests read firmware globals over a gdb attach. That is the one part of the
|
||||
suite a Windows box cannot do at all, and it used to be a hard failure rather than
|
||||
a skip -- so those tests reported a regression where there was only a missing tool.
|
||||
"""
|
||||
for var in ("GDB", "UVK5_GDB"):
|
||||
value = os.environ.get(var)
|
||||
if value and os.path.exists(value):
|
||||
return pathlib.Path(value)
|
||||
# Not plain gdb: on most hosts that is the *host* architecture's gdb, which attaches
|
||||
# to the guest, returns nothing useful, and produces a confusing parse failure
|
||||
# instead of "no debugger". The two names below can actually read an ARM target.
|
||||
for name in ("gdb-multiarch", "arm-none-eabi-gdb"):
|
||||
found = shutil.which(name)
|
||||
if found:
|
||||
return pathlib.Path(found)
|
||||
return None
|
||||
|
||||
|
||||
def firmware():
|
||||
"""Path to a firmware image to run, or None."""
|
||||
for var in ("ELF", "UVK5_FIRMWARE", "UVK5_MULTIBOOT_IMAGE"):
|
||||
value = os.environ.get(var)
|
||||
if value and os.path.exists(value):
|
||||
return pathlib.Path(value)
|
||||
patterns = [os.path.join(FIRMWARE_DIR, "*.bin"),
|
||||
os.path.join(FIRMWARE_DIR, "*.elf"),
|
||||
os.path.join(ROOT, "work", "*.bin"),
|
||||
os.path.join(ROOT, "work", "*.elf")]
|
||||
for pattern in patterns:
|
||||
for hit in sorted(glob.glob(pattern)):
|
||||
return pathlib.Path(hit)
|
||||
return None
|
||||
|
||||
|
||||
def missing(items):
|
||||
"""First thing in @items that is absent, described with how to fix it.
|
||||
|
||||
@items is a list of (path, what, how). Returns None when everything is present.
|
||||
"""
|
||||
for path, what, how in items:
|
||||
if not path or not os.path.exists(path):
|
||||
return "%s is missing (%s); %s" % (what, path or "not found", how)
|
||||
return None
|
||||
|
||||
|
||||
def skip(message):
|
||||
"""Print a skip line and return the exit code a test should use.
|
||||
|
||||
A missing prerequisite is not a failing test. Saying so loudly beats exiting
|
||||
non-zero, which is indistinguishable from a regression and is why a Windows or
|
||||
fresh checkout could never show a green run.
|
||||
"""
|
||||
print("SKIP: %s" % message)
|
||||
return 0
|
||||
+477
-7
@@ -20,15 +20,32 @@ Two things worth knowing:
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import time
|
||||
|
||||
from flask import Flask, Response, jsonify, request
|
||||
|
||||
from uvk5_image import ImageError, detect as detect_image
|
||||
from uvk5_slots import (SLOT_COUNT, erase_slot_file, slots_json,
|
||||
write_slot_file)
|
||||
from uvk5_keys import KEYS, is_valid, normalise
|
||||
from uvk5_lcd import PANEL_PATH
|
||||
from uvk5_logs import LogBuffer
|
||||
from uvk5_stream import FramePump
|
||||
|
||||
KEYPAD_PATH = "/machine/keypad"
|
||||
|
||||
# Uploaded firmware. Kept next to the checkout rather than in the system temp
|
||||
# directory: a firmware is something the user chose to load, and losing it on
|
||||
# reboot would mean uploading it again. UVK5_UPLOAD_DIR overrides it.
|
||||
MAX_UPLOAD_BYTES = 4 * 1024 * 1024
|
||||
UPLOAD_DIR = os.path.join(
|
||||
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
|
||||
"work", "firmware")
|
||||
|
||||
|
||||
def upload_dir() -> str:
|
||||
return os.environ.get("UVK5_UPLOAD_DIR", UPLOAD_DIR)
|
||||
AUDIO_PATH = "/machine/audio"
|
||||
|
||||
# Firmware thresholds, from App/misc.c:
|
||||
@@ -110,8 +127,25 @@ KEY_BINDINGS = {
|
||||
POWER_ACTIONS = ("on", "off", "reset", "pause", "resume")
|
||||
|
||||
|
||||
# The multi-system boot menu lives inside the firmware, not in a separate
|
||||
# bootloader, and the builds that ship without it (some localised releases are
|
||||
# built "without multi-system") look identical until you hold MENU at power-on
|
||||
# and nothing happens. Its banner strings are the quickest tell.
|
||||
MULTIBOOT_MARKERS = (b"F4HWN MULTIBOOT", b"RESTORE REFUSED", b"SLOT NOT VALID")
|
||||
|
||||
|
||||
def image_has_multiboot(path):
|
||||
"""True when @path looks like a build with the boot menu in it."""
|
||||
try:
|
||||
with open(path, "rb") as fh:
|
||||
blob = fh.read()
|
||||
except OSError:
|
||||
return None
|
||||
return any(marker in blob for marker in MULTIBOOT_MARKERS)
|
||||
|
||||
|
||||
def create_app(client, frame_addr: int, status_addr: int, scale: int = 4,
|
||||
supervisor=None, log=None):
|
||||
supervisor=None, log=None, image=None, boot_key=None, flash=None):
|
||||
app = Flask(__name__)
|
||||
|
||||
if log is None:
|
||||
@@ -124,6 +158,7 @@ def create_app(client, frame_addr: int, status_addr: int, scale: int = 4,
|
||||
pump.start()
|
||||
app.config["PUMP"] = pump
|
||||
app.config["SUPERVISOR"] = supervisor
|
||||
app.config["IMAGE"] = image
|
||||
|
||||
def client_ip():
|
||||
"""The address of whoever made this request.
|
||||
@@ -195,6 +230,35 @@ def create_app(client, frame_addr: int, status_addr: int, scale: int = 4,
|
||||
def index():
|
||||
return Response(render_index(scale), mimetype="text/html")
|
||||
|
||||
def panel_state():
|
||||
"""The display controller's own settings: invert, contrast, display on/off.
|
||||
|
||||
These are panel state, not framebuffer content, so they are invisible in the
|
||||
pixels themselves -- a menu entry that changes them otherwise looks like it
|
||||
did nothing. None of them when the emulator is off.
|
||||
"""
|
||||
target = active_client()
|
||||
if target is None:
|
||||
return None
|
||||
try:
|
||||
return {
|
||||
"invert": bool(target.command("qom-get", path=PANEL_PATH,
|
||||
property="invert")),
|
||||
"contrast": int(target.command("qom-get", path=PANEL_PATH,
|
||||
property="contrast")),
|
||||
"display": bool(target.command("qom-get", path=PANEL_PATH,
|
||||
property="display-on")),
|
||||
}
|
||||
except Exception as exc:
|
||||
log.add("qemu", f"panel state unavailable: {exc}")
|
||||
return None
|
||||
|
||||
def firmware_info():
|
||||
"""The loaded image, or None. Its shape and offset come from uvk5_image."""
|
||||
if image is None or image.current is None:
|
||||
return None
|
||||
return image.current.as_dict()
|
||||
|
||||
@app.get("/api/status")
|
||||
def api_status():
|
||||
target = active_client()
|
||||
@@ -205,7 +269,181 @@ def create_app(client, frame_addr: int, status_addr: int, scale: int = 4,
|
||||
except Exception as exc:
|
||||
# The emulator can die under us; that is a state to report, not a 500.
|
||||
return jsonify(powered=False, status="unreachable", error=str(exc))
|
||||
return jsonify(powered=True, speaker=speaker_on(), **info)
|
||||
return jsonify(powered=True, speaker=speaker_on(),
|
||||
panel=panel_state(), firmware=firmware_info(), **info)
|
||||
|
||||
@app.get("/api/firmware")
|
||||
def api_firmware():
|
||||
info = firmware_info()
|
||||
if info is not None and image is not None:
|
||||
info = dict(info, multiboot=image_has_multiboot(image.path))
|
||||
return jsonify(loaded=info is not None, firmware=info)
|
||||
|
||||
@app.post("/api/firmware")
|
||||
def api_firmware_upload():
|
||||
"""Boot an uploaded firmware image.
|
||||
|
||||
The request body is the image itself. Its shape is read out of the vector
|
||||
table (see uvk5_image) rather than taken on trust, because loading an image
|
||||
at the wrong offset fails silently: it runs 0x2800 bytes off and the first
|
||||
fetch reads whatever data is there.
|
||||
"""
|
||||
if image is None:
|
||||
return jsonify(error="this server was started without firmware control"), 409
|
||||
name = (request.args.get("name")
|
||||
or request.headers.get("X-Filename") or "firmware.bin")
|
||||
name = os.path.basename(name.replace("\\", "/")) or "firmware.bin"
|
||||
data = request.get_data(cache=False, as_text=False)
|
||||
if not data:
|
||||
return jsonify(error="no image in the request body"), 400
|
||||
if len(data) > MAX_UPLOAD_BYTES:
|
||||
return jsonify(error="%d bytes is too large" % len(data)), 413
|
||||
directory = upload_dir()
|
||||
os.makedirs(directory, exist_ok=True)
|
||||
path = os.path.join(directory, name)
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(data)
|
||||
# Validate before touching the running emulator: a file that is not an image
|
||||
# should leave the radio exactly as it was, not powered off with a 400.
|
||||
try:
|
||||
info = detect_image(path)
|
||||
except ImageError as exc:
|
||||
return jsonify(error=str(exc), saved=path), 400
|
||||
|
||||
if supervisor is not None and supervisor.is_running():
|
||||
# The image is picked when the process is spawned, so it has to come
|
||||
# back up to take effect. Off first, then adopt, so nothing boots the
|
||||
# previous image in between.
|
||||
supervisor.power_off()
|
||||
restart = True
|
||||
else:
|
||||
restart = False
|
||||
image.set(info)
|
||||
log.add("firmware", "%s (%s, %d bytes)" % (name, info.kind, info.size),
|
||||
ip=client_ip())
|
||||
if restart:
|
||||
try:
|
||||
supervisor.power_on()
|
||||
except Exception as exc:
|
||||
log.add("firmware", "restart failed: %s" % exc)
|
||||
return jsonify(firmware=info.as_dict(), restarted=False,
|
||||
error=str(exc)), 500
|
||||
body = dict(info.as_dict(), multiboot=image_has_multiboot(path))
|
||||
return jsonify(firmware=body, restarted=restart)
|
||||
|
||||
# ------------------------------------------------------------- firmware slots
|
||||
#
|
||||
# The multi-system firmware keeps four firmware slots plus a backup of the
|
||||
# internal image in the external flash, in the layout App/driver/mb_flash.h
|
||||
# defines (an FMB1 header plus a CRC-32, image one sector into the slot).
|
||||
# Editing them means editing the flash image the emulator boots from, and that
|
||||
# image is chosen when the process is spawned -- so an edit powers the emulator
|
||||
# off, rewrites the image and powers it on again.
|
||||
#
|
||||
# It rewrites a working copy, never the file the server was pointed at: that may
|
||||
# be the radio's real calibration dump.
|
||||
def _edit_flash(edit):
|
||||
# Power off, apply edit(path) to a working copy, power on again.
|
||||
if flash is None:
|
||||
raise RuntimeError("this server was started without flash control")
|
||||
work = os.path.join(upload_dir(), "flash-current.img")
|
||||
running = supervisor is not None and supervisor.is_running()
|
||||
if running:
|
||||
# Takes the emulator's own write-back with it, so an edit builds on what
|
||||
# the firmware actually has rather than on the launch-time file.
|
||||
supervisor.power_off()
|
||||
# Wait for the process to actually go, not just for the request to return:
|
||||
# it holds the image open while it exits, and starting the next instance
|
||||
# against a file another process still has open fails on Windows -- which
|
||||
# showed up as a slot write that left the emulator off.
|
||||
for _ in range(40):
|
||||
if not supervisor.is_running():
|
||||
break
|
||||
time.sleep(0.25)
|
||||
if os.path.abspath(work) != os.path.abspath(flash.path):
|
||||
shutil.copyfile(flash.path, work)
|
||||
flash.path = work
|
||||
result = edit(work)
|
||||
if running:
|
||||
supervisor.power_on()
|
||||
return result
|
||||
|
||||
@app.get("/api/slots")
|
||||
def api_slots():
|
||||
"""The firmware slots in the flash image the emulator is using."""
|
||||
if flash is None:
|
||||
return jsonify(error="this server was started without flash control"), 409
|
||||
try:
|
||||
return jsonify(slots_json(flash.path))
|
||||
except Exception as exc:
|
||||
return jsonify(error=str(exc)), 500
|
||||
|
||||
@app.post("/api/slots/<int:slot>")
|
||||
def api_slot_write(slot):
|
||||
"""Write an uploaded image into a slot (0 is the backup of Main)."""
|
||||
if not 0 <= slot < SLOT_COUNT:
|
||||
return jsonify(error="slot %d is out of range (0..%d)"
|
||||
% (slot, SLOT_COUNT - 1)), 400
|
||||
data = request.get_data(cache=False, as_text=False)
|
||||
if not data:
|
||||
return jsonify(error="no image in the request body"), 400
|
||||
if len(data) > MAX_UPLOAD_BYTES:
|
||||
return jsonify(error="%d bytes is too large" % len(data)), 413
|
||||
name = os.path.basename((request.args.get("name")
|
||||
or request.headers.get("X-Filename")
|
||||
or "firmware.bin").replace("\\", "/"))
|
||||
version = request.args.get("version", "")
|
||||
try:
|
||||
row = _edit_flash(lambda p: write_slot_file(p, slot, data, name, version))
|
||||
except Exception as exc:
|
||||
log.add("slots", "slot %d write failed: %s" % (slot, exc), ip=client_ip())
|
||||
return jsonify(error=str(exc)), 400
|
||||
log.add("slots", "slot %d <- %s (%d bytes, crc %s)"
|
||||
% (slot, name, len(data), "ok" if row.get("crc_ok") else "MISMATCH"),
|
||||
ip=client_ip())
|
||||
return jsonify(slot=row)
|
||||
|
||||
@app.post("/api/slots/<int:slot>/erase")
|
||||
def api_slot_erase(slot):
|
||||
"""Erase a slot, as the firmware does for its own 0x0722 command."""
|
||||
if not 0 <= slot < SLOT_COUNT:
|
||||
return jsonify(error="slot %d is out of range (0..%d)"
|
||||
% (slot, SLOT_COUNT - 1)), 400
|
||||
try:
|
||||
row = _edit_flash(lambda p: erase_slot_file(p, slot))
|
||||
except Exception as exc:
|
||||
log.add("slots", "slot %d erase failed: %s" % (slot, exc), ip=client_ip())
|
||||
return jsonify(error=str(exc)), 400
|
||||
log.add("slots", "slot %d erased" % slot, ip=client_ip())
|
||||
return jsonify(slot=row)
|
||||
|
||||
@app.post("/api/flash")
|
||||
def api_flash_upload():
|
||||
"""Use an uploaded image as the external flash, slots and all."""
|
||||
if flash is None:
|
||||
return jsonify(error="this server was started without flash control"), 409
|
||||
name = os.path.basename((request.args.get("name")
|
||||
or request.headers.get("X-Filename")
|
||||
or "flash.img").replace("\\", "/"))
|
||||
data = request.get_data(cache=False, as_text=False)
|
||||
if not data:
|
||||
return jsonify(error="no flash image in the request body"), 400
|
||||
if len(data) > MAX_UPLOAD_BYTES:
|
||||
return jsonify(error="%d bytes is too large" % len(data)), 413
|
||||
running = supervisor is not None and supervisor.is_running()
|
||||
if running:
|
||||
supervisor.power_off()
|
||||
directory = upload_dir()
|
||||
os.makedirs(directory, exist_ok=True)
|
||||
path = os.path.join(directory, name)
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(data)
|
||||
flash.path = path
|
||||
log.add("slots", "flash image <- %s (%d bytes)" % (name, len(data)),
|
||||
ip=client_ip())
|
||||
if running:
|
||||
supervisor.power_on()
|
||||
return jsonify(slots_json(path))
|
||||
|
||||
@app.get("/api/logs")
|
||||
def api_logs():
|
||||
@@ -232,6 +470,17 @@ def create_app(client, frame_addr: int, status_addr: int, scale: int = 4,
|
||||
|
||||
# Attribute the action here: the supervisor has no request context, and on
|
||||
# a shared log "who powered it off" is the useful part.
|
||||
body = request.get_json(silent=True) or {}
|
||||
if boot_key is not None:
|
||||
if action == "on" and body.get("boot_key"):
|
||||
boot_key.name = str(body["boot_key"])[:16]
|
||||
boot_key.hold_ms = int(body.get("hold_ms") or 1500)
|
||||
log.add("power", "asking for %s held from reset" % boot_key.name,
|
||||
ip=client_ip())
|
||||
else:
|
||||
# A plain On must not inherit the last boot mode.
|
||||
boot_key.clear()
|
||||
|
||||
log.add("power", f"{action} requested", ip=client_ip())
|
||||
|
||||
try:
|
||||
@@ -428,6 +677,11 @@ def render_index(scale: int) -> str:
|
||||
/* The border lives on .screenwrap so it stays put when the frame is hidden. */
|
||||
#screen {{ display:block; image-rendering:pixelated; background:#c8d6b9; }}
|
||||
.body {{ display:flex; gap:14px; align-items:flex-start; }}
|
||||
.fwbar {{ display:flex; gap:8px; align-items:center; flex-wrap:wrap;
|
||||
font-size:12px; color:#8b949e; max-width:420px; }}
|
||||
.fwbar input {{ color:#c9d1d9; font-size:12px; max-width:190px; }}
|
||||
.fwbar .hint {{ opacity:.75; }}
|
||||
#fwstate {{ color:#c9d1d9; }}
|
||||
.sides {{ display:flex; flex-direction:column; gap:8px; }}
|
||||
.pad {{ display:flex; flex-direction:column; gap:8px; }}
|
||||
.row {{ display:flex; gap:8px; }}
|
||||
@@ -458,6 +712,12 @@ def render_index(scale: int) -> str:
|
||||
*/
|
||||
#speaker {{ font-size:14px; opacity:0.25; transition:opacity 0.15s; }}
|
||||
#speaker.on {{ opacity:1; }}
|
||||
/*
|
||||
* The display controller's own settings. They never appear in the pixels -- that
|
||||
* is the whole reason they need showing: contrast and inversion live in the
|
||||
* panel, so a menu entry that changes them otherwise looks like it did nothing.
|
||||
*/
|
||||
#panel {{ font-size:11px; color:#8b949e; margin-left:8px; letter-spacing:0.3px; }}
|
||||
/*
|
||||
* Powered off is a dark panel, drawn by the wrapper so the frame itself can be
|
||||
* hidden. An earlier attempt put a dark background on the <img> alone, which
|
||||
@@ -475,6 +735,18 @@ def render_index(scale: int) -> str:
|
||||
* of view and you scroll back to read them. min-height matches height so a
|
||||
* nearly empty pane does not jump around as the first lines arrive.
|
||||
*/
|
||||
#slottable {{ width:100%; border-collapse:collapse; margin:6px 0 0;
|
||||
font:12px/1.5 ui-monospace,SFMono-Regular,Menlo,monospace;
|
||||
color:#8b949e; }}
|
||||
#slottable td {{ padding:3px 6px; border-top:1px solid #2d333b;
|
||||
white-space:nowrap; }}
|
||||
#slottable td:first-child {{ color:#c9d1d9; width:12em; }}
|
||||
#slottable input[type=file] {{ color:#8b949e; font:inherit; max-width:14em; }}
|
||||
#slottable button {{ font:inherit; color:#c9d1d9; background:#2b3138;
|
||||
border:1px solid #2d333b; border-radius:4px; padding:1px 8px; }}
|
||||
#slottable button:hover {{ background:#343b44; }}
|
||||
.mini {{ margin-left:1em; }}
|
||||
.mini input {{ margin-left:0.5em; }}
|
||||
#logtext {{ height:180px; min-height:180px; overflow-y:auto; margin:6px 0 0;
|
||||
padding:8px; background:#0d1117; border:1px solid #2d333b;
|
||||
border-radius:6px; white-space:pre-wrap; word-break:break-all;
|
||||
@@ -485,11 +757,28 @@ def render_index(scale: int) -> str:
|
||||
<div class="radio">
|
||||
<div class="powerbar">
|
||||
<button class="pwr" data-power="on">On</button>
|
||||
<button class="pwr" data-power="on" data-boot="MENU"
|
||||
title="restart and hold MENU from reset, for the boot menu (Shift+M)">Multiboot</button>
|
||||
<button class="pwr" data-power="off">Off</button>
|
||||
<button class="pwr" data-power="reset">Reset</button>
|
||||
<span id="powerstate">-</span>
|
||||
<span id="speaker" title="the firmware has enabled the audio amplifier">🔈</span>
|
||||
<span id="panel" title="display controller: contrast, inversion, panel on/off"></span>
|
||||
</div>
|
||||
<div class="fwbar">
|
||||
<label for="fwfile">Firmware</label>
|
||||
<input type="file" id="fwfile" accept=".bin,.elf">
|
||||
<span id="fwstate">-</span>
|
||||
<span class="hint">or drop a .bin anywhere on the page</span>
|
||||
</div>
|
||||
<div class="fwbar">
|
||||
<label>Firmware slots</label>
|
||||
<span id="flashstate">-</span>
|
||||
<label class="mini">flash image<input type="file" id="flashfile" accept=".img,.bin"></label>
|
||||
<span class="hint">the multi-system slots live in the external flash;
|
||||
write a .bin into one, then press Multiboot</span>
|
||||
</div>
|
||||
<table id="slottable"><tbody></tbody></table>
|
||||
<div class="screenwrap" id="screenwrap">
|
||||
<img id="screen" src="/stream" alt="radio LCD"
|
||||
width="{128 * scale}" height="{64 * scale}">
|
||||
@@ -621,7 +910,13 @@ document.querySelectorAll('.pwr').forEach(btn => {{
|
||||
}}
|
||||
document.querySelectorAll('.pwr').forEach(b => b.disabled = true);
|
||||
try {{
|
||||
const r = await fetch('/api/power/' + action, {{method: 'POST'}});
|
||||
// A boot mode needs the key held *from reset*, which only the server can do:
|
||||
// the firmware samples the keypad in the first milliseconds after reset.
|
||||
const body = btn.dataset.boot ? {{ boot_key: btn.dataset.boot }} : {{}};
|
||||
const r = await fetch('/api/power/' + action, {{
|
||||
method: 'POST',
|
||||
headers: {{ 'Content-Type': 'application/json' }},
|
||||
body: JSON.stringify(body) }});
|
||||
if (!r.ok) {{
|
||||
const j = await r.json().catch(() => ({{}}));
|
||||
document.getElementById('status').textContent =
|
||||
@@ -635,6 +930,31 @@ document.querySelectorAll('.pwr').forEach(btn => {{
|
||||
// Restart the stream: the old one ends when the emulator goes away.
|
||||
const img = document.getElementById('screen');
|
||||
img.src = '/stream?t=' + Date.now();
|
||||
// The screen is a long-lived multipart stream. If the server is restarted underneath
|
||||
// it -- which happens whenever a firmware or slot write restarts the emulator -- the
|
||||
// <img> keeps showing the last frame it received, and then nothing on the page appears
|
||||
// to work, because the picture never changes. Reconnect, and fall back to fetching
|
||||
// single frames if the stream will not come back.
|
||||
let streamRetries = 0;
|
||||
const screenEl = document.getElementById('screen');
|
||||
function connectStream() {{
|
||||
screenEl.src = '/stream?t=' + Date.now();
|
||||
}}
|
||||
screenEl.addEventListener('error', () => {{
|
||||
streamRetries += 1;
|
||||
if (streamRetries <= 2) {{
|
||||
setTimeout(connectStream, 1000);
|
||||
}} else {{
|
||||
// Single frames: one plain request each, which always recovers.
|
||||
setInterval(() => {{ screenEl.src = '/frame.png?t=' + Date.now(); }}, 250);
|
||||
}}
|
||||
}});
|
||||
setInterval(() => {{
|
||||
// A stream that is connected but silent (a stopped guest) still counts as up.
|
||||
// Restarting after every power action is already handled above; this only covers
|
||||
// the server having been replaced, which shows up as a request that never lands.
|
||||
if (screenEl.complete && screenEl.naturalWidth === 0) connectStream();
|
||||
}}, 5000);
|
||||
}}
|
||||
}});
|
||||
}});
|
||||
@@ -643,6 +963,15 @@ function showSpeaker(on) {{
|
||||
document.getElementById('speaker').classList.toggle('on', !!on);
|
||||
}}
|
||||
|
||||
function showPanel(panel) {{
|
||||
const el = document.getElementById('panel');
|
||||
if (!panel) {{ el.textContent = ''; return; }}
|
||||
const bits = ['CTR ' + panel.contrast];
|
||||
if (panel.invert) bits.push('INV');
|
||||
if (!panel.display) bits.push('PANEL OFF');
|
||||
el.textContent = bits.join(' · ');
|
||||
}}
|
||||
|
||||
function showPower(powered) {{
|
||||
const label = document.getElementById('powerstate');
|
||||
label.textContent = powered ? 'on' : 'off';
|
||||
@@ -657,12 +986,14 @@ async function poll() {{
|
||||
const s = await r.json();
|
||||
showPower(!!s.powered);
|
||||
showSpeaker(s.speaker);
|
||||
showPanel(s.panel);
|
||||
document.getElementById('status').textContent =
|
||||
s.powered ? ('guest: ' + (s.status || 'unknown'))
|
||||
: 'powered off -- press On to boot';
|
||||
}} catch (err) {{
|
||||
showPower(false);
|
||||
showSpeaker(false);
|
||||
showPanel(null);
|
||||
document.getElementById('status').textContent = 'server unreachable';
|
||||
}}
|
||||
}}
|
||||
@@ -710,6 +1041,125 @@ async function pollLogs() {{
|
||||
}}
|
||||
pollLogs();
|
||||
setInterval(pollLogs, 2000);
|
||||
|
||||
// Firmware slots. These are the multi-system firmware's own slots in the
|
||||
// external flash (App/driver/mb_flash.h): slot 0 is the backup of the running
|
||||
// image, 1..4 are the switchable ones. Writing one rewrites the flash image and
|
||||
// restarts the emulator, because the image is chosen when it is spawned.
|
||||
async function loadSlots() {{
|
||||
const tb = document.querySelector('#slottable tbody');
|
||||
const state = document.getElementById('flashstate');
|
||||
try {{
|
||||
const j = await (await fetch('/api/slots')).json();
|
||||
if (j.error) {{ state.textContent = j.error; return; }}
|
||||
const base = j.name || j.image || 'flash image';
|
||||
state.textContent = base + ' (' + Math.round(j.size / 1024) + ' KiB)';
|
||||
tb.innerHTML = '';
|
||||
for (const s of j.slots) {{
|
||||
const tr = document.createElement('tr');
|
||||
const label = s.empty ? '<i>empty</i>'
|
||||
: (s.name || '?') + ' ' + (s.fw_version || '');
|
||||
const size = s.empty ? ''
|
||||
: s.image_size + ' B' + (s.crc_ok ? '' : ' CRC MISMATCH');
|
||||
tr.innerHTML = '<td>slot ' + s.slot +
|
||||
(s.slot === 0 ? ' (Main)' : '') + '</td><td>' + label +
|
||||
'</td><td>' + size + '</td><td></td>';
|
||||
const td = tr.lastElementChild;
|
||||
const inp = document.createElement('input');
|
||||
inp.type = 'file';
|
||||
inp.accept = '.bin';
|
||||
inp.addEventListener('change', async () => {{
|
||||
const f = inp.files[0];
|
||||
if (!f) return;
|
||||
tr.querySelectorAll('input,button').forEach(b => b.disabled = true);
|
||||
const r = await fetch('/api/slots/' + s.slot + '?name=' +
|
||||
encodeURIComponent(f.name),
|
||||
{{ method: 'POST', body: f }});
|
||||
const jj = await r.json();
|
||||
if (jj.error) alert('slot ' + s.slot + ': ' + jj.error);
|
||||
await loadSlots(); poll();
|
||||
}});
|
||||
const er = document.createElement('button');
|
||||
er.textContent = 'Erase';
|
||||
er.addEventListener('click', async () => {{
|
||||
if (!confirm('Erase slot ' + s.slot + '?')) return;
|
||||
tr.querySelectorAll('input,button').forEach(b => b.disabled = true);
|
||||
const r = await fetch('/api/slots/' + s.slot + '/erase',
|
||||
{{ method: 'POST' }});
|
||||
const jj = await r.json();
|
||||
if (jj.error) alert('slot ' + s.slot + ': ' + jj.error);
|
||||
await loadSlots(); poll();
|
||||
}});
|
||||
td.append(inp, er);
|
||||
tb.appendChild(tr);
|
||||
}}
|
||||
}} catch (err) {{ state.textContent = 'slots unavailable: ' + err; }}
|
||||
}}
|
||||
const flashInput = document.getElementById('flashfile');
|
||||
if (flashInput) flashInput.addEventListener('change', async () => {{
|
||||
const f = flashInput.files[0];
|
||||
if (!f) return;
|
||||
if (!confirm('Use ' + f.name + ' as the external flash image? Slots, settings',
|
||||
' and calibration come from it.')) return;
|
||||
const r = await fetch('/api/flash?name=' + encodeURIComponent(f.name),
|
||||
{{ method: 'POST', body: f }});
|
||||
const j = await r.json();
|
||||
if (j.error) alert(j.error);
|
||||
loadSlots(); poll();
|
||||
}});
|
||||
loadSlots();
|
||||
setInterval(loadSlots, 15000);
|
||||
// Firmware upload. The file *is* the request body, so the server reads the
|
||||
// vector table itself and decides the load offset: an application image and a
|
||||
// full-flash image need different ones, and the wrong one fails silently.
|
||||
const fwstate = document.getElementById('fwstate');
|
||||
const fwfile = document.getElementById('fwfile');
|
||||
async function loadFirmware(file) {{
|
||||
fwstate.textContent = 'uploading ' + file.name + ' ...';
|
||||
try {{
|
||||
const res = await fetch('/api/firmware?name=' + encodeURIComponent(file.name), {{
|
||||
method: 'POST',
|
||||
headers: {{ 'Content-Type': 'application/octet-stream' }},
|
||||
body: file }});
|
||||
const info = await res.json();
|
||||
if (!res.ok) {{ fwstate.textContent = info.error || 'upload failed'; return; }}
|
||||
fwstate.textContent = fwLabel(info.firmware) +
|
||||
(info.restarted ? ' - rebooting' : ' - press On');
|
||||
poll();
|
||||
}} catch (err) {{
|
||||
fwstate.textContent = 'upload failed: ' + err;
|
||||
}}
|
||||
}}
|
||||
if (fwfile) fwfile.addEventListener('change', () => {{
|
||||
if (fwfile.files && fwfile.files[0]) loadFirmware(fwfile.files[0]);
|
||||
}});
|
||||
document.addEventListener('dragover', (e) => e.preventDefault());
|
||||
document.addEventListener('drop', (e) => {{
|
||||
e.preventDefault();
|
||||
const f = e.dataTransfer && e.dataTransfer.files && e.dataTransfer.files[0];
|
||||
if (f) loadFirmware(f);
|
||||
}});
|
||||
// Say when a build has no multi-system boot menu: the Multiboot button then does
|
||||
// nothing at all, which reads as the emulator being broken rather than the
|
||||
// firmware not containing that menu.
|
||||
function fwLabel(fw) {{
|
||||
let s = fw.name + ' (' + fw.kind + ')';
|
||||
if (fw.multiboot === false) s += ' - no multi-system menu';
|
||||
return s;
|
||||
}}
|
||||
async function pollFirmware() {{
|
||||
try {{
|
||||
const r = await fetch('/api/firmware');
|
||||
const info = await r.json();
|
||||
if (info.firmware) {{
|
||||
fwstate.textContent = fwLabel(info.firmware);
|
||||
}} else {{
|
||||
fwstate.textContent = 'none loaded - drop a .bin here';
|
||||
}}
|
||||
}} catch (err) {{ /* the rest of the page works without this */ }}
|
||||
}}
|
||||
pollFirmware();
|
||||
|
||||
</script>
|
||||
</body></html>"""
|
||||
|
||||
@@ -730,7 +1180,9 @@ def main() -> int:
|
||||
"this server did not start that process.")
|
||||
ap.add_argument("--qemu", default=os.path.expanduser(
|
||||
"~/qemu-build/qemu-7.2+dfsg/build/qemu-system-arm"))
|
||||
ap.add_argument("--elf", default=os.path.expanduser(
|
||||
# Named --elf for history; any .bin or .elf works, and its shape is read out
|
||||
# of the file rather than assumed. More can be uploaded from the page.
|
||||
ap.add_argument("--elf", "--firmware", dest="elf", default=os.path.expanduser(
|
||||
"~/uvk5-port/uvk5-sat/build/CW/nr7y.cw.elf"))
|
||||
ap.add_argument("--flash", default=os.path.join(
|
||||
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
|
||||
@@ -739,6 +1191,8 @@ def main() -> int:
|
||||
args = ap.parse_args()
|
||||
|
||||
from uvk5_qmp import QmpClient
|
||||
from uvk5_image import ImageSlot
|
||||
from uvk5_supervisor import BootKey, FlashSlot
|
||||
from uvk5_supervisor import Supervisor, default_launcher, wait_for_socket
|
||||
|
||||
def connect():
|
||||
@@ -749,9 +1203,24 @@ def main() -> int:
|
||||
# One buffer shared by the supervisor and the HTTP layer, so power events,
|
||||
# QEMU stderr and firmware serial all land in the same place.
|
||||
log = LogBuffer()
|
||||
|
||||
# The image the next launch boots. A slot rather than a path so an upload
|
||||
# from the page takes effect at the next power-on without a restart, and so
|
||||
# the load offset is decided by the image itself (see uvk5_image).
|
||||
image = ImageSlot()
|
||||
boot_key = BootKey()
|
||||
flash = FlashSlot(args.flash)
|
||||
try:
|
||||
image.set(os.path.expanduser(args.elf))
|
||||
except ImageError as exc:
|
||||
# Not fatal: the page can load one, and saying so beats refusing to
|
||||
# start because a default path from another machine is missing.
|
||||
log.add("firmware", "no firmware loaded yet: %s" % exc)
|
||||
print("no firmware loaded yet: %s" % exc)
|
||||
|
||||
supervisor = Supervisor(
|
||||
launch=default_launcher(args.qemu, args.flash, args.elf, args.qmp,
|
||||
gdb_port=args.gdb_port),
|
||||
launch=default_launcher(args.qemu, flash, image, boot_key,
|
||||
qmp_path=args.qmp, gdb_port=args.gdb_port),
|
||||
connect=connect, log=log)
|
||||
|
||||
if args.attach:
|
||||
@@ -762,7 +1231,8 @@ def main() -> int:
|
||||
# page behaves like walking up to a machine rather than finding it booted.
|
||||
|
||||
app = create_app(supervisor.client(), args.frame_addr, args.status_addr,
|
||||
args.scale, supervisor=supervisor, log=log)
|
||||
args.scale, supervisor=supervisor, log=log, image=image,
|
||||
boot_key=boot_key, flash=flash)
|
||||
print(f"serving on http://{args.host}:{args.port}/")
|
||||
print("attached to a running emulator" if args.attach
|
||||
else "emulator is OFF; press On in the browser to boot it")
|
||||
|
||||
+190
@@ -0,0 +1,190 @@
|
||||
# 让 f4hwn 5.9.0.CN 在本机（Windows）跑起来
|
||||
|
||||
本机实测记录。结论先说：**这个项目并不是"只能跑在 Linux"，只有外壳脚本和 unix socket 是
|
||||
Linux 的；机器模型和工具本身跨平台。** 我在本机用 MSYS2 原生编了 QEMU 7.2，把
|
||||
`qemu/py32f071.c` 编进去，用户的固件已经跑起来、能按键、能出画面。
|
||||
|
||||
## 固件是什么
|
||||
|
||||
`04-20260827_白头佬汉化版_f4hwn.5.9.0.bin`，114,324 字节。
|
||||
|
||||
* 身份串：`UV-K5 Firmware, EGZUMER+F4HWN v5.9.0.CN`
|
||||
* 向量表：SP=0x20004000，Reset=0x08002d49 —— 与仓库参考固件逐字节相同，说明它是
|
||||
**PY32F071 的应用镜像**，链接基址 `0x08002800`（`F:\pi` 里的逆向报告独立确认了同一基址）
|
||||
* 它是应用镜像、不含 0..0x27FF 的出厂引导程序，所以**可以直接当 `-kernel` 用**
|
||||
|
||||
## 为什么要包一层 ELF（重要）
|
||||
|
||||
仓库 `py32f071.c` 的注释说 "`.elf/.bin` 都能启动"，这句话对 `.bin` **不成立**：
|
||||
|
||||
* QEMU 7.2 的 `armv7m_load_kernel(cpu, file, mem_base=0x2800, size)` 对 ELF 用程序头里的地址，
|
||||
对裸 `.bin` 则按 `mem_base` 加载 —— 也就是**容器地址 0x2800**；
|
||||
* 而容器地址 0..0x1D800 是 flash 的别名区（映射到 flash 偏移 0x2800 起），所以裸 bin 会被写到
|
||||
flash 偏移 0x5000，**整整偏移 0x2800**，CPU 从地址 0 取向量表时读到的是空白，直接跑飞。
|
||||
|
||||
`tools/bin2elf.py` 就是补这个：把 `.bin` 包成一个 `PT_LOAD` 在 `0x08002800`、
|
||||
入口取镜像自身复位向量的 ELF32/ARM。
|
||||
|
||||
## 中文字体在 SPI flash 里，不在固件里
|
||||
|
||||
发行包里 `01/02/03` 三个文件是**首次刷机**才要的：`02` 是 16x16、`03` 是 8x8 的 GB2312 点阵
|
||||
字体包（UF2 容器）。它们烧到 SPI NOR 的固定位置：
|
||||
|
||||
UF2 头里的目标地址 内容 大小
|
||||
0x000A0000 16x16 字模 261,888 B（8184 字模 × 32 B）
|
||||
0x000E0000 8x8 字模 65,536 B（8192 字模 × 8 B）
|
||||
|
||||
所以 `make_flash.py` 现在支持 `--blob`，`.uf2` 按自身块地址落盘：
|
||||
|
||||
python tools/make_flash.py \
|
||||
--blob 0:work/f4hwn/02-大字体16x16.uf2 \
|
||||
--blob 0:work/f4hwn/03-小字体8x8.uf2
|
||||
|
||||
不写进去会怎样：字体区是空的（0xFF），汉字全变实心块。
|
||||
|
||||
## 关键地址（已用固件源码交叉验证）
|
||||
|
||||
| 符号 | 地址 | 大小 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| gFrameBuffer | `0x200012BE` | 896 = 7 行 × 128 | 显示区，面板第 1..7 页 |
|
||||
| gStatusLine | `0x2000163E` | 128 | 状态行，面板第 0 页 |
|
||||
|
||||
两个地址**都不是 128 字节对齐**，相差正好 896 字节（帧缓冲在前、状态行在后，与声明顺序相反；
|
||||
参考固件的 `0x200013DC / 0x2000175C` 也是这个关系）。
|
||||
|
||||
这两个值是**算出来的，不是看出来的**，方法可复用到任何新固件：
|
||||
|
||||
1. 抓 16 KB SRAM：`python work/qmp.py dump 0x20000000 0x4000 work/s4.bin`。
|
||||
2. 从源码里挑"内容已知、落点已知"的位图（`App/driver/st7565.c`、`App/ui/status.c`、
|
||||
`App/bitmaps.c`）：
|
||||
* `gFontPowerSave` 在状态行 +0、`gFontDWR` +18、`gFontPttClassic` +54、
|
||||
`BITMAP_BatteryLevel1` +111（= `LCD_WIDTH - 17`）；
|
||||
* `BITMAP_VFO_Default`（VFO 箭头）由 `memcpy(p_line0 + 0, ...)` 画在**帧行偏移 0**。
|
||||
3. 在 SRAM 里搜这些字节串：状态行四个位图的间距要**同时**成立，只有一个基址满足；
|
||||
箭头的落点直接给出帧缓冲基址。两者独立得出 `0x2000163E` 与 `0x200012BE`（相差 896，吻合）。
|
||||
|
||||
之前我把基址取成 128 对齐的 `0x20001280 / 0x20001600`，偏了 **0x3E = 62 字节**。后果不是整体平移，
|
||||
而是**每行 62 字节环绕折行**：渲染出的每一行 = 上一真实行的尾部 62 列 + 本行的头部 66 列，
|
||||
字会从第 66 列被劈开，状态行还混进了帧行尾部——看起来就是"没对准"。
|
||||
|
||||
快速自检（对齐正确时应成立）：帧行 3 是源码 `memset(gFrameBuffer[3], 0, 128)` 清掉的中缝，
|
||||
应整行全 0；上/下 VFO 的帧行 0≡4、1≡5；帧行 2 与 6 只该在活动 VFO 的说明文字上不同。
|
||||
|
||||
面板侧两个细节（不影响取帧，但解释列号为什么 +4）：驱动写 `Line + 176`、`Column + 4`，
|
||||
`cmds[]` 用 `0xA1`（SEG 反向）+ `0xC0`（COM 正常）。
|
||||
|
||||
## 怎么跑
|
||||
|
||||
powershell -File work\run-emulator.ps1 # 起 QEMU（QMP tcp:4444，GDB tcp:1234）
|
||||
powershell -File work\run-webui.ps1 # 起网页遥控，然后开 http://127.0.0.1:8080/
|
||||
powershell -File work\restore-flash.ps1 # 还原 flash.img（先停模拟器）
|
||||
|
||||
命令行按键（走 TCP）：
|
||||
|
||||
python tools/key.py --socket 127.0.0.1:4444 MENU DOWN DOWN
|
||||
|
||||
## 本机上的构件
|
||||
|
||||
| 路径 | 说明 |
|
||||
| --- | --- |
|
||||
| `F:\dsh-build\qemu-7.2.0` | QEMU 7.2.0 源码 + 打入的模型、patched SysTick、uv-k5-v3 注册 |
|
||||
| `F:\dsh-build\qemu-7.2.0\build\qemu-system-arm.exe` | 编译产物；运行需要 `F:\msys64\mingw64\bin` 在 PATH |
|
||||
| `F:\msys64` | MSYS2（gcc 16.2 / glib 2.90 / pixman / meson / ninja / gdb / perl） |
|
||||
| `work/f4hwn/*.elf` | 由 .bin 包的 ELF |
|
||||
| `assets/flash.img` | 校准 + 字体；`work/flash-base.img` 是干净副本 |
|
||||
|
||||
## 为了在 Windows 上跑起来，对仓库做了什么
|
||||
|
||||
1. `qemu/py32f071.c`：补 `#include "qapi/visitor.h"`。原文件用 `visit_type_uint64` 却没包含声明它的
|
||||
头，对**原版 QEMU 7.2 编译不过**（作者的环境里应该是被别的头间接带入的）。
|
||||
2. `tools/bin2elf.py`：新增（见上）。
|
||||
3. `tools/make_flash.py`：新增 `--blob ADDR:FILE`，`.uf2` 按块地址解析。
|
||||
4. `tools/uvk5_qmp.py`、`tools/key.py`：QMP 端点除 unix 路径外接受 `host:port`。
|
||||
5. `tools/uvk5_supervisor.py`：`wait_for_socket` 支持 TCP 端点。
|
||||
6. `tools/uvk5_lcd.py`、`tools/uvk5_stream.py`：帧暂存目录不再硬编码 `/dev/shm`（Windows 没有），
|
||||
退回系统临时目录。
|
||||
|
||||
Linux 上的行为都没变：地址默认值仍是 unix 路径，`/dev/shm` 存在时仍用 `/dev/shm`。
|
||||
|
||||
## 面板级设置：对比度与反显
|
||||
|
||||
菜单里的 **SetCtr（对比度）** 和 **SetInv（反显）** 属于面板，不属于帧缓冲：
|
||||
|
||||
```c
|
||||
gSetting_set_ctr = ...; // App/app/menu.c
|
||||
ST7565_ContrastAndInv(); // → 0xE2、0x81 + (21 + set_ctr)、0xA6|set_inv
|
||||
```
|
||||
|
||||
它们只往 ST7565 发命令，`gFrameBuffer` 一个字节都不改。网页渲染的是帧缓冲，所以"改了没反应"是必然的
|
||||
——除非把控制器本身也建模。现在 SPI1 后面挂了最小的 ST7565 模型（A0 接 PA6、CS 接 PB2，与
|
||||
`App/driver/st7565.c` 一致），解析 `0xA6/0xA7`（反显）、`0xAE/0xAF`（开屏）、`0x81 <值>`（对比度），
|
||||
并暴露三个**只读**属性：
|
||||
|
||||
qom-get /machine/panel invert 反显位（0xA7 之后为真）
|
||||
qom-get /machine/panel contrast 0x81 后面那个值
|
||||
qom-get /machine/panel display-on 0xAF / 0xAE
|
||||
|
||||
* **反显会真的作用到画面**：`tools/uvk5_lcd.py` 渲染时按该位取反，所以 60 号设置现在看得见。
|
||||
* **对比度只报告不渲染**：那是模拟量（玻璃多黑），渲染不出来。本机实测值 `36 = 21 + 15`，
|
||||
正好等于当时存着的对比度设置，说明命令一直都在发。
|
||||
* **display-on 只报告不动作**：软复位 `0xE2` 是否清掉开屏位我无法确证，拿不确定的语义去把用户画面
|
||||
变黑比不动作更糟。
|
||||
|
||||
## 固件日志为什么要在网页里看得见
|
||||
|
||||
模型把串口按 `SERIAL <行>` 打到 **stderr**，而只有**服务器自己启动 QEMU** 时才会去读这个管道
|
||||
（`uvk5_supervisor` 在连接成功后 `pump_stream` 到日志缓冲）。所以：
|
||||
|
||||
* `work/run-webui.ps1` 默认**自己启动模拟器**（页面上的 On/Off 也就是真的了），
|
||||
日志面板因此能看到固件串口（启动横幅 `UV-K5 Firmware, EGZUMER+F4HWN v5.9.0.CN` 就在里面）；
|
||||
* 加 `-Attach` 则回到"附着到别处启动的模拟器"，此时服务器看不到那个 stderr 流，面板里只有
|
||||
QEMU/电源事件——**这正是之前看不到 debug 日志的原因**。
|
||||
|
||||
这条线上一根线上跑着两种东西：固件自己的可读输出，和 **CPS 编程协议（二进制）**。后者按文本解码会
|
||||
把面板刷成一屏控制字符、把可读的那行埋掉。所以 `uvk5_logs.describe_line()` 现在对"大部分字节不可打印"
|
||||
的行只给**长度 + 十六进制头**，可读行照旧——两种数据都还在，只是不再互相盖住：
|
||||
|
||||
[serial] UV-K5 Firmware, EGZUMER+F4HWN v5.9.0.CN
|
||||
[serial] <binary 15 bytes> f0 aa 55 02 04 80 01 02 03 04 05 06 07 08 09
|
||||
[serial] <binary 255 bytes> 0e 0f 10 11 12 13 14 60 0c 1f 06 f0 15 c3 07 … +231 bytes
|
||||
|
||||
回归测试在 `tools/test_uvk5_logs.py::test_pump_stream_summarises_binary_serial`。
|
||||
|
||||
## 我在这一路上搞错和修掉的东西
|
||||
|
||||
* **更正：固件确实在读 SPI flash。** 我先前写的"26 秒零访问"是**测量假象**——PowerShell 的
|
||||
`2>` 重定向把 QEMU 的 stderr 写成了 UTF-16LE，而我的过滤器在找 `FLASHREAD`/`LCDW` 这样的
|
||||
ASCII 行，于是"什么都没找到"被我当成了"什么都没发生"。按 UTF-16 解码后：SPI1 上是完整的
|
||||
ST7565 序列（`e2 a2 c0 a1 a6 a4 24 81 1f 2b…`），SPI2 上读的全是 `0x00A0xx`（设置区，48 个不同
|
||||
地址，片选翻转 30 次）。
|
||||
* **再更正一次（同一个坑的第二种形态）：字体包确实被读了。** 上面那句"字体包没被读"是我把探针
|
||||
**截断在前 80 次读**得出的——正是 AGENTS 里"没弄清数据形态之前不要截断诊断日志"警告过的错。
|
||||
把上限放到 4000 次、并顺手在菜单里转一圈让固件画汉字之后，实测 3168 次读里：
|
||||
`0x0A0000`(3)、`0x0B0000`(11)、`0x0C0000`(6)、`0x0E0000`(15) —— 都在字体区；另外
|
||||
`0x1E0000` 有 **1024 次、每步正好 32 字节**（= 连续走完 32 KB 的一张字体表）。
|
||||
* **外置 SPI flash 的分区**（由 `gitee.com/oldlicn/betula-multi-system-tool` 的官方数据反推，
|
||||
不靠那张分区图）：
|
||||
|
||||
| 偏移 | 大小 | 内容 |
|
||||
| --- | --- | --- |
|
||||
| `0x000000` | 128 KB | BL/多系统引导 + 设置（固件读 `0x00A0xx`）+ 校准（`0x010000`，与 `make_flash.py` 一致） |
|
||||
| `0x020000` | 4 × 128 KB | 四个固件槽（官方"清空 0x20000-0x40000 … 0x80000-0xA0000"正好四段） |
|
||||
| `0x0A0000` | 256 KB | 用户字体包 16x16（UF2 目标地址就是这里） |
|
||||
| `0x0E0000` | 64 KB | 用户字体包 8x8 |
|
||||
| `0x100000` | 1 MB | 出厂资源区：含拼音串，且 **`0x1E0000` 处有 32 KB 字体表**，固件开机会整张走过 |
|
||||
|
||||
仓库里 16 份官方恢复数据（每份 128 KB）可以直接拼回**真机整片 2 MB 数据**；
|
||||
把这些字节拼出来放在 `F:\dsh-build\flashdump\factory-2MB.img`，补上它之后再跑，
|
||||
**画面上的字会变**——说明模拟器原来的镜像缺了固件真正在用的字体数据
|
||||
（`work/flash-with-factory-resource.img` 就是这个"原厂资源区 + 用户字体包 + 校准"的合成）。
|
||||
剩下没落实的是：屏幕上的 16 像素大字与 `0xA0000` 的字模包**仍不能逐字节对上**
|
||||
（2/24），与 `0x1E0000` 那张表也对不上（0/8），所以"哪个来源供哪一块文字"还没定论。
|
||||
* **修：Windows 上 `rename()` 不覆盖已存在的文件。** flash 回写走"写临时文件再改名"，
|
||||
于是每一次回写都失败（`cannot replace`），设置因此**从不落盘**，而反复的告警把主循环挤住，
|
||||
连 QMP 的问候语都发不出来（表现为"power on failed: timed out"）。改用 `g_rename()`
|
||||
（需 `<glib/gstdio.h>`；Windows 上是 MoveFileEx+替换，Unix 上就是 rename）。现在电源能开、设置能存。
|
||||
* **修：power on 失败时看不到原因。** supervisor 只在连接成功后才去读 QEMU 的 stderr，
|
||||
于是失败只剩一句"QMP 端口没出现"。现在会把 QEMU stderr 的**尾部**记进日志——上面那个 rename
|
||||
问题正是这样浮出来的。
|
||||
* **修：启动器在 TCP 端点下用错参数。** `default_launcher` 现在按端点类型生成
|
||||
`unix:…` 或 `tcp:host:port`（Windows 的 QEMU 根本没有 unix socket）。
|
||||
@@ -0,0 +1,19 @@
|
||||
# Boot one firmware image in the emulator, paused, with a key held from reset.
|
||||
#
|
||||
# powershell -File work/boot-radio.ps1 -Image <path> -Qmp <port> -Gdb <port> [-Flash <img>]
|
||||
#
|
||||
# The quoting matters: passing a Windows path with spaces or commas straight through
|
||||
# cmd /c from a caller that is itself quoted mangles it, which is what "unsupported
|
||||
# machine type" and friends look like from outside. A script keeps one layer of it.
|
||||
param(
|
||||
[Parameter(Mandatory=$true)][string]$Image,
|
||||
[int]$Qmp = 4470,
|
||||
[int]$Gdb = 1260,
|
||||
[string]$Flash = "$PSScriptRoot\ms-scratch.img"
|
||||
)
|
||||
$env:PATH = 'F:\msys64\mingw64\bin;' + $env:PATH
|
||||
$env:UVK5_FLASH_IMAGE = $Flash
|
||||
$qemu = 'F:\dsh-build\qemu-7.2.0\build\qemu-system-arm.exe'
|
||||
& $qemu -M uv-k5-v3 -S -nographic -monitor none -serial null `
|
||||
-qmp "tcp:127.0.0.1:$Qmp,server=on,wait=off" `
|
||||
-kernel $Image -gdb "tcp::$Gdb" 2> "$PSScriptRoot\boot-radio-err.log"
|
||||
@@ -0,0 +1,9 @@
|
||||
# Put the SPI flash image back to its known-good state.
|
||||
#
|
||||
# The firmware writes settings back into assets/flash.img (the PY25Q16 model
|
||||
# flushes on exit), so a session can leave it changed. Stop the emulator first:
|
||||
# a running QEMU holds the old image in memory and overwrites the file on exit.
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$root = Join-Path $PSScriptRoot '..'
|
||||
Copy-Item "$PSScriptRoot\flash-base.img" (Join-Path $root 'assets\flash.img') -Force
|
||||
Write-Host "restored assets/flash.img from work/flash-base.img"
|
||||
@@ -0,0 +1,26 @@
|
||||
# Start the emulated radio with the f4hwn 5.9.0.CN firmware (Windows build).
|
||||
#
|
||||
# powershell -File work\run-emulator.ps1
|
||||
#
|
||||
# QEMU here is a native Windows build made with MSYS2, so it needs the MSYS2
|
||||
# mingw64 DLLs on PATH. QMP is TCP: a Windows QEMU has no unix sockets, which is
|
||||
# the one thing about this project that really was Linux-only.
|
||||
param(
|
||||
[string]$Kernel = "$PSScriptRoot\f4hwn\EGZUMER+F4HWN-v5.9.0.CN.elf",
|
||||
[string]$Flash = "$PSScriptRoot\..\assets\flash.img",
|
||||
[string]$Serial = "$PSScriptRoot\serial.log",
|
||||
[int]$QmpPort = 4444,
|
||||
[int]$GdbPort = 1234
|
||||
)
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$env:PATH = 'F:\msys64\mingw64\bin;' + $env:PATH
|
||||
$qemu = 'F:\dsh-build\qemu-7.2.0\build\qemu-system-arm.exe'
|
||||
|
||||
Write-Host "kernel : $Kernel"
|
||||
Write-Host "flash : $Flash"
|
||||
Write-Host "QMP : tcp:127.0.0.1:$QmpPort GDB: tcp::$GdbPort serial: $Serial"
|
||||
& $qemu -M "uv-k5-v3,flash-image=$Flash" -kernel $Kernel `
|
||||
-display none -monitor none `
|
||||
-serial "file:$Serial" `
|
||||
-qmp "tcp:127.0.0.1:$QmpPort,server=on,wait=off" `
|
||||
-gdb "tcp::$GdbPort"
|
||||
@@ -0,0 +1,37 @@
|
||||
# Serve the web remote control.
|
||||
#
|
||||
# powershell -File work\run-webui.ps1 # the page starts the emulator
|
||||
# powershell -File work\run-webui.ps1 -Attach # attach to run-emulator.ps1
|
||||
#
|
||||
# Owning the process is what puts the firmware's serial output in the page's log
|
||||
# pane: the model prints it to stderr as "SERIAL ..." and the server reads QEMU's
|
||||
# stderr. With -Attach the server never sees that stream, so the pane stays empty.
|
||||
# Owning it also makes the On/Off buttons real.
|
||||
param(
|
||||
[int]$Port = 8080,
|
||||
[int]$QmpPort = 4444,
|
||||
[string]$Frame = '0x200012BE', # gFrameBuffer, proven against the firmware source
|
||||
[string]$Status = '0x2000163E', # gStatusLine
|
||||
[string]$Kernel = "$PSScriptRoot\f4hwn\EGZUMER+F4HWN-v5.9.0.CN.elf",
|
||||
[string]$Flash = [System.IO.Path]::GetFullPath("$PSScriptRoot\..\assets\flash.img"),
|
||||
[string]$Qemu = 'F:\dsh-build\qemu-7.2.0\build\qemu-system-arm.exe',
|
||||
[switch]$Attach
|
||||
)
|
||||
$ErrorActionPreference = 'Stop'
|
||||
# The QEMU this server spawns is a native Windows build, so the MSYS2 mingw64 DLLs
|
||||
# have to be on the child's PATH -- inherited from here, since the child gets this
|
||||
# environment. Without it QEMU dies at load and "power on" reports only that the QMP
|
||||
# port never appeared.
|
||||
$env:PATH = 'F:\msys64\mingw64\bin;' + $env:PATH
|
||||
$py = 'C:\Users\Administrator\.dsh\dsh-runtimes\dsh-primary-runtime\dependencies\python\python.exe'
|
||||
Set-Location (Join-Path $PSScriptRoot '..')
|
||||
|
||||
$common = @('tools\webui.py', '--qmp', "127.0.0.1:$QmpPort",
|
||||
'--frame-addr', $Frame, '--status-addr', $Status, '--port', $Port)
|
||||
if ($Attach) {
|
||||
Write-Host "attaching to an emulator already listening on 127.0.0.1:$QmpPort"
|
||||
& $py @common --attach
|
||||
} else {
|
||||
Write-Host "the page will start: $Qemu"
|
||||
& $py @common --qemu $Qemu --elf $Kernel --flash $Flash
|
||||
}
|
||||
Reference in new issue
Block a user