Add a test runner, and a test that it can fail

The suite had grown to ten separate invocations that had to be remembered and pasted
in the right order, which is how regressions slip through: it is too easy to run the
two tests near what you changed and miss the one that broke. Now:

    bash tools/run_tests.sh        # everything, 11 suites, a few minutes
    bash tools/run_tests.sh -q     # unit tests only, ~15 s, no emulator

The build is checked first and a failure stops everything, because ninja leaves the
previous binary in place and the tests would otherwise report results for code that
was never compiled.

Two defects in the runner's own first draft, both caught before it was trusted:

It used `if "$@" | sed ...; then`, which tests sed's exit status rather than the
test's. sed practically always succeeds, so every test would have been counted as
passing no matter what failed -- a runner that silently cannot fail is worse than no
runner. Fixed with PIPESTATUS[0], and tools/test_run_tests.sh now asserts that a
failing test is counted and named, that the runner exits non-zero, and that the
accounting survives binary noise in test output.

That noise was the second defect: gdb-driven tests emit stray bytes, which made the
combined log a "binary file" as far as grep was concerned and silently swallowed the
summary line. Output now passes through tr -cd first.

Full run: 11 passed, 0 failed.
This commit is contained in:
mckero committed 2026-08-29 05:42:11 +01:00
1 parent 95bad1614e
commit 8d1a1c4415
4 files changed
+221

No files matched your search

+9
View File
@@ -86,6 +86,8 @@ keypresses silently stop working. Run the test after touching that code;
test_smeter.py the S-meter reads a signal when monitoring
test_ptt.py PTT keys the radio and releases cleanly
test_scan.py a busy band does not stall a scan
run_tests.sh runs all of the above, build-checked first
test_run_tests.sh that the runner actually notices failures
lib_kill_emulator.sh cleanup that only ever kills emulators
webui.py web remote control: live LCD plus clickable keypad
dn42_firewall.sh restrict the web UI port to DN42 sources
@@ -125,6 +127,13 @@ Needs a QEMU 7.2 source tree, `meson`, `ninja`, `libfdt-dev`, `libglib2.0-dev`,
Then check the build actually works, which takes about a minute:
bash tools/run_tests.sh # everything, a few minutes
bash tools/run_tests.sh -q # unit tests only, ~15 s, no emulator
The runner checks the build first and refuses to continue if it fails, because ninja
leaves the previous binary in place and the tests would otherwise pass against code
that was never compiled. Individual tests still run standalone:
python3 tools/keypad_test.py
python3 tools/test_flash_persist.py
python3 tools/test_freq_entry.py