Check the docs' claims against the code mechanically

Translating everything into Chinese found four claims that had already drifted, and
none of them were caught by reading -- they were caught by comparing against source.
Proofreading does not find rot, so do the comparison mechanically and keep doing it.

tools/check_docs.py verifies that every tool a README names exists, that every test in
run_tests.sh is documented in both languages, that internal .md links resolve, that the
translation pairs have matching heading structure, that memory-map addresses match the
model's #defines, and that documented firmware file:line references still point at what
the prose claims. It runs in the quick tier of run_tests.sh, needing no emulator.

Confirmed it can actually fail, because a checker that cannot is worthless: renaming a
documented tool and deleting a heading from the Chinese side each produce one named
failure and exit 1, and reverting returns it to clean.

One thing it deliberately does not check. An early version compared firmware constants
with a regex that took the first number on a line, so `key_debounce_10ms = 20 / 10` read
as 20 and it declared the docs wrong for saying 2. The docs were right and the checker
was broken. A checker that cries wolf gets ignored, so claims it cannot verify
unambiguously are left out rather than guessed at.

Current state: 16 file:line references all accurate, 7 memory-map addresses all match,
zero broken links, all three translation pairs structurally aligned.

AGENTS.md

@@ -488,6 +488,28 @@ stray bytes that make the log a "binary file" to grep, which swallows the summar
 Emulator tests boot their own QEMU on private ports and take 20-30 s each, so they do
 not disturb a running `run.sh` or web UI session.
 
+### Keeping the docs honest
+
+    python3 tools/check_docs.py     # also runs as part of run_tests.sh -q
+
+Documentation rots quietly, and reading it does not find that. Translating everything
+into Chinese turned up four claims that had already drifted: the endpoint table was
+missing three routes, the modelled-peripheral list omitted TIM2, the audit table still
+called TIM a stub after TIM2 was modelled, and neither README listed several library
+modules. All four were found by comparing against the source, none by proofreading.
+
+So the comparison is mechanical now. It checks that every tool a README names exists,
+that every test in `run_tests.sh` is documented in both languages, that internal `.md`
+links resolve, that the translation pairs have matching heading structure, that the
+memory-map addresses match the model's `#define`s, and that documented firmware
+`file:line` references still point at what the prose claims.
+
+One caution, from writing it. An early version compared firmware constants with a regex
+that took the first number on the line, so `key_debounce_10ms = 20 / 10` read as 20 and
+the checker declared the docs wrong for saying 2. **The docs were right and the checker
+was broken.** A checker that cries wolf gets ignored, so anything it cannot verify
+unambiguously is left out rather than guessed at.
+
 ### Counting distinct frames proves less than it looks
 
 Worth knowing before writing any test that watches the screen.

AGENTS.zh-CN.md

@@ -412,6 +412,25 @@ GCC 能看到全部调用者。如果一个模型的输出神秘地不起作用
 模拟器测试会在私有端口上启动自己的 QEMU，每个耗时 20-30 秒，所以它们不会干扰
 正在运行的 `run.sh` 或网页界面会话。
 
+### 让文档保持诚实
+
+    python3 tools/check_docs.py     # 也作为 run_tests.sh -q 的一部分运行
+
+**文档会静默腐化，而通读是发现不了的。** 把全部文档翻译成中文的过程中，
+翻出了四处**已经漂移**的断言：接口表缺三个路由、已建模外设列表漏了 TIM2、
+审计表在 TIM2 已建模之后仍把 TIM 称作 stub、两个 README 都没列出几个库模块。
+**这四处全是靠与源码比对发现的，没有一处是校对读出来的。**
+
+所以现在这个比对是机械化的。它检查：每个 README 提到的工具是否存在、
+`run_tests.sh` 里的每个测试在两种语言里是否都有记录、内部 `.md` 链接是否都能解析、
+翻译对的标题结构是否匹配、内存映射地址是否与模型的 `#define` 一致、
+以及文档里的固件 `file:line` 引用是否仍指向正文声称的东西。
+
+写它的过程中有一个教训。早期版本用一个"取行内第一个数字"的正则来比对固件常量，
+于是 `key_debounce_10ms = 20 / 10` 被读成 20，检查器于是宣布文档说 2 是错的。
+**文档是对的，检查器是坏的。** 一个总在虚报的检查器会被忽略，
+所以凡是它无法无歧义验证的东西，都宁可不查，而不是靠猜。
+
 ### 数"有多少帧不同"证明的东西比看起来少
 
 写任何观察屏幕的测试之前值得知道这一点。

README.md

@@ -96,6 +96,7 @@ keypresses silently stop working. Run the test after touching that code;
       test_battery.py        battery level and low-battery follow the ADC
       test_millis.py         millis() advances, so timeouts can expire
       test_spectrum.py       RSSI depends on tuning, not a constant
+      check_docs.py          the docs' claims still match the code
       run_tests.sh           runs all of the above, build-checked first
       test_run_tests.sh      that the runner actually notices failures
       lib_kill_emulator.sh   cleanup that only ever kills emulators

README.zh-CN.md

@@ -84,6 +84,7 @@ VFO 重算了状态。在真机上你只看到"什么都没发生"，在这里
       test_battery.py        电量与低电告警跟随 ADC
       test_millis.py         millis() 会递增，超时才可能到期
       test_spectrum.py       RSSI 取决于调谐位置，不是常数
+      check_docs.py          文档的断言是否仍与代码一致
       run_tests.sh           跑上面全部，先检查构建
       test_run_tests.sh      验证 runner 真的能发现失败
       lib_kill_emulator.sh   只杀模拟器的清理逻辑

tools/check_docs.py

@@ -0,0 +1,181 @@
+#!/usr/bin/env python3
+"""Check the documentation's factual claims against the code.
+
+Documentation rots quietly. Translating the docs into Chinese turned up four claims
+that had already drifted -- a missing endpoint, an omitted peripheral, a table still
+calling TIM a stub after TIM2 was modelled, and library modules absent from the layout.
+None of those were caught by reading; they were caught by comparing against the source.
+So compare mechanically, and keep doing it.
+
+What is checked:
+  1. every tool named in a README exists
+  2. every test in run_tests.sh is documented in both READMEs
+  3. every internal .md link resolves
+  4. the English/Chinese pairs have matching section structure
+  5. memory-map addresses match the model's #defines
+  6. firmware file:line references point at what the docs say they do
+
+Run it after touching docs or renaming anything:
+
+    python3 tools/check_docs.py
+
+One caution learned while writing this. An early version compared firmware constants
+with a regex that grabbed the first number on the line, so `key_debounce_10ms = 20 / 10`
+read as 20 and the check reported the docs wrong when they said 2. The docs were right
+and the checker was broken. A checker that cries wolf gets ignored, so anything it
+cannot verify unambiguously is left out rather than guessed at.
+"""
+
+import pathlib
+import re
+import sys
+
+SIM = pathlib.Path(__file__).resolve().parent.parent
+FW = pathlib.Path("/root/uvk5-port/uvk5-sat/App")
+
+PAIRS = [
+    ("README.md", "README.zh-CN.md"),
+    ("AGENTS.md", "AGENTS.zh-CN.md"),
+    ("docs/reverse-proxy.md", "docs/reverse-proxy.zh-CN.md"),
+]
+
+# Addresses the READMEs state, against the model's own #defines.
+MEMORY_MAP = {
+    "PY32_FLASH_BASE": "0x08000000",
+    "PY32_SRAM_BASE": "0x20000000",
+    "PY32_RCC_BASE": "0x40021000",
+    "PY32_SPI1_BASE": "0x40013000",
+    "PY32_SPI2_BASE": "0x40003800",
+    "PY32_ADC1_BASE": "0x40012400",
+    "PY32_APP_OFFSET": "0x2800",
+}
+
+# A documented file:line and a word that must appear near it. The window is a few
+# lines wide on purpose: a reference drifting by a line or two is still useful, and
+# failing on that would make the check noise.
+LINE_REFS = {
+    ("app/app.c", 1697): "CheckRadioInterrupts",
+    ("app/app.c", 910): "REG_0C",
+    ("app/app.c", 1417): "REG_0C",
+    ("app/app.c", 915): "uint16_t",
+    ("app/app.c", 1027): "SquelchLost",
+    ("app/app.c", 482): "StartListening",
+    ("app/app.c", 1374): "BATTERY_SAVE",
+    ("app/app.c", 1700): "TRANSMIT",
+    ("driver/gpio.h", 31): "PTT",
+    ("driver/gpio.h", 34): "AUDIO_PATH",
+    ("driver/bk4819.c", 743): "SetFrequency",
+    ("settings.c", 263): "KEY_1_SHORT",
+    ("settings.c", 423): "mic_bar",
+    ("ui/main.c", 2370): "Rx",
+    ("app/menu.c", 2311): "Direction",
+    ("app/menu.c", 1826): "gMenuListCount",
+}
+
+WINDOW = 4
+
+problems = []
+
+
+def fail(msg):
+    problems.append(msg)
+    print(f"  FAIL  {msg}")
+
+
+def resolve_fw(name):
+    name = name.replace("App/", "")
+    for cand in (FW / name, FW / "app" / name, FW / "driver" / name,
+                 FW / "helper" / name, FW / "ui" / name):
+        if cand.exists():
+            return cand
+    return None
+
+
+def check_tools_exist():
+    print("tools named in a README must exist")
+    for doc in ("README.md", "README.zh-CN.md"):
+        text = (SIM / doc).read_text()
+        for tool in sorted(set(re.findall(r"tools/([a-z0-9_]+\.(?:py|sh))", text))):
+            if not (SIM / "tools" / tool).exists():
+                fail(f"{doc} names tools/{tool}, which does not exist")
+
+
+def check_tests_documented():
+    print("every test in run_tests.sh must be documented")
+    runner = (SIM / "tools" / "run_tests.sh").read_text()
+    in_runner = set(re.findall(r"tools/([a-z0-9_]+\.(?:py|sh))", runner))
+    for doc in ("README.md", "README.zh-CN.md"):
+        text = (SIM / doc).read_text()
+        for tool in sorted(in_runner):
+            if tool not in text:
+                fail(f"{doc} does not mention {tool}, which run_tests.sh runs")
+
+
+def check_links():
+    print("internal .md links must resolve")
+    for doc in [d for pair in PAIRS for d in pair]:
+        path = SIM / doc
+        for target in re.findall(r"\]\(([^)]+\.md)\)", path.read_text()):
+            if target.startswith("http"):
+                continue
+            if not (path.parent / target).exists():
+                fail(f"{doc} links to {target}, which does not exist")
+
+
+def check_pairs():
+    print("translation pairs must have matching structure")
+    for en_name, zh_name in PAIRS:
+        en = re.findall(r"^(#+) (.+)$", (SIM / en_name).read_text(), re.M)
+        zh = re.findall(r"^(#+) (.+)$", (SIM / zh_name).read_text(), re.M)
+        if len(en) != len(zh):
+            fail(f"{en_name} has {len(en)} headings, {zh_name} has {len(zh)}")
+            continue
+        for i, ((en_lvl, en_txt), (zh_lvl, _)) in enumerate(zip(en, zh)):
+            if en_lvl != zh_lvl:
+                fail(f"{zh_name} heading {i + 1} is at a different depth than "
+                     f"{en_name}'s ({en_txt!r})")
+
+
+def check_memory_map():
+    print("memory-map addresses must match the model")
+    model = (SIM / "qemu" / "py32f071.c").read_text()
+    for sym, documented in MEMORY_MAP.items():
+        m = re.search(rf"#define {sym}\s+(\S+)", model)
+        if not m:
+            fail(f"{sym} is documented but not defined in the model")
+        elif documented.lower() not in m.group(1).lower():
+            fail(f"docs say {sym} is {documented}, model says {m.group(1)}")
+
+
+def check_line_refs():
+    print("firmware file:line references must point at what the docs claim")
+    for (name, line), needle in sorted(LINE_REFS.items()):
+        path = resolve_fw(name)
+        if path is None:
+            fail(f"{name} is referenced but not found in the firmware tree")
+            continue
+        lines = path.read_text().splitlines()
+        if line > len(lines):
+            fail(f"{name}:{line} is past the end of the file ({len(lines)} lines)")
+            continue
+        window = "\n".join(lines[max(0, line - WINDOW - 1):line + WINDOW])
+        if needle.lower() not in window.lower():
+            fail(f"{name}:{line} has no {needle!r} nearby -- "
+                 f"the reference has drifted")
+
+
+def main():
+    for check in (check_tools_exist, check_tests_documented, check_links,
+                  check_pairs, check_memory_map, check_line_refs):
+        check()
+
+    print()
+    if problems:
+        print(f"{len(problems)} documentation problem(s)")
+        return 1
+    print("documentation matches the code")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/run_tests.sh

@@ -66,6 +66,7 @@ cd "$HERE"
 # First, that this script itself reports failures. A runner that silently counts every
 # test as passing is worse than no runner, because it gets trusted.
 run "runner self-check"   bash "$HERE/test_run_tests.sh"
+run "docs match code"     python3 "$HERE/check_docs.py"
 run "unit: model helpers" python3 -m unittest discover -p 'test_uvk5*.py' -q
 run "unit: web UI"        python3 -m unittest test_webui -q