diff --git a/AGENTS.md b/AGENTS.md index bfb68bf..a8c30bf 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -488,6 +488,28 @@ stray bytes that make the log a "binary file" to grep, which swallows the summar Emulator tests boot their own QEMU on private ports and take 20-30 s each, so they do not disturb a running `run.sh` or web UI session. +### Keeping the docs honest + + python3 tools/check_docs.py # also runs as part of run_tests.sh -q + +Documentation rots quietly, and reading it does not find that. Translating everything +into Chinese turned up four claims that had already drifted: the endpoint table was +missing three routes, the modelled-peripheral list omitted TIM2, the audit table still +called TIM a stub after TIM2 was modelled, and neither README listed several library +modules. All four were found by comparing against the source, none by proofreading. + +So the comparison is mechanical now. It checks that every tool a README names exists, +that every test in `run_tests.sh` is documented in both languages, that internal `.md` +links resolve, that the translation pairs have matching heading structure, that the +memory-map addresses match the model's `#define`s, and that documented firmware +`file:line` references still point at what the prose claims. + +One caution, from writing it. An early version compared firmware constants with a regex +that took the first number on the line, so `key_debounce_10ms = 20 / 10` read as 20 and +the checker declared the docs wrong for saying 2. **The docs were right and the checker +was broken.** A checker that cries wolf gets ignored, so anything it cannot verify +unambiguously is left out rather than guessed at. + ### Counting distinct frames proves less than it looks Worth knowing before writing any test that watches the screen. diff --git a/AGENTS.zh-CN.md b/AGENTS.zh-CN.md index dba709d..505b869 100644 --- a/AGENTS.zh-CN.md +++ b/AGENTS.zh-CN.md @@ -412,6 +412,25 @@ GCC 能看到全部调用者。如果一个模型的输出神秘地不起作用 模拟器测试会在私有端口上启动自己的 QEMU,每个耗时 20-30 秒,所以它们不会干扰 正在运行的 `run.sh` 或网页界面会话。 +### 让文档保持诚实 + + python3 tools/check_docs.py # 也作为 run_tests.sh -q 的一部分运行 + +**文档会静默腐化,而通读是发现不了的。** 把全部文档翻译成中文的过程中, +翻出了四处**已经漂移**的断言:接口表缺三个路由、已建模外设列表漏了 TIM2、 +审计表在 TIM2 已建模之后仍把 TIM 称作 stub、两个 README 都没列出几个库模块。 +**这四处全是靠与源码比对发现的,没有一处是校对读出来的。** + +所以现在这个比对是机械化的。它检查:每个 README 提到的工具是否存在、 +`run_tests.sh` 里的每个测试在两种语言里是否都有记录、内部 `.md` 链接是否都能解析、 +翻译对的标题结构是否匹配、内存映射地址是否与模型的 `#define` 一致、 +以及文档里的固件 `file:line` 引用是否仍指向正文声称的东西。 + +写它的过程中有一个教训。早期版本用一个"取行内第一个数字"的正则来比对固件常量, +于是 `key_debounce_10ms = 20 / 10` 被读成 20,检查器于是宣布文档说 2 是错的。 +**文档是对的,检查器是坏的。** 一个总在虚报的检查器会被忽略, +所以凡是它无法无歧义验证的东西,都宁可不查,而不是靠猜。 + ### 数"有多少帧不同"证明的东西比看起来少 写任何观察屏幕的测试之前值得知道这一点。 diff --git a/README.md b/README.md index 1e838f5..0e5c8aa 100644 --- a/README.md +++ b/README.md @@ -96,6 +96,7 @@ keypresses silently stop working. Run the test after touching that code; test_battery.py battery level and low-battery follow the ADC test_millis.py millis() advances, so timeouts can expire test_spectrum.py RSSI depends on tuning, not a constant + check_docs.py the docs' claims still match the code run_tests.sh runs all of the above, build-checked first test_run_tests.sh that the runner actually notices failures lib_kill_emulator.sh cleanup that only ever kills emulators diff --git a/README.zh-CN.md b/README.zh-CN.md index 85ca78c..f5f24ca 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -84,6 +84,7 @@ VFO 重算了状态。在真机上你只看到"什么都没发生",在这里 test_battery.py 电量与低电告警跟随 ADC test_millis.py millis() 会递增,超时才可能到期 test_spectrum.py RSSI 取决于调谐位置,不是常数 + check_docs.py 文档的断言是否仍与代码一致 run_tests.sh 跑上面全部,先检查构建 test_run_tests.sh 验证 runner 真的能发现失败 lib_kill_emulator.sh 只杀模拟器的清理逻辑 diff --git a/tools/check_docs.py b/tools/check_docs.py new file mode 100755 index 0000000..7d4dbe4 --- /dev/null +++ b/tools/check_docs.py @@ -0,0 +1,181 @@ +#!/usr/bin/env python3 +"""Check the documentation's factual claims against the code. + +Documentation rots quietly. Translating the docs into Chinese turned up four claims +that had already drifted -- a missing endpoint, an omitted peripheral, a table still +calling TIM a stub after TIM2 was modelled, and library modules absent from the layout. +None of those were caught by reading; they were caught by comparing against the source. +So compare mechanically, and keep doing it. + +What is checked: + 1. every tool named in a README exists + 2. every test in run_tests.sh is documented in both READMEs + 3. every internal .md link resolves + 4. the English/Chinese pairs have matching section structure + 5. memory-map addresses match the model's #defines + 6. firmware file:line references point at what the docs say they do + +Run it after touching docs or renaming anything: + + python3 tools/check_docs.py + +One caution learned while writing this. An early version compared firmware constants +with a regex that grabbed the first number on the line, so `key_debounce_10ms = 20 / 10` +read as 20 and the check reported the docs wrong when they said 2. The docs were right +and the checker was broken. A checker that cries wolf gets ignored, so anything it +cannot verify unambiguously is left out rather than guessed at. +""" + +import pathlib +import re +import sys + +SIM = pathlib.Path(__file__).resolve().parent.parent +FW = pathlib.Path("/root/uvk5-port/uvk5-sat/App") + +PAIRS = [ + ("README.md", "README.zh-CN.md"), + ("AGENTS.md", "AGENTS.zh-CN.md"), + ("docs/reverse-proxy.md", "docs/reverse-proxy.zh-CN.md"), +] + +# Addresses the READMEs state, against the model's own #defines. +MEMORY_MAP = { + "PY32_FLASH_BASE": "0x08000000", + "PY32_SRAM_BASE": "0x20000000", + "PY32_RCC_BASE": "0x40021000", + "PY32_SPI1_BASE": "0x40013000", + "PY32_SPI2_BASE": "0x40003800", + "PY32_ADC1_BASE": "0x40012400", + "PY32_APP_OFFSET": "0x2800", +} + +# A documented file:line and a word that must appear near it. The window is a few +# lines wide on purpose: a reference drifting by a line or two is still useful, and +# failing on that would make the check noise. +LINE_REFS = { + ("app/app.c", 1697): "CheckRadioInterrupts", + ("app/app.c", 910): "REG_0C", + ("app/app.c", 1417): "REG_0C", + ("app/app.c", 915): "uint16_t", + ("app/app.c", 1027): "SquelchLost", + ("app/app.c", 482): "StartListening", + ("app/app.c", 1374): "BATTERY_SAVE", + ("app/app.c", 1700): "TRANSMIT", + ("driver/gpio.h", 31): "PTT", + ("driver/gpio.h", 34): "AUDIO_PATH", + ("driver/bk4819.c", 743): "SetFrequency", + ("settings.c", 263): "KEY_1_SHORT", + ("settings.c", 423): "mic_bar", + ("ui/main.c", 2370): "Rx", + ("app/menu.c", 2311): "Direction", + ("app/menu.c", 1826): "gMenuListCount", +} + +WINDOW = 4 + +problems = [] + + +def fail(msg): + problems.append(msg) + print(f" FAIL {msg}") + + +def resolve_fw(name): + name = name.replace("App/", "") + for cand in (FW / name, FW / "app" / name, FW / "driver" / name, + FW / "helper" / name, FW / "ui" / name): + if cand.exists(): + return cand + return None + + +def check_tools_exist(): + print("tools named in a README must exist") + for doc in ("README.md", "README.zh-CN.md"): + text = (SIM / doc).read_text() + for tool in sorted(set(re.findall(r"tools/([a-z0-9_]+\.(?:py|sh))", text))): + if not (SIM / "tools" / tool).exists(): + fail(f"{doc} names tools/{tool}, which does not exist") + + +def check_tests_documented(): + print("every test in run_tests.sh must be documented") + runner = (SIM / "tools" / "run_tests.sh").read_text() + in_runner = set(re.findall(r"tools/([a-z0-9_]+\.(?:py|sh))", runner)) + for doc in ("README.md", "README.zh-CN.md"): + text = (SIM / doc).read_text() + for tool in sorted(in_runner): + if tool not in text: + fail(f"{doc} does not mention {tool}, which run_tests.sh runs") + + +def check_links(): + print("internal .md links must resolve") + for doc in [d for pair in PAIRS for d in pair]: + path = SIM / doc + for target in re.findall(r"\]\(([^)]+\.md)\)", path.read_text()): + if target.startswith("http"): + continue + if not (path.parent / target).exists(): + fail(f"{doc} links to {target}, which does not exist") + + +def check_pairs(): + print("translation pairs must have matching structure") + for en_name, zh_name in PAIRS: + en = re.findall(r"^(#+) (.+)$", (SIM / en_name).read_text(), re.M) + zh = re.findall(r"^(#+) (.+)$", (SIM / zh_name).read_text(), re.M) + if len(en) != len(zh): + fail(f"{en_name} has {len(en)} headings, {zh_name} has {len(zh)}") + continue + for i, ((en_lvl, en_txt), (zh_lvl, _)) in enumerate(zip(en, zh)): + if en_lvl != zh_lvl: + fail(f"{zh_name} heading {i + 1} is at a different depth than " + f"{en_name}'s ({en_txt!r})") + + +def check_memory_map(): + print("memory-map addresses must match the model") + model = (SIM / "qemu" / "py32f071.c").read_text() + for sym, documented in MEMORY_MAP.items(): + m = re.search(rf"#define {sym}\s+(\S+)", model) + if not m: + fail(f"{sym} is documented but not defined in the model") + elif documented.lower() not in m.group(1).lower(): + fail(f"docs say {sym} is {documented}, model says {m.group(1)}") + + +def check_line_refs(): + print("firmware file:line references must point at what the docs claim") + for (name, line), needle in sorted(LINE_REFS.items()): + path = resolve_fw(name) + if path is None: + fail(f"{name} is referenced but not found in the firmware tree") + continue + lines = path.read_text().splitlines() + if line > len(lines): + fail(f"{name}:{line} is past the end of the file ({len(lines)} lines)") + continue + window = "\n".join(lines[max(0, line - WINDOW - 1):line + WINDOW]) + if needle.lower() not in window.lower(): + fail(f"{name}:{line} has no {needle!r} nearby -- " + f"the reference has drifted") + + +def main(): + for check in (check_tools_exist, check_tests_documented, check_links, + check_pairs, check_memory_map, check_line_refs): + check() + + print() + if problems: + print(f"{len(problems)} documentation problem(s)") + return 1 + print("documentation matches the code") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/run_tests.sh b/tools/run_tests.sh index 8e95f69..aac794f 100755 --- a/tools/run_tests.sh +++ b/tools/run_tests.sh @@ -66,6 +66,7 @@ cd "$HERE" # First, that this script itself reports failures. A runner that silently counts every # test as passing is worse than no runner, because it gets trusted. run "runner self-check" bash "$HERE/test_run_tests.sh" +run "docs match code" python3 "$HERE/check_docs.py" run "unit: model helpers" python3 -m unittest discover -p 'test_uvk5*.py' -q run "unit: web UI" python3 -m unittest test_webui -q