Check that documented tool flags exist

The remaining class of claim check_docs.py could not see: whether the commands in the
docs would actually run. A renamed or removed option is the classic form of command
rot, and the one that wastes a reader's time most directly -- they paste the line and
it fails. All 9 documented flags across screenshot.py, webui.py and restore_flash.sh
are real.

The check earned its own lesson, recorded in both languages. Its first version matched
only to the end of the line, so on a wrapped command it saw --frame-addr and nothing
after the backslash: 4 of 9 flags, and it reported a clean run. A check that silently
covers a quarter of what it claims is worse than no check, because the clean result is
believed. Continuations are joined before matching now.

Confirmed it fails when it should: renaming --frame-addr to something no tool accepts
produces two named failures and exit 1, and reverting returns it to clean.

check_docs.py now runs seven checks.
This commit is contained in:
mckero committed 2026-08-29 16:58:28 +01:00
1 parent b32335d8c0
commit ee80939c78
3 files changed
+58 -4

No files matched your search

+13 -2
View File
@@ -501,8 +501,19 @@ modules. All four were found by comparing against the source, none by proofreadi
So the comparison is mechanical now. It checks that every tool a README names exists, So the comparison is mechanical now. It checks that every tool a README names exists,
that every test in `run_tests.sh` is documented in both languages, that internal `.md` that every test in `run_tests.sh` is documented in both languages, that internal `.md`
links resolve, that the translation pairs have matching heading structure, that the links resolve, that the translation pairs have matching heading structure, that the
memory-map addresses match the model's `#define`s, and that documented firmware memory-map addresses match the model's `#define`s, that every long flag a doc passes to
`file:line` references still point at what the prose claims. a tool actually exists in it, and that documented firmware `file:line` references still
point at what the prose claims.
The flag check earned its own lesson. Its first version matched only to the end of the
line, so on a wrapped command like
python3 tools/screenshot.py --frame-addr 0x200013DC \
--status-addr 0x2000175C --port 1234 --out screen.png
it saw `--frame-addr` and nothing else -- 4 of 9 flags, and it reported a clean run.
**A check that silently covers a quarter of what it claims is worse than no check**,
because the clean result is believed. Continuations are joined before matching now.
One caution, from writing it. An early version compared firmware constants with a regex One caution, from writing it. An early version compared firmware constants with a regex
that took the first number on the line, so `key_debounce_10ms = 20 / 10` read as 20 and that took the first number on the line, so `key_debounce_10ms = 20 / 10` read as 20 and
+10
View File
@@ -424,8 +424,18 @@ GCC 能看到全部调用者。如果一个模型的输出神秘地不起作用
所以现在这个比对是机械化的。它检查:每个 README 提到的工具是否存在、 所以现在这个比对是机械化的。它检查:每个 README 提到的工具是否存在、
`run_tests.sh` 里的每个测试在两种语言里是否都有记录、内部 `.md` 链接是否都能解析、 `run_tests.sh` 里的每个测试在两种语言里是否都有记录、内部 `.md` 链接是否都能解析、
翻译对的标题结构是否匹配、内存映射地址是否与模型的 `#define` 一致、 翻译对的标题结构是否匹配、内存映射地址是否与模型的 `#define` 一致、
文档传给某个工具的每个长参数是否真的存在于该工具中、
以及文档里的固件 `file:line` 引用是否仍指向正文声称的东西。 以及文档里的固件 `file:line` 引用是否仍指向正文声称的东西。
参数检查本身也留下了一个教训。它的第一版只匹配到行尾,所以对一条这样换行的命令
python3 tools/screenshot.py --frame-addr 0x200013DC \
--status-addr 0x2000175C --port 1234 --out screen.png
它只看到了 `--frame-addr`,别的都没看到 —— 9 个参数里查了 4 个,然后**报告一切干净**。
**一个静默地只覆盖了自己所声称范围四分之一的检查,比没有检查更糟**,
因为那个"干净"的结果会被相信。现在会先合并续行再做匹配。
写它的过程中有一个教训。早期版本用一个"取行内第一个数字"的正则来比对固件常量, 写它的过程中有一个教训。早期版本用一个"取行内第一个数字"的正则来比对固件常量,
于是 `key_debounce_10ms = 20 / 10` 被读成 20,检查器于是宣布文档说 2 是错的。 于是 `key_debounce_10ms = 20 / 10` 被读成 20,检查器于是宣布文档说 2 是错的。
**文档是对的,检查器是坏的。** 一个总在虚报的检查器会被忽略, **文档是对的,检查器是坏的。** 一个总在虚报的检查器会被忽略,
+35 -2
View File
@@ -13,7 +13,8 @@ What is checked:
3. every internal .md link resolves 3. every internal .md link resolves
4. the English/Chinese pairs have matching section structure 4. the English/Chinese pairs have matching section structure
5. memory-map addresses match the model's #defines 5. memory-map addresses match the model's #defines
6. firmware file:line references point at what the docs say they do 6. every long flag a doc passes to a tool exists in that tool
7. firmware file:line references point at what the docs say they do
Run it after touching docs or renaming anything: Run it after touching docs or renaming anything:
@@ -147,6 +148,37 @@ def check_memory_map():
fail(f"docs say {sym} is {documented}, model says {m.group(1)}") fail(f"docs say {sym} is {documented}, model says {m.group(1)}")
def check_documented_flags():
"""Every long flag a doc attributes to a tool must exist in that tool.
A renamed or removed option is the classic form of command rot, and it is the one
that wastes a reader's time most directly: they paste the line and it fails.
Backslash continuations are joined first. Without that, the regex stops at the
newline and only sees the first flag of a wrapped command -- which checked 4 of the
9 flags here and reported a clean run. A check that silently covers a quarter of
what it claims is worse than no check.
"""
print("documented tool flags must exist")
text = "\n".join(
(SIM / doc).read_text() for pair in PAIRS for doc in pair)
joined = re.sub(r"\\\s*\n\s*", " ", text)
claims = {}
for m in re.finditer(r"tools/([a-z0-9_]+\.(?:py|sh))([^\n]*)", joined):
for flag in re.findall(r"(--[a-z][a-z-]+)", m.group(2)):
claims.setdefault(m.group(1), set()).add(flag)
for tool, flags in sorted(claims.items()):
path = SIM / "tools" / tool
if not path.exists():
continue # already reported by check_tools_exist
src = path.read_text()
for flag in sorted(flags):
if flag not in src:
fail(f"docs pass {flag} to {tool}, which does not accept it")
def check_line_refs(): def check_line_refs():
print("firmware file:line references must point at what the docs claim") print("firmware file:line references must point at what the docs claim")
for (name, line), needle in sorted(LINE_REFS.items()): for (name, line), needle in sorted(LINE_REFS.items()):
@@ -166,7 +198,8 @@ def check_line_refs():
def main(): def main():
for check in (check_tools_exist, check_tests_documented, check_links, for check in (check_tools_exist, check_tests_documented, check_links,
check_pairs, check_memory_map, check_line_refs): check_pairs, check_memory_map, check_documented_flags,
check_line_refs):
check() check()
print() print()