Build overlay apps without the Arm toolchain, and say how far verification got

pip install ziglang cross-compiles to thumb-freestanding-eabi, which is enough to build a .app with no arm-none-eabi-gcc and no Docker. Measured refusals: --defsym, -Ttext and --section-start come back as unsupported linker args, so the VMA is resolved into a copy of app.ld; -T is forwarded (a missing script errors); --image-base is accepted but page-aligns the segments into 0x200103C8 and 0x20020C94. tools/elf2bin.py extracts allocated sections rather than program headers, because lld maps the ELF header and phdr table as a 180-byte LOAD of its own -- following the headers starts the image at 0x20000000 and APP_ERR_VMA. One division pulled in __aeabi_uidiv, which a -nostdlib blob cannot have: Minesweeper now avoids division entirely. app_main carries the .text.entry attribute upstream's apps use, so the entry is first for the loader's jump to offset 0.

Minesweeper builds to 2408 bytes of code against a 4096-byte budget. Installed through the page, the firmware reads the slot header twice and then exactly code_size bytes from slot+0x1000 -- the only read of that size in the boot log -- so the blob shape, header, CRC, VMA and offset are all accepted. Whether control reaches the overlay is unproven: the 100 ms PC probe saw no overlay address, no APP ERROR screen appears, and the app does not draw. test_elf2bin pins the phantom-header-segment lesson; both AGENTS files record the rest.

AGENTS.md

@@ -661,6 +661,33 @@ written back by its own write-back anyway. Rolling the image back is what worked
 its in-memory image back on exit, so an edit made while a guest was live can be overwritten by the
 copy that guest was holding -- which is also how the marker reappeared after being cleared.
 
+**Building an overlay app without the Arm toolchain: `pip install ziglang`.** There is no
+`arm-none-eabi-gcc` and no Docker on the machine this was written on, but Zig ships a C compiler
+that cross-compiles to `thumb-freestanding-eabi`, which is enough. What Zig refuses, each measured:
+`--defsym`, `-Ttext`, `--section-start` (its linker-argument whitelist) -- so the VMA is resolved
+into a copy of `app.ld` with `sed` instead; `-T` *is* forwarded, because a nonexistent script makes
+it error; and `--image-base` is accepted but useless here, because it page-aligns the segments
+(0x20000280 becomes LOADs at 0x200103C8 and 0x20020C94).
+
+Two traps cost the most time. **lld maps the ELF header and program-header table as a LOAD of its
+own** -- 52 + 4x32 = 180 bytes at the image base with no section content -- so `tools/elf2bin.py`
+extracts *allocated sections* rather than program headers; following the program headers starts the
+image at 0x20000000 and the loader refuses it with APP_ERR_VMA. And **one division pulled in
+`__aeabi_uidiv`**, which does not exist in a `-nostdlib` blob: the fix is to remove the divisions
+(repeated subtraction; `& 127` with a reject instead of `% 81`), not to link a soft-divide routine
+into a 4 KiB overlay.
+
+The entry must be first in the image, because the loader jumps to blob offset 0, so `app_main`
+carries the same `__attribute__((section(".text.entry"), used))` that upstream's apps use
+(`cube3d_app.c:183`).
+
+Measured with the result installed through the page: the firmware reads the slot header twice and then
+**exactly `code_size` bytes from slot + 0x1000** (2408 for Minesweeper's 2408-byte code, and it is
+the only read of that size in the whole boot log). So the blob's shape, its header, the CRC, the VMA
+and the offset are all accepted. **Whether control then reaches the overlay is still unproven**: the PC
+probe samples every 100 ms and saw no overlay address, no `APP ERROR` screen appears, and the app does
+not draw. That is where this stands -- the pipeline is verified up to the load, not up to execution.
+
 ## The keypad: two real bugs, both fixed
 
 The old note here said "keys reach the firmware but the UI does not react" and

AGENTS.zh-CN.md

@@ -535,6 +535,27 @@ app 区被读过 —— 而多系统菜单读的是**固件槽**，不是应用
 **而当"改了不生效"时，过期的写回依然值得怀疑**：模拟器退出时会把内存里的镜像写回文件，所以客人还活着
 时做的改动，可能被它手里那份副本覆盖 —— 这也是标记被清掉后又重新出现的原因。
 
+**没有 Arm 工具链也能编叠加应用：`pip install ziglang`。** 写它的机器上既没有 `arm-none-eabi-gcc`
+也没有 Docker，但 Zig 自带 C 编译器、能交叉到 `thumb-freestanding-eabi`，够用了。Zig 会拒绝什么（都实测过）：
+`--defsym`、`-Ttext`、`--section-start`（被它的链接参数白名单挡掉）；所以 VMA 用 `sed` 写进一份
+`app.ld` 的副本；`-T` **是**会被转发的（给一个不存在的脚本它会报错）；`--image-base` 虽然接受但没有用，
+因为它把段按页对齐（0x20000280 变成 0x200103C8 与 0x20020C94 两个 LOAD）。
+
+最花时间的是两个坑。**lld 会把 ELF 头与程序头表本身也映射成一个 LOAD** —— 52 + 4×32 = 180 字节、位于镜像
+基址、里面没有任何节内容 —— 所以 `tools/elf2bin.py` **按已分配节提取，而不是按程序头**；跟着程序头走会让
+镜像从 0x20000000 开始，加载器以 APP_ERR_VMA 拒绝。另一个是**一次除法就会牵出 `__aeabi_uidiv`**，而它在
+`-nostdlib` 的块里并不存在 —— 正确的修法是**把除法去掉**（重复减法；用 `& 127` 加拒绝代替 `% 81`），
+而不是往 4 KiB 的叠加区里链一个软除法。
+
+入口必须在镜像最前面，因为加载器跳到块偏移 0，所以 `app_main` 带着上游自带应用用的同一个
+`__attribute__((section(".text.entry"), used))`（`cube3d_app.c:183`）。
+
+用页面装上去后的实测：固件把槽头读了两次，然后**正好从 槽+0x1000 读了 `code_size` 字节**
+（Minesweeper 的 2408 字节代码就读了 2408 字节，而且这是整份启动日志里唯一一次这种大小的读取）。也就是说：
+块的形状、头、CRC、VMA、偏移**全部被接受**。**但控制权是否真的交到叠加区仍未证实**：PC 探针每 100 ms 采样
+一次、没采到叠加区地址，也没有出现 `APP ERROR` 屏，应用也没有画出来。现状就是如此 —— **管线验证到了
+"加载"这一步，还没到"执行"**。
+
 ## 键盘：两个真 bug，都已修复
 
 这里原来的笔记写的是"按键到达了固件但界面不反应"，并且归咎于机器模型。结果发现有**两个

apps/minesweeper/build.sh

@@ -14,19 +14,35 @@ APP_API_MIN=1
 APP_VMA=${APP_VMA:-0x20000280}
 OUT="${APP_NAME// /}"
 
-CC=${CC:-arm-none-eabi-gcc}
-OBJCOPY=${OBJCOPY:-arm-none-eabi-objcopy}
-command -v "$CC" >/dev/null 2>&1 || {
-  echo "no $CC on PATH; install the Arm GNU Toolchain (arm-none-eabi) first" >&2; exit 2; }
+# Two routes, because only one of them was available where this was written:
+#   1. the Arm GNU Toolchain, which is what upstream's own build.sh expects
+#   2. Zig's bundled clang (pip install ziglang), which cross-compiles to
+#      thumb-freestanding-eabi and needs no arm-none-eabi-gcc at all
+ZIG=""
+if ! CC=${CC:-$(command -v arm-none-eabi-gcc 2>/dev/null)} || [ -z "$CC" ]; then
+  ZIG=${ZIG:-$(command -v zig 2>/dev/null || echo "python -m ziglang")}
+  CC="$ZIG cc"
+  echo "no arm-none-eabi-gcc; using $ZIG cc"
+fi
 
 CFLAGS="-mcpu=cortex-m0plus -mthumb -Os -std=gnu11 -ffreestanding -fno-builtin -fno-common \
   -fomit-frame-pointer -ffunction-sections -fdata-sections -Wall -Wextra"
-LDFLAGS="-nostdlib -nostartfiles -T app.ld -Wl,--defsym,APP_VMA=${APP_VMA} \
-  -Wl,--gc-sections -Wl,-Map=${APP}.map -Wl,--build-id=none"
+# Zig refuses --defsym, -Ttext and --section-start, so the VMA is resolved into a copy of
+# the linker script instead, and the entry is pinned by the section attribute the source
+# already carries (.text.entry, exactly as upstream's apps do).
+sed "s/APP_VMA *= *DEFINED(APP_VMA) *? *APP_VMA *: *0x[0-9A-Fa-f]*;/APP_VMA = ${APP_VMA};/" \
+    app.ld > app-resolved.ld
+LDFLAGS="-nostdlib -T app-resolved.ld -Wl,-e,app_main -Wl,--gc-sections -Wl,--build-id=none"
 
 rm -f ./*.app ./*.elf ./*.bin
-"$CC" $CFLAGS $LDFLAGS -o "${APP}.elf" "${APP}_app.c"
-"$OBJCOPY" -O binary "${APP}.elf" "${APP}.bin"
+# shellcheck disable=SC2086
+$CC $CFLAGS $LDFLAGS -o "${APP}.elf" "${APP}_app.c"
+# objcopy if there is one, otherwise the section-based extractor from this repository
+if command -v arm-none-eabi-objcopy >/dev/null 2>&1; then
+  arm-none-eabi-objcopy -O binary "${APP}.elf" "${APP}.bin"
+else
+  python3 "$(dirname "${BASH_SOURCE[0]}")/../../../tools/elf2bin.py" "${APP}.elf" "${APP}.bin" --min "${APP_VMA}"
+fi
 python3 pack_app.py "${APP}.bin" "${OUT}.app" --name "$APP_NAME" --ver "$APP_VER" \
         --vma "$APP_VMA" --api-min "$APP_API_MIN"
 ls -l "${OUT}.app"

tools/elf2bin.py

@@ -0,0 +1,71 @@
+#!/usr/bin/env python3
+"""Extract the loadable bytes of an overlay-app ELF as a flat binary.
+
+objcopy -O binary is the normal tool; this reads the ELF itself. Allocated *sections*
+are used rather than program headers, because lld maps the ELF header and program
+header table as a LOAD of its own (52 + 4x32 = 180 bytes at the image base) with no
+section content in it -- following the program headers puts a phantom 180-byte region
+below the overlay VMA and makes the image start at the wrong address.
+"""
+import argparse
+import struct
+import sys
+
+SHT_SYMTAB, SHT_NOBITS = 2, 8
+SHF_ALLOC, SHF_WRITE, SHF_EXECINSTR = 0x2, 0x1, 0x4
+
+
+def sections(blob: bytes):
+    if len(blob) < 52 or blob[0] != 0x7F or blob[1:4] != b"ELF" or blob[4] != 1 or blob[5] != 1:
+        raise SystemExit("not a 32-bit little-endian ELF")
+    e_shoff, = struct.unpack_from("<I", blob, 0x20)
+    e_shentsize, e_shnum, _ = struct.unpack_from("<HHH", blob, 0x2E)
+    if not e_shoff or not e_shnum:
+        return None
+    out = []
+    for i in range(e_shnum):
+        f = struct.unpack_from("<IIIIIIIIII", blob, e_shoff + i * e_shentsize)
+        addr, offset, size, flags, typ = f[3], f[4], f[5], f[2], f[1]
+        if typ == SHT_NOBITS:                     # .bss has no file bytes
+            out.append((addr, offset, 0, size, typ))
+        elif flags & SHF_ALLOC and size:
+            out.append((addr, offset, size, size, typ))
+    return out or None
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("elf")
+    ap.add_argument("bin")
+    ap.add_argument("--min", type=lambda v: int(v, 0), default=None,
+                    help="require the image to start here (the overlay VMA)")
+    ap.add_argument("--quiet", action="store_true")
+    args = ap.parse_args(argv)
+
+    blob = open(args.elf, "rb").read()
+    secs = sections(blob)
+    if not secs:
+        raise SystemExit("no allocated sections; nothing to extract")
+    base = min(s[0] for s in secs if s[2])        # the first section with file content
+    if args.min is not None and base != args.min:
+        raise SystemExit("image starts at 0x%08X but the overlay VMA is 0x%08X -- the blob "
+                         "would be rejected with APP_ERR_VMA" % (base, args.min))
+    end = max(addr + memsz for addr, _, _, memsz, _ in secs)
+    out = bytearray(end - base)                   # .bss stays zero, as the loader wants
+    for addr, offset, filesz, memsz, _ in secs:
+        if filesz:
+            out[addr - base:addr - base + filesz] = blob[offset:offset + filesz]
+    open(args.bin, "wb").write(out)
+    name = lambda p: p.replace("\\", "/").split("/")[-1]
+    if not args.quiet:
+        print("%s -> %s: %d bytes at 0x%08X (%d allocated section(s), %d of them with content)"
+              % (name(args.elf), name(args.bin), len(out), base,
+                 len(secs), sum(1 for s in secs if s[2])))
+        if len(out) > 0x1000 and args.min == 0x20000280:
+            print("WARNING: %d bytes exceeds the 4096-byte overlay budget" % len(out), file=sys.stderr)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/test_elf2bin.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+"""The ELF extractor must follow *sections*, not program headers.
+
+lld maps the ELF header and program-header table as a LOAD of its own -- 52 + 4x32 = 180
+bytes at the image base, with no section content in it. An extractor that follows program
+headers therefore puts a phantom 180-byte region below the first real section, and the
+image is refused with APP_ERR_VMA or, worse, built from the wrong base. These tests build
+the ELFs by hand, so they need no toolchain.
+"""
+import struct
+import unittest
+import os
+import tempfile
+
+import elf2bin
+
+
+def build_elf(body=b"\x11" * 64, with_phantom_phdr=False):
+    """A minimal 32-bit little-endian ELF with one allocated section holding @body.
+
+    With @with_phantom_phdr the first program header is a LOAD covering the ELF header and
+    the program-header table itself -- no section content, exactly what lld emits -- which
+    must not become part of the extracted image.
+    """
+    ehsize, phentsize, shentsize = 52, 32, 40
+    phnum = 2 if with_phantom_phdr else 1
+    phoff = ehsize
+    data_off = phoff + phnum * phentsize
+    shoff = data_off + len(body)
+    blob = bytearray(shoff + shentsize)
+    blob[0:4] = b"\x7fELF"
+    blob[4] = 1                                  # 32-bit
+    blob[5] = 1                                  # little endian
+    struct.pack_into("<I", blob, 0x1C, phoff)
+    struct.pack_into("<H", blob, 0x2A, phentsize)
+    struct.pack_into("<H", blob, 0x2C, phnum)
+    struct.pack_into("<I", blob, 0x20, shoff)
+    struct.pack_into("<H", blob, 0x2E, shentsize)
+    struct.pack_into("<H", blob, 0x30, 1)
+    blob[data_off:data_off + len(body)] = body   # the section's file content
+    i = 0
+    if with_phantom_phdr:
+        struct.pack_into("<IIIIIIII", blob, phoff, 1, 0, 0x20000000, 0x20000000,
+                         ehsize + phentsize, 0, 4, 4)
+        i = 1
+    struct.pack_into("<IIIIIIII", blob, phoff + i * phentsize, 1, data_off, 0x20000280,
+                     0x20000280, len(body), len(body), 5, 4)
+    struct.pack_into("<IIIIIIIIII", blob, shoff, 0, 1, 0x6, 0x20000280,
+                     data_off, len(body), 0, 0, 4, 0)
+    return bytes(blob)
+
+
+class TestExtract(unittest.TestCase):
+    def _extract(self, blob, minimum=0x20000280):
+        path = os.path.join(tempfile.mkdtemp(), "in.elf")
+        out = os.path.join(os.path.dirname(path), "out.bin")
+        with open(path, "wb") as fh:
+            fh.write(blob)
+        elf2bin.main([path, out, "--min", hex(minimum), "--quiet"])
+        with open(out, "rb") as fh:
+            return fh.read()
+
+    def test_a_plain_single_section_elf(self):
+        blob = build_elf()
+        self.assertEqual(self._extract(blob), b"\x11" * 64)
+
+    def test_the_phantom_header_segment_is_not_part_of_the_image(self):
+        """This is the case that broke the first version: 0x20000000 is not the base."""
+        blob = build_elf(with_phantom_phdr=True)
+        self.assertEqual(self._extract(bytes(blob)), b"\x11" * 64)
+
+    def test_a_wrong_base_is_refused_rather_than_silently_shifted(self):
+        blob = build_elf()
+        with self.assertRaises(SystemExit):
+            self._extract(bytes(blob), minimum=0x20000000)
+
+    def test_something_that_is_not_an_elf_is_refused(self):
+        with self.assertRaises(SystemExit):
+            self._extract(b"not an elf at all" + b"\x00" * 64)
+
+
+if __name__ == "__main__":
+    unittest.main()