From f938d0b3bd170ebbab7ad3be2e48042df2ab1522 Mon Sep 17 00:00:00 2001 From: QIU SHENGMING Date: Fri, 2 Oct 2026 08:32:22 +0800 Subject: [PATCH] Build overlay apps without the Arm toolchain, and say how far verification got pip install ziglang cross-compiles to thumb-freestanding-eabi, which is enough to build a .app with no arm-none-eabi-gcc and no Docker. Measured refusals: --defsym, -Ttext and --section-start come back as unsupported linker args, so the VMA is resolved into a copy of app.ld; -T is forwarded (a missing script errors); --image-base is accepted but page-aligns the segments into 0x200103C8 and 0x20020C94. tools/elf2bin.py extracts allocated sections rather than program headers, because lld maps the ELF header and phdr table as a 180-byte LOAD of its own -- following the headers starts the image at 0x20000000 and APP_ERR_VMA. One division pulled in __aeabi_uidiv, which a -nostdlib blob cannot have: Minesweeper now avoids division entirely. app_main carries the .text.entry attribute upstream's apps use, so the entry is first for the loader's jump to offset 0. Minesweeper builds to 2408 bytes of code against a 4096-byte budget. Installed through the page, the firmware reads the slot header twice and then exactly code_size bytes from slot+0x1000 -- the only read of that size in the boot log -- so the blob shape, header, CRC, VMA and offset are all accepted. Whether control reaches the overlay is unproven: the 100 ms PC probe saw no overlay address, no APP ERROR screen appears, and the app does not draw. test_elf2bin pins the phantom-header-segment lesson; both AGENTS files record the rest. --- AGENTS.md | 27 +++++++++++++ AGENTS.zh-CN.md | 21 ++++++++++ apps/minesweeper/build.sh | 32 +++++++++++---- tools/elf2bin.py | 71 +++++++++++++++++++++++++++++++++ tools/test_elf2bin.py | 83 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 226 insertions(+), 8 deletions(-) create mode 100644 tools/elf2bin.py create mode 100644 tools/test_elf2bin.py diff --git a/AGENTS.md b/AGENTS.md index a4cb58c..1b8b13b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -661,6 +661,33 @@ written back by its own write-back anyway. Rolling the image back is what worked its in-memory image back on exit, so an edit made while a guest was live can be overwritten by the copy that guest was holding -- which is also how the marker reappeared after being cleared. +**Building an overlay app without the Arm toolchain: `pip install ziglang`.** There is no +`arm-none-eabi-gcc` and no Docker on the machine this was written on, but Zig ships a C compiler +that cross-compiles to `thumb-freestanding-eabi`, which is enough. What Zig refuses, each measured: +`--defsym`, `-Ttext`, `--section-start` (its linker-argument whitelist) -- so the VMA is resolved +into a copy of `app.ld` with `sed` instead; `-T` *is* forwarded, because a nonexistent script makes +it error; and `--image-base` is accepted but useless here, because it page-aligns the segments +(0x20000280 becomes LOADs at 0x200103C8 and 0x20020C94). + +Two traps cost the most time. **lld maps the ELF header and program-header table as a LOAD of its +own** -- 52 + 4x32 = 180 bytes at the image base with no section content -- so `tools/elf2bin.py` +extracts *allocated sections* rather than program headers; following the program headers starts the +image at 0x20000000 and the loader refuses it with APP_ERR_VMA. And **one division pulled in +`__aeabi_uidiv`**, which does not exist in a `-nostdlib` blob: the fix is to remove the divisions +(repeated subtraction; `& 127` with a reject instead of `% 81`), not to link a soft-divide routine +into a 4 KiB overlay. + +The entry must be first in the image, because the loader jumps to blob offset 0, so `app_main` +carries the same `__attribute__((section(".text.entry"), used))` that upstream's apps use +(`cube3d_app.c:183`). + +Measured with the result installed through the page: the firmware reads the slot header twice and then +**exactly `code_size` bytes from slot + 0x1000** (2408 for Minesweeper's 2408-byte code, and it is +the only read of that size in the whole boot log). So the blob's shape, its header, the CRC, the VMA +and the offset are all accepted. **Whether control then reaches the overlay is still unproven**: the PC +probe samples every 100 ms and saw no overlay address, no `APP ERROR` screen appears, and the app does +not draw. That is where this stands -- the pipeline is verified up to the load, not up to execution. + ## The keypad: two real bugs, both fixed The old note here said "keys reach the firmware but the UI does not react" and diff --git a/AGENTS.zh-CN.md b/AGENTS.zh-CN.md index ce7def4..f762e06 100644 --- a/AGENTS.zh-CN.md +++ b/AGENTS.zh-CN.md @@ -535,6 +535,27 @@ app 区被读过 —— 而多系统菜单读的是**固件槽**，不是应用 **而当"改了不生效"时，过期的写回依然值得怀疑**：模拟器退出时会把内存里的镜像写回文件，所以客人还活着 时做的改动，可能被它手里那份副本覆盖 —— 这也是标记被清掉后又重新出现的原因。 +**没有 Arm 工具链也能编叠加应用：`pip install ziglang`。** 写它的机器上既没有 `arm-none-eabi-gcc` +也没有 Docker，但 Zig 自带 C 编译器、能交叉到 `thumb-freestanding-eabi`，够用了。Zig 会拒绝什么（都实测过）： +`--defsym`、`-Ttext`、`--section-start`（被它的链接参数白名单挡掉）；所以 VMA 用 `sed` 写进一份 +`app.ld` 的副本；`-T` **是**会被转发的（给一个不存在的脚本它会报错）；`--image-base` 虽然接受但没有用， +因为它把段按页对齐（0x20000280 变成 0x200103C8 与 0x20020C94 两个 LOAD）。 + +最花时间的是两个坑。**lld 会把 ELF 头与程序头表本身也映射成一个 LOAD** —— 52 + 4×32 = 180 字节、位于镜像 +基址、里面没有任何节内容 —— 所以 `tools/elf2bin.py` **按已分配节提取，而不是按程序头**；跟着程序头走会让 +镜像从 0x20000000 开始，加载器以 APP_ERR_VMA 拒绝。另一个是**一次除法就会牵出 `__aeabi_uidiv`**，而它在 +`-nostdlib` 的块里并不存在 —— 正确的修法是**把除法去掉**（重复减法；用 `& 127` 加拒绝代替 `% 81`）， +而不是往 4 KiB 的叠加区里链一个软除法。 + +入口必须在镜像最前面，因为加载器跳到块偏移 0，所以 `app_main` 带着上游自带应用用的同一个 +`__attribute__((section(".text.entry"), used))`（`cube3d_app.c:183`）。 + +用页面装上去后的实测：固件把槽头读了两次，然后**正好从 槽+0x1000 读了 `code_size` 字节** +（Minesweeper 的 2408 字节代码就读了 2408 字节，而且这是整份启动日志里唯一一次这种大小的读取）。也就是说： +块的形状、头、CRC、VMA、偏移**全部被接受**。**但控制权是否真的交到叠加区仍未证实**：PC 探针每 100 ms 采样 +一次、没采到叠加区地址，也没有出现 `APP ERROR` 屏，应用也没有画出来。现状就是如此 —— **管线验证到了 +"加载"这一步，还没到"执行"**。 + ## 键盘：两个真 bug，都已修复 这里原来的笔记写的是"按键到达了固件但界面不反应"，并且归咎于机器模型。结果发现有**两个 diff --git a/apps/minesweeper/build.sh b/apps/minesweeper/build.sh index 6d58690..a7dd24d 100644 --- a/apps/minesweeper/build.sh +++ b/apps/minesweeper/build.sh @@ -14,19 +14,35 @@ APP_API_MIN=1 APP_VMA=${APP_VMA:-0x20000280} OUT="${APP_NAME// /}" -CC=${CC:-arm-none-eabi-gcc} -OBJCOPY=${OBJCOPY:-arm-none-eabi-objcopy} -command -v "$CC" >/dev/null 2>&1 || { - echo "no $CC on PATH; install the Arm GNU Toolchain (arm-none-eabi) first" >&2; exit 2; } +# Two routes, because only one of them was available where this was written: +# 1. the Arm GNU Toolchain, which is what upstream's own build.sh expects +# 2. Zig's bundled clang (pip install ziglang), which cross-compiles to +# thumb-freestanding-eabi and needs no arm-none-eabi-gcc at all +ZIG="" +if ! CC=${CC:-$(command -v arm-none-eabi-gcc 2>/dev/null)} || [ -z "$CC" ]; then + ZIG=${ZIG:-$(command -v zig 2>/dev/null || echo "python -m ziglang")} + CC="$ZIG cc" + echo "no arm-none-eabi-gcc; using $ZIG cc" +fi CFLAGS="-mcpu=cortex-m0plus -mthumb -Os -std=gnu11 -ffreestanding -fno-builtin -fno-common \ -fomit-frame-pointer -ffunction-sections -fdata-sections -Wall -Wextra" -LDFLAGS="-nostdlib -nostartfiles -T app.ld -Wl,--defsym,APP_VMA=${APP_VMA} \ - -Wl,--gc-sections -Wl,-Map=${APP}.map -Wl,--build-id=none" +# Zig refuses --defsym, -Ttext and --section-start, so the VMA is resolved into a copy of +# the linker script instead, and the entry is pinned by the section attribute the source +# already carries (.text.entry, exactly as upstream's apps do). +sed "s/APP_VMA *= *DEFINED(APP_VMA) *? *APP_VMA *: *0x[0-9A-Fa-f]*;/APP_VMA = ${APP_VMA};/" \ + app.ld > app-resolved.ld +LDFLAGS="-nostdlib -T app-resolved.ld -Wl,-e,app_main -Wl,--gc-sections -Wl,--build-id=none" rm -f ./*.app ./*.elf ./*.bin -"$CC" $CFLAGS $LDFLAGS -o "${APP}.elf" "${APP}_app.c" -"$OBJCOPY" -O binary "${APP}.elf" "${APP}.bin" +# shellcheck disable=SC2086 +$CC $CFLAGS $LDFLAGS -o "${APP}.elf" "${APP}_app.c" +# objcopy if there is one, otherwise the section-based extractor from this repository +if command -v arm-none-eabi-objcopy >/dev/null 2>&1; then + arm-none-eabi-objcopy -O binary "${APP}.elf" "${APP}.bin" +else + python3 "$(dirname "${BASH_SOURCE[0]}")/../../../tools/elf2bin.py" "${APP}.elf" "${APP}.bin" --min "${APP_VMA}" +fi python3 pack_app.py "${APP}.bin" "${OUT}.app" --name "$APP_NAME" --ver "$APP_VER" \ --vma "$APP_VMA" --api-min "$APP_API_MIN" ls -l "${OUT}.app" diff --git a/tools/elf2bin.py b/tools/elf2bin.py new file mode 100644 index 0000000..69a019f --- /dev/null +++ b/tools/elf2bin.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python3 +"""Extract the loadable bytes of an overlay-app ELF as a flat binary. + +objcopy -O binary is the normal tool; this reads the ELF itself. Allocated *sections* +are used rather than program headers, because lld maps the ELF header and program +header table as a LOAD of its own (52 + 4x32 = 180 bytes at the image base) with no +section content in it -- following the program headers puts a phantom 180-byte region +below the overlay VMA and makes the image start at the wrong address. +""" +import argparse +import struct +import sys + +SHT_SYMTAB, SHT_NOBITS = 2, 8 +SHF_ALLOC, SHF_WRITE, SHF_EXECINSTR = 0x2, 0x1, 0x4 + + +def sections(blob: bytes): + if len(blob) < 52 or blob[0] != 0x7F or blob[1:4] != b"ELF" or blob[4] != 1 or blob[5] != 1: + raise SystemExit("not a 32-bit little-endian ELF") + e_shoff, = struct.unpack_from(" %s: %d bytes at 0x%08X (%d allocated section(s), %d of them with content)" + % (name(args.elf), name(args.bin), len(out), base, + len(secs), sum(1 for s in secs if s[2]))) + if len(out) > 0x1000 and args.min == 0x20000280: + print("WARNING: %d bytes exceeds the 4096-byte overlay budget" % len(out), file=sys.stderr) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/test_elf2bin.py b/tools/test_elf2bin.py new file mode 100644 index 0000000..8bc270a --- /dev/null +++ b/tools/test_elf2bin.py @@ -0,0 +1,83 @@ +#!/usr/bin/env python3 +"""The ELF extractor must follow *sections*, not program headers. + +lld maps the ELF header and program-header table as a LOAD of its own -- 52 + 4x32 = 180 +bytes at the image base, with no section content in it. An extractor that follows program +headers therefore puts a phantom 180-byte region below the first real section, and the +image is refused with APP_ERR_VMA or, worse, built from the wrong base. These tests build +the ELFs by hand, so they need no toolchain. +""" +import struct +import unittest +import os +import tempfile + +import elf2bin + + +def build_elf(body=b"\x11" * 64, with_phantom_phdr=False): + """A minimal 32-bit little-endian ELF with one allocated section holding @body. + + With @with_phantom_phdr the first program header is a LOAD covering the ELF header and + the program-header table itself -- no section content, exactly what lld emits -- which + must not become part of the extracted image. + """ + ehsize, phentsize, shentsize = 52, 32, 40 + phnum = 2 if with_phantom_phdr else 1 + phoff = ehsize + data_off = phoff + phnum * phentsize + shoff = data_off + len(body) + blob = bytearray(shoff + shentsize) + blob[0:4] = b"\x7fELF" + blob[4] = 1 # 32-bit + blob[5] = 1 # little endian + struct.pack_into("