A capability-based microkernel written in Rust, targeting AArch64 (ARMv8-A). Runs on QEMU virt machine and boots end-to-end on a real Raspberry Pi 4B from the same binary — DTB-driven platform discovery, GICv2 + spin-table SMP, kernel image linked at a fixed higher-half VA with the load physical address discovered at runtime. Inspired by seL4 and Zircon, but with its own object model.

Lockjaw is part pet project, part experiment, part tutorial. 100% of the source code in this repository was written by Claude (Anthropic's AI) under direction from Ben. Ben writes prompts, plans, design feedback, and review verdicts; Claude writes the code. The project is staged to demonstrate three claims:

1. Claude can write OS-level systems software. Page fault handlers, exception vectors, context-switch assembly, MMU bring-up, IPC state machines, DMA cache-coherence envelopes, the SDHCI driver that reads from a real Pi 4B SD card — the things that have historically demanded years of low-level expertise to get right. The commit log is the proof.
2. An OS can be written in Rust with minimal hand-written asm! and minimal unsafe. The whole kernel binary contains a handful of audited assembly compound transactions (boot trampoline, MMU enable, higher-half pivot, EL0 drop, exception save/restore, context switch) and a handful of audited unsafe helpers (cache maintenance, GIC IAR read, the MMIO atomic-writer transmute, the kernel-image relink machinery). Adding a new audited site fails the build — xtask AST scanners (check-driver-unsafe, check-kernel-driver-unsafe, check-kernel-alloc, check-console-chokepoint) enforce the surface by construction. Everything else is safe Rust, much of it generated from TOML specs.
3. A single human paired with an AI can build systems software at a scope that historically required a sizable team and a multi-year timeline. SMP across 4 CPUs, real Pi 4B hardware, userspace musl POSIX personality, FAT32 mounted over an eMMC2 ADMA2 DMA path, 17 userspace processes, 1308 unit tests + 424 integration assertions across four QEMU permutations. None of this is in a research-prototype state. The same binary boots QEMU and the Pi 4B from the same make invocation, and the test suite runs both.

The project doubles as a tutorial: every commit is small and atomic, the development journals (docs/journals/) record design rationale in the paired-review workflow shape used throughout, and the architecture book (docs/architecture/) explains the substrate-debt and push/pull/plan-apply patterns that keep this Rust kernel reviewable by one person.

Lockjaw is a from-scratch microkernel that explores a middle ground between seL4's rigorous user-controlled memory model and Zircon's pragmatic handle-based API. The kernel never dynamically allocates memory. Userspace requests physical pages, then either maps them for its own use or donates them to the kernel to create objects like threads, IPC endpoints, and handle tables.

The design follows a few core principles:

• Kernel never allocates. All object memory comes from user-donated pages (PageSets). The kernel has only a fixed-size boot region in BSS.
• Handle-based access control. Every kernel object is accessed through an integer handle with an associated rights bitmask. No handle, no access.
• Vulkan-inspired create-info pattern. Each object type has its own create-info struct used for both size queries and creation. Same struct, no mismatch.
• Proven stack safety. A custom build tool analyzes the call graph and per-function stack sizes from four entry points (_start, _secondary_start, __vec_sync_lower, __vec_irq) on every build. Indirect calls must be annotated or the build fails.
• Map or donate, never both. A PageSet is consumed when donated for a kernel object. Consume is a transactional two-phase operation (validate + apply) that walks every live process's handle table, clears stale cross-process exported handles, and frees the header — no tombstones, no leaks.
• Verified IPC state machine. The IPC endpoint logic is driven by a pure state machine model that is exhaustively explored at test time -- all reachable states, all transitions, all effect orderings verified. Kernel IPC handlers match on typed decision enums (SendDecision, ReceiveDecision, CallDecision, ReplyDecision) returned by lockjaw-types. No inline state branching in kernel code.
• Pull over push. Kernel code is organized by integration shape: pull (types drives sequencing), plan/apply (types returns a decision, kernel executes), or push (kernel calls helpers). Push is treated as highest review-risk; the extraction rubric converts push to pull wherever possible.
• All MMIO through the device manager. Drivers cannot map arbitrary physical addresses. The device manager discovers hardware from the DTB and issues tracked PageSets for MMIO pages. Only processes that receive an MMIO PageSet can map device memory.
• Unforgeable caller identity. IPC endpoints carry kernel-assigned opaque caller tokens. When a handle is exported, the kernel assigns a monotonic per-endpoint token stored in the handle entry. Servers query the token after receive to scope resources per-client. Tokens identify handle lineage, not processes — delegates inherit the original token. Token 0 is receive-only; send/call with token 0 is rejected by the kernel.
• Decoupled link VA from load PA. The kernel image is linked at a fixed higher-half VA in its own L0[1] region (0xFFFF_0080_0000_0000), independent of the physical address firmware loads it at. The boot trampoline discovers the actual load PA via PC-relative (adr _start), computes a per-boot phys offset, and maps the runtime PA range at the linker VA via 4 KB L3 PTEs. Same binary boots on QEMU (load PA 0x40200000) and Pi 4B (load PA 0x80000); neither linker ORIGIN nor any code path needs adjustment. Userspace TTBR0 carries no kernel entries — no kernel identity, no device MMIO. The only PA-aware input is the DTB.
• Typed VA regimes. Three sibling newtypes — PhysAddr for physical memory, KernelVa for the KVM allocator pool (0xFFFF_8000_0000_0000, where typed kernel objects live), KernelImageVa for the kernel image region (0xFFFF_0080_0000_0000). compile_fail doctests prove they cannot be assigned across regimes. Per-thread kernel stack base is a typed KernelStackBase::{Image(KernelImageVa), Pool(KernelVa)} enum so finish_exit's free-path choice is match-driven and the wrong path is unrepresentable.
• TOML specs generate typed register and wire DTOs — the spec IS the source of truth. Every MMIO register layout (PL011, SDHCI, GIC distributor/CPU/redistributor, virtio-mmio, fw_cfg, cprman), every AArch64 system register touched by the kernel (TTBR0/1, SCTLR, MAIR, TCR, ICC_, CNTV_, cache identifiers), and every device-protocol wire DTO (virtio descriptors, fw_cfg DMA control blocks, ramfb config) is declared in regspecs/*.toml / sysregspecs/*.toml / wirespecs/*.toml and codegen'd into typed accessors at build time. gen-regs --check / gen-sysregs --check / gen-wires --check run before every build and fail if a generated file drifts from its spec. Codegen emits compile_fail doctest tripwires next to each compound MMIO read/write so demoting the contract from compound back to the safe default breaks host cargo test --doc. Drivers and kernel device files cannot hand-pack register bits, hand-sequence the read/write barriers around a compound transaction, or hand-byteswap a wire DTO. Adding a new device starts by writing a TOML spec, not by writing read_volatile/write_volatile against magic offsets.

Lockjaw boots on QEMU and Raspberry Pi 4B with up to 4 cores, manages virtual memory with a buddy allocator supporting contiguous DMA allocation, handles interrupts, runs preemptively scheduled threads across multiple CPUs with a Giant Kernel Lock, serves 32 syscalls from EL0 userspace, passes messages between threads via synchronous IPC with Reply objects and kernel-assigned caller tokens for multi-client isolation, and runs seventeen isolated userspace processes loaded from ELF binaries spanning the init / personality / driver / test surface: init + hello + posix-server + posix-hello (musl-built fopen + malloc + stdio client), device-manager + the five drivers it provisions MMIO for (pl011, ramfb, virtio-blk, cprman, emmc2), partition-manager + fat32-server + fat32-test (full block stack), and the test clients display-test / clock-test / sleep-test / neon-canary. Userspace touches hardware through a typed driver substrate (regspec-generated MMIO accessors, wirespec-generated DTOs, run_dma_transfer cache-coherence envelope, #![deny(unsafe_code)] at every driver root). The kernel touches hardware through a parallel typed substrate (lockjaw-regs MMIO + lockjaw-sysregs DSL + audited helpers for the three remaining asm! compound transactions). On Pi 4B the eMMC2/cprman path reads from the real SD card through the full stack: [BLOCKDEV] /dev/sd0 ready → fat32: mounted → [FAT32-TEST] read 17 bytes: hello from fat32 → posix-hello: hello from fat32. All boot diagnostic output routes through a single CONSOLE_WRITER chokepoint whose installed value is regime-stable across the MMU pivot and the drop to EL0.

=== Lockjaw Microkernel v0.1.0 ===
Target: AArch64 (ARMv8-A)
Platform: UART=0xfe201000 GICD=0xff841000 GICv2 RAM=0x0+0x3b400000   # ← Pi 4B; QEMU prints GICv3
Enabling MMU (identity map)...
MMU enabled — UART still working!
Enabling higher-half kernel mapping...
Higher-half active — UART at 0xffff0000fe201000
[SMP] CPU 1 released (spin-table)
[SMP] CPU 2 released (spin-table)
[SMP] CPU 3 released (spin-table)
Pivoted to higher-half (TTBR1).
CPU 0 initialized (TPIDR_EL1)
GIC initialized, timer PPI 27 enabled
Scheduler started.
Loading init process...
Dropping to EL0...
Hello from userspace init!
init: hello spawned OK
init: device-manager spawned OK
init: cprman-driver spawned OK
init: pl011-driver spawned OK
init: ramfb-driver spawned OK
init: blk-driver spawned OK
init: fat32-server spawned OK
init: posix-server spawned OK
init: emmc2-driver spawned OK
init: partition-manager spawned OK
devmgr: parsed DTB, 119 devices
[CPRMAN] EMMC2 set_rate(200_000_000) -> actual=200000000 enabled=1
[EMMC2:READY] rca=0xaaaa capacity=119GiB bus=4bit card_clk=25MHz   # ← real SD card on Pi 4B
[BLOCKDEV] /dev/sd0 ready: 512B x 249737216 blocks; selftest read OK
partmgr: MBR FAT32 partition found
fat32: mounted, cluster_size=32768 bytes, root_cluster=2
posix-server: POSIX_INIT OK
hello, lockjaw                          # ← from a real musl-built static binary
[FAT32-TEST] read 17 bytes: hello from fat32
posix-hello: hello from fat32           # ← fopen + fread on /HELLO.TXT via FAT32 IPC
posix-hello: malloc 1MB ok              # ← musl malloc -> mmap -> server mmap_table
posix-hello: malloc 8MB ok              # ← single-PageSet 64 MiB-capable mmap
[NEON-CANARY] PASS                      # ← per-CPU NEON state isolation verified

Phase 1 -- Boot to UART. Bare-metal Rust binary boots on QEMU virt, prints to PL011 UART via MMIO, has a formatted kprintln! macro and a panic handler that prints file/line/message.

Phase 2 -- Memory Management. Buddy allocator over 128 MB of RAM (32,768 pages) with contiguous multi-page allocation for DMA buffers. AArch64 4-level page tables with identity mapping, then higher-half kernel mapping via TTBR1 (kernel at 0xFFFF_0000_xxxx_xxxx). Unmapped guard page below the kernel stack with a canary value checked on every context switch.

Phase 3 -- Exceptions and Interrupts. Exception vector table with full register save/restore (31 GPRs + ELR/SPSR/ESR). GICv3 interrupt controller initialization. Virtual timer firing every 10ms for preemptive scheduling. Structured crash diagnostics: ESR decode, address classification, stack overflow detection, thread ID, syscall breadcrumb.

Phase 4 -- Kernel Object Model. Typed kernel objects created in user-donated pages via the Vulkan-style create-info pattern (query size, allocate PageSet, donate, create). Handle tables with insert/lookup/remove and rights checking (Read, Write, Grant). PageSets consumed on donation to prevent reuse.

Phase 5 -- Threads and Context Switching. Thread Control Blocks with per-thread stacks. Assembly context_switch saves/restores callee-saved registers (SavedContext struct with compile-time offset assertions against the assembly) and swaps SP. Round-robin scheduler driven by the timer interrupt. Preemptive multithreading verified with interleaved output from concurrent threads.

Phase 6 -- Syscall Interface. Userspace code runs at EL0 (unprivileged). SVC traps to kernel via separate lower-EL exception vector. Syscall dispatch on x8 register. Typed error returns: x0 = SyscallError (always), x1 = value, x1-x4 = IPC message words. User page tables in TTBR0 with PXN/UXN security bits.

Phase 7 -- IPC. Synchronous rendezvous message passing through Endpoint objects. Four message registers (x1-x4) transferred between threads. Send/receive with blocking, call/reply for client/server patterns using per-client Reply objects (eliminates multi-caller corruption). Non-blocking receive. Multiplexed wait (sys_wait_any) with threshold-based readiness. IPC state machine exhaustively verified: 89 reachable states (3-thread model), 8 invariants checked. The kernel's IPC is driven entirely by the verified model.

Phase 8 -- Userspace Processes. Per-process TTBR0 page tables swapped by the scheduler on context switch. ELF64 parser loads the init process from an embedded binary. Init runs at EL0 and spawns child processes entirely from userspace. Bootstrap channel protocol (Zircon-inspired): child calls handle 0, parent exports handles via sys_export_handle, replies with indices.

Phase 9 -- Userspace Drivers. UART driver runs entirely in userspace. Receives its server endpoint and device-manager endpoint via bootstrap. Event loop using sys_wait_any multiplexes IPC requests and hardware interrupts. Notification objects serve as timeline semaphores for IRQ delivery. Init prints messages through the UART driver via IPC.

Phase 10 -- Device Manager and Display Driver. Device manager process parses the Flattened Device Tree (DTB) at boot to discover hardware. Serves CMD_CLAIM_DEVICE, CMD_PROBE_DEVICE (with explicit status codes: PROBE_OK/END/CLAIMED/ERR), and CMD_CLAIM_BY_ADDR (TOCTOU-safe claim by stable MMIO address) requests from drivers via IPC. Probe uses absolute indexing over all matching DTB nodes for stable concurrent enumeration. Creates tracked MMIO PageSets with sub-page offset support (multiple virtio-mmio devices share a 4K page). ramfb display driver claims fw_cfg from the device manager, allocates a contiguous DMA framebuffer, configures the display via the fw_cfg DMA protocol, and renders a test pattern.

Phase 11 -- SMP. Secondary CPUs booted via PSCI CPU_ON. Per-CPU stacks in the linker script (2MB-aligned, 4 guard+stack pairs). Per-CPU data via TPIDR_EL1 with narrow accessors. Giant Kernel Lock (ticket lock from lockjaw-types, host-testable with multi-threaded tests) serializes all kernel execution. Scheduler model adapted for per-CPU current threads. Exception handlers acquire/release GKL. Kernel threads run cooperatively under the GKL with IRQs masked. Idle threads release GKL and halt in wfi. Process entry releases GKL before eret to EL0. INTID 0 reserved for future cross-core reschedule SGI (parked until fine-grained locking).

Phase 12 -- PageSet Lifecycle. Mapping tracking, ownership transfer with ProcessTransferPlan (deduplication), refcounting with free-on-zero, process exit cleanup via ProcessTeardownPlan with construction-safe narrowing (separate step variants for with/without address space, making illegal unmap-during-teardown unrepresentable). Cross-process handle revocation: consume_pageset is now a transactional two-phase (validate + apply) operation that walks every live process's handle table, clears stale exported handles, decrements per-handle refcount/map_count, and frees the header — replacing the previous tombstone-leak pattern. sys_create_process restructured to push every fallible step into the validate phase (scheduler::has_room precheck, parent_handle_to_copy validation, consume_validate per header) so the apply phase cannot fail mid-stream. Variable-size PageSetHeader: 16-byte fixed metadata followed by an inline u64 array spanning multiple physically-contiguous header pages. Page-addr access is gated by a BackedHeader<'a> wrapper that carries trusted (count, backing_pages) witnesses from the global PageSetTable rather than from the on-disk header itself, so a corrupted header cannot silently truncate or extend operations. Lifts the previous 510-pages-per-set cap to a practical 64 MiB.

Phase 13 -- Caller Tokens. HandleEntry redesigned with typed HandleKind enum (repr(C, u8) with per-type metadata: caller_token on Endpoint, mapped_va_page on PageSet). Kernel assigns monotonic u64 tokens per endpoint on sys_export_handle and create_process handle copy. Token 0 = receive-only; send/call with token 0 is rejected. Servers query tokens via SYS_QUERY_CALLER_TOKEN (syscall 26). Tokens identify handle lineage for capability delegation. Integration test verifies nonzero token delivery.

Phase 14 -- VirtIO Block Driver. VirtIO MMIO transport with modern (non-legacy) device support. Pure types in lockjaw-types (register offsets, virtqueue layout calculator, feature negotiation model, block request types). Virtqueue runtime in userlib with volatile access and AArch64 memory barriers (dmb ishst/ish/ishld). BlockEngine trait + run_block_server() framework (same pattern as display DDI). Per-device GIC trigger mode (sys_bind_irq flags parameter). Device-manager probe protocol with explicit status codes (PROBE_OK/END/CLAIMED/ERR). Sub-page MMIO offset for virtio-mmio devices (8 per 4K page). Driver selftest reads sector 0 and prints content.

Phase 15 -- Real Hardware Portability (Raspberry Pi 4B). Boot path made portable to real AArch64 hardware. Firmware DTB pointer preserved from x0 at entry. Lightweight FDT platform scanner runs early in boot to discover RAM base/size, UART/GIC/timer MMIO addresses, GIC version (v2 vs v3), and SMP boot method (PSCI vs spin-table) from the device tree. All hardcoded MMIO addresses removed — platform consumers wired to DTB-discovered values. GIC split into v2 and v3 drivers with runtime enum dispatch (Pi 4B uses GICv2; QEMU virt uses GICv3). Position-independent boot with a higher-half pivot — kernel can load at any physical address; __kernel_start linker symbol replaces hardcoded KERNEL_LOAD_ADDR. DTB-driven SMP boot: PSCI/HVC for QEMU, spin-table (write entry to cpu-release-addr, dsb, sev) for Pi 4B. BuddyAllocator capacity bumped from 32 K pages (128 MB) to 262 K pages (1 GB) for real-hardware memory sizes. core::fmt replaced with a custom print module — 12% .text savings, removes a class of vtable function pointers in .rodata that real-hardware secure boot pipelines would have to allow-list. New xtask check-vtables build check scans .rodata/.data for absolute code pointers and fails the build on unauthorized ones (with an allow-list for legitimate cases like compiler jump tables). FDT parser hardened against real-hardware DTB layouts. make pi4 produces kernel8.img ready to copy to a Pi 4B SD card boot partition.

Phase 16 -- POSIX Personality (Phases 0-2). Real musl programs allocate memory and read files end-to-end on Lockjaw. Personality server (user/posix-server/) bootstraps with init, parses an embedded ELF, builds the Linux initial stack (argc/argv/auxv with AT_PAGESZ + AT_RANDOM), spawns the child via sys_create_process with a syscall endpoint as handle 0, and dispatches Linux syscalls received over IPC. Three musl patches in musl-lockjaw/: crt_arch.h (SP adjustment), syscall_arch.h (SVC redirect), and shim.c (per-syscall dispatch + bootstrap handshake + local brk handling + per-process mmap tracker, with fail-fast lj_die() for any transport or bootstrap error). Shared-buffer IPC (one page per client) with the asymmetric Lockjaw reply ABI (messages in x2-x5, reply in x1-x4) used correctly. Real ELF loader handles unaligned LOAD segments. Implemented:

• Phase 0 (puts via shared buffer): puts("hello, lockjaw") from a statically-linked patched-musl binary. write, writev, exit_group, set_tid_address (stub), ioctl (stub), brk (local).
• Phase 1 (filesystem): openat / read / close route through posix-server to a FAT32 filesystem server (user/fat32-server/) over a shared-buffer FS-IPC protocol (open/read/close request/reply messages). Per-client OpenTable in fat32-server scoped by caller_token; per-handle DMA buffer PageSet exported to posix-server. fat32-server uses a BlockEngine-shaped BlockClient to talk to the virtio-blk driver. The FsClient + FdTable infrastructure on the posix-server side mirrors the FdTable shape (caller_token isolation, per-fd resource tracking, deferred-close queue for transport-failure rollback).
• Phase 2 (mmap + stdio): musl's malloc above the brk threshold goes through mmap(NULL, len, RW, MAP_PRIVATE|MAP_ANONYMOUS). Personality server's per-client mmap_table allocates a PageSet, picks a base_va from a bump VA allocator, exports the handle to the client. Shim's failure-ordered handshake (NR_MMAP IPC -> sys_map_pages -> tracker insert) with explicit NR_MMAP_ROLLBACK if any post-export step fails. Variable-size PageSet header (Phase 2.K, see Phase 12) lets one PageSet back up to 64 MiB. Multi-L2 page-table mapping (Phase 2.M) lets a single mapping span multiple L2 regions transactionally (classify, pre-allocate, apply — same shape as consume_pageset_validate/apply). 8 MiB malloc gate verified end-to-end. fopen + fread + fclose exercises Phase 1 through musl stdio (which mallocs the FILE struct via mmap), proving Phase 2's mmap-backed malloc supports stdio.
• Phase 3+ (filesystem write, threads via futex, processes via posix_spawn, pipes, signals) still aspirational.

Phase 17 -- Handle Revocation. Two-phase consume_pageset (validate + apply) walks every live process's handle table, clears stale exported handles, replaces tombstone-leak pattern. sys_create_process restructured to push every fallible step into the validate phase.

Phase 18 -- Kernel Objects to KVA. Every typed kernel object (Endpoint, Notification, Reply, ProcessObject, HandleTable, TCB, per-thread kernel stack) migrated from page_alloc::alloc_page() + KernelMut::<T>::from_paddr(...) (the linear higher-half map) to kvm::alloc_kernel_pages(N) + KernelMut::<T>::from_kva(...) (a dedicated KVM pool at L0[256]). Each HandleKind::Foo { paddr } variant flipped to HandleKind::Foo { kva }. Distinct OwnedKvmRange / MappedKvmRange types make the wrong free path a compile error. Surfaced and fixed a latent POSIX MAP_ANONYMOUS contract bug — pageset_table::alloc_pages was returning user-mmap'd frames non-zero, which mallocng's slot-header validation crashed on once the migration shifted which physical frames the buddy hands out at user-mmap time. After the migration, no typed kernel struct is addressed through +KERNEL_VA_OFFSET arithmetic; the type system enforces it.

Phase 19 -- Kernel Image Relink + Pi 4B Bring-Up Validation. Kernel image relinked at a fixed higher-half VA in a dedicated L0[1] region (0xFFFF_0080_0000_0000), decoupled from the physical load address. Boot trampoline discovers actual load PA via PC-relative (adr _start) and computes KERNEL_PHYS_OFFSET = load_PA - LINKER_BASE. init_kernel_image_map walks every kernel image page and writes 4 KB L3 PTEs mapping load_PA + offset → LINKER_BASE + offset (4 KB granule chosen specifically so any load-PA alignment works, including Pi 4B's 0x80000). Pivot uses the boot-discovered shift instead of a constant KERNEL_VA_OFFSET. New KernelImageVa newtype keeps the kernel-image VA regime distinct from the KVM pool's KernelVa. New KernelStackBase enum (Image / Pool variants) makes the kernel-stack regime explicit at the type level so finish_exit's free-path choice cannot regress. New xtask check-linker-symbols enforces an audit doc listing every linker-symbol-to-integer site with classification. Userspace TTBR0 no longer carries the kernel identity map (L1[1] kernel RAM block + L2[4] device MMIO are gone — userspace device drivers still get MMIO via the normal sys_map_pages + MAP_FLAG_DEVICE path). Same binary boots end-to-end on QEMU virt and a real Pi 4B (firmware relocates to PA 0x80000); the only PA-aware input is the DTB.

Phase 20 -- Typed MMIO Substrate + Driver Regime by Construction. Drivers no longer hand-pack register bits, hand-byteswap wire DTOs, hand-sequence DMA cache ops, or touch raw syscalls. Typed register accessors generated from regspecs (lockjaw-regs::{pl011, sdhci, virtio_mmio, fw_cfg, cprman}); typed wire DTOs generated from wirespecs (lockjaw-types::wire::{virtio, fwcfg, ramfb}); each TOML format documented at docs/reference/regspec-format.md and docs/reference/wirespec-format.md. OwnedDmaMapping<O> / DmaBacking<O> carry the DMA allocation origin in the type (sealed DmaOrigin: BuddyOrigin for coherent-bus pages, DmaPoolOrigin for the cache-maintenance pool — only the latter implements the sealed SyncCapable trait). run_dma_transfer (lockjaw-userlib::dma_transfer) is the coherence envelope: the driver declares the DmaRegions + a DmaCompletion impl that says "when is the device done" + a kick closure that programs and issues; the framework owns the clean→kick→await→invalidate ordering, including the B2.2 pre-clean of FromDevice regions and the post-completion invalidate. Handing a BuddyOrigin mapping to a sync is a compile error (the dma_region method is impl<O: SyncCapable>); forgetting the pre-clean or running the invalidate before the device finished is structurally impossible. emmc2's ADMA2 read converted to a two-envelope shape — outer buffer (FromDevice / Immediate / per-block kick loop) + per-block inner descriptor (ToDevice / SdhciDataCompletion wrapping the IRQ wait + the load-bearing B4.1 DAT_INHIBIT drain). The driver DmaBuf slot now owns a move-only DmaBacking<DmaPoolOrigin> and hands out &-borrows; the old all-zero sentinel is gone. #![deny(unsafe_code)] at every driver crate root (the boot-stub macro expansion is the single audited #[allow(unsafe_code)] site). lockjaw-userlib's root pub use syscall::* trimmed to the two-name allowlist (sys_exit, sys_debug_puts); forbidden sys_* are no longer reachable from use lockjaw_userlib::*;. The only remaining path is the explicit lockjaw_userlib::syscall::sys_foo, which the check-driver-unsafe xtask flags via syn AST analysis (covers brace imports, renames, UFCS qself, spaced paths, turbofish), a visit_macro token-stream walk (covers macro_rules! bodies and macro invocation args), and an ident_str raw-ident normalization (r#syscall matches syscall). Wired into make build alongside the existing gen-regs / gen-wires --check steps. Driver-substrate book chapter at docs/architecture/04-driver-substrate.md frames DMA coherence by construction as the worked example; the SdhciCommandInit<S> no-bypass operation layer (the rubric R3 remainder — gating raw DMA PAs and command issuance behind a type-state layer) is tracked in docs/tracking/tech-debt.md against the second SDHCI consumer. Pi 4B re-validated after the userspace-wide re-export restructure: [BLOCKDEV] /dev/sd0 ready ... selftest read OK, [FAT32-TEST] read 17 bytes: hello from fat32, posix-hello: hello from fat32, [NEON-CANARY] PASS.

Phase 21 -- Kernel-Side Substrate Hardening (NK + K + C series). Three parallel construction-safety regimes close the kernel's mirror of Phase 20's userspace work.

(a) Typed-front-door alloc seal (NK0-NK8). The "kernel never allocates" property goes from design intent to structurally enforced. Sealed-witness pattern: BootstrapAllocator / RuntimeAllocator / UserspaceAllocator marker types are constructable only via bootstrap_phase::Runner (one site, kmain) and with_userspace_for_dispatch (one site, syscall handler). mm::page_alloc / mm::kvm free functions are demoted to pub(in crate::mm) or fully module-private. KVM page-table tree is pre-allocated at bootstrap so the runtime walker can no longer grow metadata (NK1 deleted the corresponding AllocL2/AllocL3 states from the walker — illegal grow now unrepresentable). PageSet headers come from a bootstrap-allocated pool with typed AllocError. sys_create_thread, sys_create_process, and sys_map_pages now use single-PageSet donate-and-claim, so every fallible step lives in the validate phase and the apply phase is infallible. cargo xtask check-kernel-alloc (NK7 Mechanism B) is a syn AST scan that rejects pub-fn regression on the demoted free fns, alloc_* methods (inherent or trait) on RuntimeAllocator/RuntimeKvmAllocator, escape-valve constructors (Box/Vec/String/Rc/Arc/BTreeMap/...), call-site allowlist violations for the seal keystones, and direct .alloc_*() from any file outside the allowlist. Macro-token scan backstops macro_rules! bodies.

(b) Kernel-driver framework (K0-K9). Parallel to Phase 20's userspace driver substrate, but for kernel device files. Shared substrate crates moved to repo root: lockjaw-mmio, lockjaw-regs (typed MMIO accessors generated from regspecs/*.toml), lockjaw-sysregs (typed AArch64 sysreg DSL generated from sysregspecs/*.toml), and a new lockjaw-kernel-drivers crate hosting per-device region wrappers (Pl011Region, GicDistributorRegion, GicV2CpuRegion, GicV3RedistributorRegion) plus audited helpers (cache_ops for dc civac / dc cvac, gic_irq_ack for the compound IAR read). MutableMappedRegs<T> in lockjaw-mmio is a single-owner mutable-base substrate distinct from MappedRegs<T>'s no-aliasing invariant — supports PL011's two-phase boot lifecycle (pre-MMU PA then post-MMU VA). Every file in KERNEL_DRIVER_FILES (pl011.rs, timer.rs, mmu.rs, gic/mod.rs, gic/v2.rs, gic/v3.rs, cache.rs) carries #![deny(unsafe_code)] and routes every register access through the typed substrate. Three per-fn carve-outs remain for MMU compound transactions whose dsb/tlbi/isb/eret sequencing cannot decompose into per-sysreg DSL calls (enable_mmu / enable_mmu_secondary / drop_to_el0_with_ttbr0). cargo xtask check-kernel-driver-unsafe is a syn AST visitor + token-stream scan that bans raw ptr::read_volatile/write_volatile, banned-crate imports (lockjaw_regs/lockjaw_sysregs direct rather than via lockjaw-kernel-drivers re-export), and asm! mrs|msr mnemonics outside the per-fn carve-out. Compound reads/writes pinned by codegen-emitted compile_fail doctest tripwires at host cargo test --doc.

(c) Kernel console substrate (C0-C3). All kernel diagnostic output (kprintln!, crash handler, panic handler, canary failure, stack overflow report, sys_debug_puts) routes through a single lockjaw_kernel_drivers::console::write_byte / write_str chokepoint backed by an AtomicWriter slot installed at boot. KVA-stable install: pl011.rs declares pub(crate) static WRITE_BYTE_RAW_FN: fn(u8) = write_byte_raw; — the linker writes write_byte_raw's linker-baked high-VA into the static's 8 bytes at link time. The install helpers read those bytes via core::ptr::read_volatile(&raw const FP_STATIC) (forces an actual fetch rather than letting the compiler constant-fold to a PC-relative adr that yields the LOAD-PA pre-pivot). The pre-MMU install subtracts kernel_image_pivot_shift() to recover the load-PA; the post-MMU install uses the high-VA directly. TTBR1's L0[1] kernel-image mapping is untouched by _pivot_to_higher_half and drop_to_el0_with_ttbr0, so the stored value translates through every regime. The KERNEL_PHYS_OFFSET shift is stashed by the _start asm trampoline before bl kmain, not by Rust — init_kernel_image_map becomes a debug_assert_eq! cross-checker so the offset is valid throughout kmain. Witness chain: install_pre_mmu returns a ProvisionalConsole with module-private constructor; install_post_mmu consumes it and returns a Console; exceptions::init(Console) consumes the witness — deleting either install is a compile error. cargo xtask check-console-chokepoint enforces (i) only console_install.rs calls CONSOLE_WRITER.install, (ii) only pl011.rs / console_install.rs / main.rs name WRITE_BYTE_RAW_FN, (iii) no console::* call appears in secondary_main before enable_mmu_secondary(), and (iv) CONSOLE_WRITER carries neither #[link_section] nor #[no_mangle] (the cacheable Inner-Shareable publish/observe chain depends on the default kernel-image section). Verified end-to-end on Pi 4B: post-pivot kprintln reaches UART, all 17 userspace processes spawn through the same chokepoint, no regression in boot output.

The kernel's unsafe usage is gated by xtask AST scanners at every build (see the Build tools section). Audited helpers, regime checks, and codegen-emitted compile_fail tripwires together convert what used to be reviewer discipline into structural enforcement:

• Userspace drivers route every register access through lockjaw-regs typed accessors and every DMA through run_dma_transfer's cache-coherence envelope; #![deny(unsafe_code)] at every driver crate root; check-driver-unsafe bans direct lockjaw_userlib::syscall::sys_* calls except the sys_exit / sys_debug_puts allowlist.
• Kernel device files (the eight KERNEL_DRIVER_FILES) likewise carry #![deny(unsafe_code)]. Raw ptr::read_volatile/write_volatile and asm! mrs|msr are banned by check-kernel-driver-unsafe outside the three audited MMU compound-transaction carve-outs (enable_mmu / enable_mmu_secondary / drop_to_el0_with_ttbr0). Audited helpers cache_ops (dc civac / dc cvac) and gic_irq_ack (compound IAR read) are the only #[allow(unsafe_code)] sites in lockjaw-kernel-drivers/src/*.rs; the K9 audited-helper-count tripwire fires if a third appears.
• Kernel allocator is sealed end-to-end. BootstrapAllocator / RuntimeAllocator / UserspaceAllocator marker types are constructable only from the single boot site (bootstrap_phase::Runner) and the syscall-dispatch site (with_userspace_for_dispatch). mm::page_alloc / mm::kvm free functions are demoted to module-private; the runtime KVM walker no longer has any grow-metadata path because the page-table tree is pre-allocated at bootstrap. check-kernel-alloc AST-scans the kernel for pub-fn regression on the demoted free fns and rejects escape-valve constructors (Box/Vec/Rc/Arc/...).
• Diagnostic byte output routes through a single CONSOLE_WRITER chokepoint with a KVA-stable linker-baked install. check-console-chokepoint enforces a four-property witness chain — only console_install.rs calls .install, only three files name WRITE_BYTE_RAW_FN, no console::* before enable_mmu_secondary, no #[link_section]/#[no_mangle] on the slot static.
• Pre-existing patterns from earlier phases remain in force: KernelRef/KernelMut typed wrappers concentrate the KernelVa/KernelImageVa cast; object_ops facade provides narrow safe IPC/notification operations; BlockToken compile-time-enforces that no &mut T to a shared kernel object survives a block_current(); UserAddressSpace mediates safe copy_from_user; PageSetRef/HandleTableRef wrap validated IDs.

The remaining unsafe blocks in src/syscall/handler.rs are at architectural boundaries the type system cannot abstract (page table writes, GIC MMIO acknowledgement, capability-graph traversal across handle tables). Their count fluctuates as the donate-and-claim conversion of more syscalls lands; the regime checks above ensure each one stays at a boundary, not as an internal escape valve.

Three layers of automated testing run on every build: