From 4384b272c3f838cea396db73a41c54cfd36ea45b Mon Sep 17 00:00:00 2001 From: Graeme Geldenhuys Date: Wed, 24 Jun 2026 18:51:40 +0100 Subject: [PATCH] docs: FreeBSD x86_64 backend + multi-target ports/adapters architecture Design for a freebsd-x86_64 native target via direct syscalls (Strategy B, static ET_EXEC, no libc/sysroot), and the general ports-and-adapters architecture (TTargetToolkit Abstract Factory + TTargetRegistry, TKernelABI and TPlatformLayout ports) that lets multiple targets coexist with no IFDEFs and target divergence selected at runtime. --- docs/freebsd-x86_64-backend-design.adoc | 489 ++++++++++++++++++++++++ docs/native-target-architecture.adoc | 377 ++++++++++++++++++ 2 files changed, 866 insertions(+) create mode 100644 docs/freebsd-x86_64-backend-design.adoc create mode 100644 docs/native-target-architecture.adoc diff --git a/docs/freebsd-x86_64-backend-design.adoc b/docs/freebsd-x86_64-backend-design.adoc new file mode 100644 index 0000000..ee8da94 --- /dev/null +++ b/docs/freebsd-x86_64-backend-design.adoc @@ -0,0 +1,489 @@ += FreeBSD x86_64 Native Backend and Cross-Compilation — Design +:author: Graeme Geldenhuys +:revdate: 2026-06-24 +:toc: left +:toclevels: 3 +:source-highlighter: rouge +:icons: font +:sectanchors: +:sectnums: + +== Context + +Blaise's default backend is the native x86_64 code generator with an +in-process assembler and ELF linker. No external assembler or linker is +invoked: source compiles directly to a runnable ELF executable. Today the +only supported target is `linux-x86_64`. + +This document specifies the addition of a `freebsd-x86_64` target and, more +broadly, the work required to make the native backend a genuine +*cross-compiler* — producing a FreeBSD (and later Windows) binary while +running on a Linux host, with no FreeBSD machine, no QEMU, and no external +toolchain in the loop. + +FreeBSD/x86_64 is the right first cross-target: it shares the System V AMD64 +ABI and the ELF container with Linux, so the instruction selector, register +allocator, ABI lowering, and the bulk of the ELF linker are reused unchanged. +The differences are confined to a small, well-bounded surface. + +=== Is this possible? + +Yes. The conclusion of the investigation below is that a FreeBSD x86_64 +backend is achievable with *no changes to instruction selection or the +register allocator*, and that the existing architecture has already been +deliberately staged for it. The work is real but bounded — it is +parameterisation and one new system-call surface, not a second backend. + +The single largest decision is how the FreeBSD binary reaches the kernel. +That decision (libc-linked vs. direct-syscall) shapes the whole effort and is +discussed in <>. + +== Current architecture: what already generalises + +The codebase has been written with the second target in mind. The following +are already target-parameterised and require *no* per-OS work: + +[cols="2,4",options="header"] +|=== +| Component | State + +| Target model (`blaise.codegen.target.pas`) +| `TTargetOS` already enumerates `osLinux, osFreeBSD, osWindows, osMacOS`; + `TTargetCPU` has `cpuX86_64`. `ParseTargetName` already accepts + `"freebsd-x86_64"`. The host/`--target` plumbing exists. + +| Instruction selection / regalloc / ABI (`blaise.codegen.native.x86_64.pas`) +| System V AMD64 ABI is shared between Linux and FreeBSD x86_64. Argument + classification, the record-return classifier + (`ClassifyRecordReturn` in `blaise.codegen.native.backend.pas`), SSE vs + integer eightbyte rules — all identical. No change expected. + +| ELF object/relocation core (`blaise.linker.elf.pas`) +| The relocation set handled + (`R_X86_64_64/PC32/PLT32/GOTPCREL/RELATIVE/TPOFF32/JUMP_SLOT/GLOB_DAT/...`) + is architecture-defined, not OS-defined. Section merging, symbol + resolution, and address layout are OS-agnostic. + +| Linker target record (`TLinkTarget`) +| Already carries `OSABI` (with `ELFOSABI_FREEBSD = 9` already defined), + `EMachine`, `BaseAddr`, `PageSize` as per-target knobs rather than + hard-coded constants. + +| OPDF debug section +| The `.opdf.*` sections are non-allocatable and ride through the linker + unchanged into the final ELF (the merger explicitly keeps them). The + format is container-level identical on any ELF target. +|=== + +The design intent is recorded in the source: `blaise.codegen.target.pas` +states that "FreeBSD/x86_64 shares the System V AMD64 ABI with Linux, so the +x86_64 instruction selection will be reused; the OS differences are confined +to the link line." The linker header comment notes the same for `OSABI` and +the `-pre`-existing FreeBSD constant. + +== Where Linux assumptions are baked in + +The OS-specific assumptions are concentrated in three places. These are the +entire FreeBSD work surface. + +=== The runtime reaches the kernel through glibc + +`runtime/src/main/pascal/rtl.platform.posix.pas` is *pure Pascal* but is not +syscall-direct: every primitive is an `external name` binding to a libc +symbol — `open`, `read`, `write`, `lseek`, `close`, `fstat`, `stat`, +`mkdir`, `clock_gettime`, `fork`, `execvp`, `waitpid`, `nanosleep`, +`getenv`, `exit`, etc. + +Consequences for FreeBSD: + +* *Symbol level — portable.* The POSIX names are the same on FreeBSD libc. + The Pascal source needs no change for the call sites. +* *Struct level — NOT portable.* The hand-rolled `TStatBuf` and `TTm` + records in this unit use *Linux* `struct stat` / `struct tm` field + ordering and sizes. FreeBSD's `struct stat` has a different layout + (e.g. `st_ino` is 64-bit and field order differs; FreeBSD 12+ changed + `ino_t`/`dev_t` widths). `TStatBuf.Size`, `.Mtime`, `.Mode` would read + garbage offsets if reused verbatim. `struct tm` is largely compatible + but `tm_gmtoff`/`tm_zone` placement must be verified. +* *Constant level — NOT portable.* `O_CREAT`, `O_TRUNC`, `O_APPEND`, + `S_IFDIR`, `CLOCK_REALTIME`, `WNOHANG`, `SEEK_*` are hard-coded Linux + values in the `const` block. Several differ on FreeBSD (notably the + `O_*` flag bits and `WNOHANG`). + +=== The entry/startup model assumes glibc + +`blaise.codegen.native.x86_64.pas` (`EmitProgram`) emits an exported +`main(argc, argv)` and relies on the CRT's `_start` → +`__libc_start_main(main, ...)` to drive it. The internal linker +(`LinkViaInternalLinker` in `blaise.codegen.native.driver.pas`) links: + +* the host CRT objects discovered by `FindCrtObjects` in `uToolchain.pas` + (`Scrt1.o`, `crti.o`, `crtbeginS.o`, `crtendS.o`, `crtn.o`), searched + under `/usr/lib/x86_64-linux-gnu`, `/usr/lib/gcc/...`; +* the program object and `blaise_rtl.a`; +* in dynamic mode, against `libc.so.6` via the interpreter + `/lib64/ld-linux-x86-64.so.2` (both strings hard-coded in + `TLinker.BuildDynamic`). + +`__libc_start_main`, `Scrt1.o`, `libc.so.6`, and the glibc loader path are +glibc-specific. None exist on FreeBSD, whose libc startup convention, +CRT objects (`crt1.o`), and run-time linker (`/libexec/ld-elf.so.1`) +differ. + +=== Toolchain discovery is Linux-pathed + +`uToolchain.pas`: + +* `FindCrtObjects` searches Linux library directories only. +* `FindRTLArchive`/`ResolveToolchain` locate one `blaise_rtl.a` — built for + the host. A FreeBSD binary needs a FreeBSD-built `blaise_rtl.a`. +* `TLinker` is always constructed via `LinuxX86_64Target()`; the driver + never selects a FreeBSD `TLinkTarget`. + +[#syscall-strategy] +== The central decision: how does the FreeBSD binary reach the kernel? + +This is the one decision that shapes everything. Two viable strategies: + +=== Strategy A — link against FreeBSD libc (cross-link) + +Keep the existing libc-binding model. Produce a dynamically-linked ELF that +imports FreeBSD `libc.so.7`, uses interpreter `/libexec/ld-elf.so.1`, sets +`OSABI = ELFOSABI_FREEBSD`, and is started by FreeBSD `crt1.o`. + +* *Pros:* minimal new code; reuses the entire dynamic-link path; the RTL + stays libc-based; matches how a normal FreeBSD binary is built. +* *Cons (for a true Linux-hosted cross-compile):* the Linux host does not + have FreeBSD's `crt1.o` / `crti.o` / `crtn.o` or a `libc.so.7` import + stub. To cross-link you must stage a *FreeBSD sysroot* on the Linux host + (a copy of FreeBSD's `/usr/lib` CRT objects + libc) and point + `FindCrtObjects`/the interp/needed-lib at it. The CRT objects are + redistributable; this is the standard cross-toolchain approach. + +=== Strategy B — direct syscalls, no libc (freestanding) + +Bypass libc entirely on FreeBSD: emit our own `_start` that reads +`argc/argv` off the stack, and replace the `external name 'open'` bindings +with a FreeBSD syscall stub layer. Produce a *static* `ET_EXEC` +(the linker's Phase B static path already exists for exactly this +"hand-written object talks to the kernel through raw syscalls" case). + +* *Pros:* genuine zero-dependency cross-compile from Linux with *nothing* + staged — no FreeBSD sysroot, no CRT objects, no libc. This is the purest + expression of the stated goal ("cross-compile a FreeBSD binary from my + Linux system"). The produced binary is also self-contained on the target. +* *Cons:* requires a FreeBSD syscall layer in the RTL, and — critically — + the runtime's libc dependency is *deeper than file I/O*. An audit of the + `external name` bindings across `runtime/` shows the RTL also binds: ++ +[cols="2,4",options="header"] +!=== +! libc symbol(s) ! Why a raw-syscall replacement is non-trivial + +! `pthread_create/join/mutex_*` (`blaise_thread.pas`) +! Threading. Reimplementing on raw `thr_new`/`_umtx_op` FreeBSD syscalls is + real work, not a thin stub. + +! `mmap/munmap/mremap` (`blaise_mem.pas`) +! The memory manager. `mmap`/`munmap` map to syscalls cleanly, but FreeBSD + has no `mremap`; the allocator's in-place-grow path needs a fallback. + +! `memcpy/memset/memcmp` +! Hot primitives. Either bind to a freestanding implementation or emit + inline — cannot assume libc is present. + +! `localtime_r/gmtime_r/timegm` (`rtl.platform.posix.pas`) +! Timezone-aware date math. No syscall equivalent; needs a pure-Pascal + tz/calendar implementation or shipping the rules. + +! `setjmp`/`longjmp` (`_blaise_longjmp`, `blaise_setjmp_x86_64.s`) +! Exception unwinding. Already hand-written assembly — portable, but must be + verified for the FreeBSD stack/`jmp_buf` layout. + +! `atexit` +! Finalisation. Replaceable with our own registry plus an `exit`-time flush. +!=== ++ +FreeBSD's syscall ABI itself is straightforward (number in `%rax`, args in +`rdi, rsi, rdx, r10, r8, r9` — same registers as Linux, *different syscall +numbers*; errors via the carry flag, not a negative return). `struct stat` +layout still differs (same issue as Strategy A). The honest cost is the +non-I/O leaves above, not the syscall mechanism. + +=== Decision: Strategy B (direct syscalls) + +*Strategy B is chosen.* It is the only route that delivers the stated goal — +cross-compiling a runnable FreeBSD binary from a Linux host with no external +tools and nothing staged — and the static `ET_EXEC` / `_start` path it needs +already exists in the linker (Phase B). Strategy A is recorded above only as +the fallback shape for users who later want a conventional libc-linked FreeBSD +binary; it is not on this implementation path. + +The audit above (the libc surface beyond file I/O — threads, the memory +manager, libc string primitives, timezone-aware date math) is *not* a reason +to avoid Strategy B; it is the *work breakdown* for Strategy B. Each libc leaf +becomes a method on a target-keyed kernel port and is reimplemented hardest- +last: + +. *Easy leaves first* — `open/read/write/close/lseek/fstat/stat/mkdir/unlink/ + rename/getcwd/chdir/getenv/exit/nanosleep/pipe/dup2/fork/execvp/waitpid` + map directly to FreeBSD syscalls. +. *`memcpy/memset/memcmp`* — bind to freestanding implementations (or emit + inline); never assume libc is present. +. *`localtime_r/gmtime_r/timegm`* — a pure-Pascal calendar/timezone + implementation (no syscall equivalent). +. *Memory manager (`mmap/munmap`, no `mremap` on FreeBSD)* — syscalls plus a + grow-path fallback. +. *Threads (`pthread_*`)* — FreeBSD `thr_new`/`_umtx_op`; the hardest leaf, + done last. +. *`setjmp/longjmp`* — already hand-written asm; verify the FreeBSD + `jmp_buf`/stack layout. +. *`atexit`* — our own registry flushed at `exit`. + +=== No `{$IFDEF}`, no scattered target code — ports and adapters + +A hard requirement on this work: FreeBSD support must NOT be expressed with +`{$IFDEF}` directives or `include`-file substitution, and target-specific code +must not be scattered across units. Blaise already rejects that model — +`rtl.platform.pas` is explicitly `{$IFDEF}`-free and selects behaviour by +subclassing. This backend completes that idiom. + +The full structural design lives in +`docs/native-target-architecture.adoc`. In brief, each target is a *family of +adapter classes* produced by an Abstract Factory (`TTargetToolkit`) resolved by +name from a `TTargetRegistry`, with two new runtime ports introduced for +Strategy B: + +* `TKernelABI` — the raw syscall + `_start` entry leaf (adapters + `TKernelLinuxX86_64`, `TKernelFreeBSDX86_64`). +* `TPlatformLayout` — `struct stat`/`struct tm` layouts and the OS constants + (`O_*`, signals, `CLOCK_*`, `WNOHANG`, `SEEK_*`), so the Linux values + currently hard-coded inside `TRtlPlatformPosix` move behind a per-target + port instead of leaking into a "shared" adapter. + +Target divergence therefore lives in *which adapter object is instantiated*, +decided once at startup — and the test runner can select the FreeBSD adapter +at run time to test it natively on a FreeBSD host, or structurally from Linux. +The implementation steps below assume the structural refactor in the +architecture doc (introduce toolkit/registry, extract `TPlatformLayout`, +introduce `TKernelABI`) lands first as behaviour-preserving refactors against +the existing Linux suite. + +== Implementation plan + +Each step states a verification check, per the project's goal-driven +convention. Steps 0a–0c are the behaviour-preserving structural refactor from +`docs/native-target-architecture.adoc` (no FreeBSD code, no behaviour change — +verified against the existing Linux suite/fixpoints). Steps 1–9 add the +FreeBSD target as a new adapter family on those clean seams. Strategy B +(direct syscalls, static `ET_EXEC`) is assumed throughout; there is no +dynamic-link / sysroot work on this path. + +=== Step 0a — Toolkit + registry (refactor, no behaviour change) + +* Introduce `TTargetToolkit` (Abstract Factory) and `TTargetRegistry` + (name → toolkit). Register the existing Linux target as + `TLinuxX86_64Toolkit`. Re-express `--target` resolution and + `TargetHasNativeBackend` via the registry instead of the current hard-wired + `LinuxX86_64Target()` / `case`. +* *Verify:* full suite + all fixpoints unchanged (pure refactor). + +=== Step 0b — Extract `TPlatformLayout` (refactor, no behaviour change) + +* Move the `TStatBuf`/`TTm` record layouts and the `O_*`/`S_*`/`CLOCK_*`/ + `WNOHANG`/`SEEK_*` constants out of `rtl.platform.posix.pas` and behind a + `TPlatformLayout` port. The Linux adapter supplies today's values verbatim. + No `{$IFDEF}`; the layout object is selected at startup. +* *Verify:* full suite unchanged (the Linux layout reproduces current values). + +=== Step 0c — Introduce `TKernelABI` (refactor, no behaviour change) + +* Add the `TKernelABI` port (syscall + entry-stub leaf). The first Linux + adapter may still *delegate to the existing libc bindings* so the seam lands + with zero behaviour change before any syscall rewrite. +* Lift the ARC field-kind walkers and record-return classifier into + `TNativeBackend` as template methods with abstract per-CPU leaves, as the + backend base class already plans. +* *Verify:* full suite + native fixpoint unchanged. + +=== Step 1 — FreeBSD link target + toolkit skeleton + +* Add `TFreeBSDX86_64Toolkit` and register it. Its `MakeLinkTarget` returns a + FreeBSD `TLinkTarget` with `OSABI := ELFOSABI_FREEBSD`, + `EMachine := EM_X86_64`, and the FreeBSD load base / page size + (`0x200000` alignment — confirm against a reference FreeBSD `ET_EXEC`). + `MakeBackend` reuses `TX86_64Backend` (shared System V ABI). +* `blaise.codegen.target.pas`: `TargetHasNativeBackend` returns `True` for + `(osFreeBSD, cpuX86_64)` via the registry. +* *Verify:* `--target freebsd-x86_64` no longer errors "backend not yet + implemented"; a unit test asserts the emitted ELF header `EI_OSABI` == 9. + +=== Step 2 — FreeBSD `TPlatformLayout` adapter + +* Add `TLayoutFreeBSDX86_64`: the FreeBSD `struct stat` field offsets (note + FreeBSD 12+ widened `ino_t`/`dev_t`; pin to a target major version), + `struct tm` layout, and the FreeBSD `O_*` / `S_IFDIR` / `CLOCK_REALTIME` / + `WNOHANG` / `SEEK_*` constant values. +* *Verify:* a FreeBSD `fstat`-based `FileExists`/`FileAge`/`DirectoryExists` + returns correct results under emulation (Step 9) — static checks cannot + catch a wrong struct offset. + +=== Step 3 — Freestanding `_start` entry stub + +* The FreeBSD `TKernelABI` adapter's entry contract emits `_start` (not + `__libc_start_main`): read `argc` at `[%rsp]`, `argv` at `[%rsp+8]`, align + the stack, call `_SetArgs`, run the body, then `exit` via syscall. The + linker's static `ET_EXEC` / `_start` path (Phase B) already supports this — + no GOT/PLT, no interp, no CRT. +* *Verify:* `freebsd-x86_64` link produces an `ET_EXEC` with entry `_start` + and no `PT_INTERP`; linker unit test asserts the ELF shape. + +=== Step 4 — FreeBSD syscall leaf (the libc-replacement work) + +* Provide a FreeBSD syscall trampoline (a small `.s` stub like + `blaise_setjmp_x86_64.s`, or compiler-emitted): number in `%rax`, args in + `rdi/rsi/rdx/r10/r8/r9`, `syscall`, then branch on the *carry flag* to + translate FreeBSD's CF-error convention into the errno-style negative return + the existing Pascal call sites (`if Fd < 0 ...`) expect. +* Add a FreeBSD syscall-number table (e.g. open=5, read=3, write=4, close=6, + exit=1, plus `fstat`/`lseek`/`mmap`/`munmap`/… — FreeBSD numbers, *not* + Linux's). +* Implement the `TKernelABI` leaves against the trampoline, then the harder + leaves per the work breakdown above (`memcpy/memset/memcmp` freestanding; + pure-Pascal `localtime_r/gmtime_r/timegm`; `mmap`-based allocator with a + no-`mremap` grow fallback; `thr_new`/`_umtx_op` threads last; verify the + FreeBSD `jmp_buf` for setjmp/longjmp; an `atexit` registry). +* *Verify:* a FreeBSD hello-world + file-I/O smoke test produce correct output + under emulation. + +=== Step 5 — Per-target RTL archive + +* The runtime build (`runtime/`) produces a FreeBSD `blaise_rtl-freebsd- + x86_64.a` (compiled with `--target freebsd-x86_64`) containing *only* the + FreeBSD adapter family. `FindRTLArchive` selects the archive matching the + *target*, not the host. +* *Verify:* `ResolveToolchain(freebsd-target).RTLPath` resolves to the FreeBSD + archive on a Linux host; the produced binary contains no Linux adapter code. + +=== Step 6 — Drive target selection through the linker + +* `LinkViaInternalLinker` currently always builds `LinuxX86_64Target` and + calls `FindCrtObjects` unconditionally. Drive it from the resolved toolkit: + select the FreeBSD `TLinkTarget`, and for the Strategy-B static path skip + CRT discovery and dynamic mode entirely (static `_start`, no interp). +* *Verify:* `--target freebsd-x86_64` end-to-end emits a FreeBSD `ET_EXEC` on + a Linux host with no external tools invoked. + +=== Step 7 — `--target` CLI surface + +* Ensure `Blaise.pas` threads `--target freebsd-x86_64` into `AOpts.Target` + and that defaulting (no `--target`) still resolves to the host. +* *Verify:* `blaise --source hello.pas --target freebsd-x86_64 --output hello` + produces a file whose ELF header is FreeBSD/x86_64. + +=== Step 8 — Cross-compile verification under emulation + +The native fixpoint and e2e suites run host binaries; FreeBSD output cannot +execute on the Linux CI host. Add a verification lane: + +* *Static checks (host):* assert the emitted ELF's `EI_OSABI`, `e_type`, + entry symbol (`_start`), and absence of `PT_INTERP` via the existing ELF + reader. These run on every host, including Linux CI, and via the registry + can cover *every* registered target's codegen/link shape. +* *Dynamic checks (emulated/native):* run the produced FreeBSD binary under a + FreeBSD VM or `qemu-user` + a FreeBSD root in CI, executing a hello-world + and a file-I/O smoke test. This is the only way to catch `struct stat` / + syscall-number mistakes that static checks miss. On a FreeBSD host the test + runner self-targets (`HostTarget()` → FreeBSD toolkit) and exercises the + adapter natively — see the testing section of + `docs/native-target-architecture.adoc`. +* *Verify:* CI lane green: FreeBSD hello-world prints `Hello` and exits 0 + under emulation. + +== The broader cross-compilation picture (Windows, and Linux-hosted everything) + +The user's end goal is "cross-compile from pretty much any platform to +another target" — Linux → FreeBSD → Windows, all from Linux. FreeBSD is the +near case because it is ELF + System V ABI. Recording the generalisation +here so the FreeBSD work is shaped to extend rather than be re-done: + +* *Container abstraction.* FreeBSD reuses the ELF writer. Windows needs a + *PE/COFF* writer and macOS a *Mach-O* writer behind the same `TLinker` + symbol/relocation core (the linker header already anticipates "a sibling + writer behind the same TLinker symbol/relocation core"). This is the + large future item; FreeBSD does not touch it. +* *ABI abstraction.* Windows x64 uses a *different calling convention* + (RCX/RDX/R8/R9, shadow space, different callee-saved set). The record + classifier already has an `osWindows` branch (`rcWin64Agg`), so the seam + exists, but Windows is materially more backend work than FreeBSD. +* *Syscall/startup abstraction.* Each OS gets its own entry stub and either + a syscall leaf (Strategy B) or an import-library leaf (Win32: `kernel32`). + Building the FreeBSD direct-syscall leaf as a clean, target-keyed + `TKernelABI` adapter (Step 4) is the template every later OS follows. +* *Host independence.* Strategy B's "no host CRT, no host libc" property is + what makes Linux-hosted cross-compilation tool-free. Every target that + can be driven by direct syscalls (Linux, FreeBSD) should support it; + targets that cannot (Windows) fall back to a staged import library / + sysroot, same as Strategy A. + +== Risks and mitigations + +[cols="2,1,3",options="header"] +|=== +| Risk | Likelihood | Mitigation + +| FreeBSD `struct stat` layout wrong → silent garbage in file sizes/dates. +| High. +| This is the most error-prone item. Pin the layout against FreeBSD's + `sys/stat.h` for the targeted major version; add an emulated CI test that + stats a known file and asserts its size — static checks cannot catch this. + +| FreeBSD syscall numbers differ from Linux (Strategy B). +| High (if reused naively). +| Use a FreeBSD-specific number table; do NOT share Linux's. Error + reporting is via carry flag, not negative return — the trampoline must + translate. + +| No FreeBSD execution environment in CI → bugs ship untested. +| Medium. +| Stand up `qemu-user` + FreeBSD root, or a FreeBSD CI runner, for the + dynamic lane (Step 8). Without it, only static ELF-shape assertions are + possible. + +| FreeBSD major-version ABI drift (`ino_t`/`dev_t` widening in FreeBSD 12). +| Medium. +| Target a specific FreeBSD major (e.g. 13/14) and document it; the struct + layout is version-pinned. + +| Self-hosting on FreeBSD (a FreeBSD-native `blaise`) is a further step. +| Low for this task. +| Out of scope here: this design delivers Linux-hosted *cross-compilation* to + FreeBSD. A self-hosting FreeBSD stage-1 is a later milestone once the + cross output is proven. +|=== + +== Summary + +A `freebsd-x86_64` native backend is feasible and the codebase is already +staged for it: the instruction selector, register allocator, System V ABI +lowering, ELF relocation core, and OPDF debug sections are all reused +unchanged. The entire FreeBSD-specific surface is (1) a FreeBSD `TLinkTarget` +(OSABI, base, page size), (2) a FreeBSD `TPlatformLayout` (`struct stat`/ +`struct tm` layouts and OS constants), and (3) a FreeBSD `TKernelABI` (direct +syscalls + `_start` entry stub). + +The chosen kernel-access strategy is *direct syscalls into a static `ET_EXEC`* +(Strategy B): the linker's static path already supports it, and it is the only +route that delivers the stated goal — cross-compiling a runnable FreeBSD binary +from a Linux host with no external tools and nothing staged. Its cost is the +libc-replacement leaves (threads, allocator, libc string ops, date math), +which become methods on the FreeBSD `TKernelABI` adapter, done hardest-last. +A conventional libc-linked path (Strategy A) is documented only as a fallback +shape and is not on this implementation path. + +Structurally, FreeBSD is *one new adapter family* — composed by an Abstract +Factory toolkit, resolved from a registry, with zero `{$IFDEF}` and no target +code scattered across units (see `docs/native-target-architecture.adoc`). +The same shape generalises: Windows (PE/COFF + Win64 ABI + `kernel32` imports) +and macOS (Mach-O) become further adapter families behind the same linker core, +container writer behind a Bridge, and Template-Method backend base. diff --git a/docs/native-target-architecture.adoc b/docs/native-target-architecture.adoc new file mode 100644 index 0000000..365fe33 --- /dev/null +++ b/docs/native-target-architecture.adoc @@ -0,0 +1,377 @@ += Native Multi-Target Architecture — Ports, Adapters, and Target Toolkits +:author: Graeme Geldenhuys +:revdate: 2026-06-24 +:toc: left +:toclevels: 3 +:source-highlighter: rouge +:icons: font +:sectanchors: +:sectnums: + +== Purpose + +Blaise's native backend must grow from one target (`linux-x86_64`) to many +(`freebsd-x86_64` next, then Windows/macOS, then arm64). This document +defines the *structural* approach for that growth so that target divergence +is expressed through object selection at run time, never through `{$IFDEF}` +directives or `include`-file substitution. + +The explicit non-goal is the FPC model, where the target is frozen at compile +time and platform code is woven through every unit with conditional +compilation. That model is hard to read, hard to test (you can only test the +target you built for), and couples every unit to every platform. Blaise +already rejects it: `runtime/src/main/pascal/rtl.platform.pas` states "No +`{$IFDEF}` directives appear anywhere in this unit or its concrete +implementations." This document generalises that principle across the whole +native toolchain. + +== What already exists + +Blaise has *already* adopted the hexagonal (ports-and-adapters) pattern in two +places. The work in this document finishes and unifies that start; it does +not introduce a foreign idea. + +[cols="2,2,2,3",options="header"] +|=== +| Layer | Port (abstract) | Adapter (concrete) | Source + +| Runtime OS surface +| `TRtlPlatform` +| `TRtlPlatformPosix` +| `runtime/.../rtl.platform.pas` + `rtl.platform.posix.pas` + +| Code generation +| `TNativeBackend` +| `TX86_64Backend` +| `compiler/.../blaise.codegen.native.backend.pas` + `...native.x86_64.pas` + +| Link target facts +| `TLinkTarget` +| `LinuxX86_64Target()` +| `compiler/.../blaise.linker.elf.pas` +|=== + +`TNativeBackend` already applies *Template Method* deliberately: its header +records that the ARC field-kind walk and the record-return classifier +(`ClassifyRecordReturn`) are target-independent and live in the base, while +"only the leaf emission (which register holds the base, which mnemonic +loads/stores it) is CPU-specific," with a plan to "lift those walkers here as +template methods whose leaf steps are abstract per-CPU primitives." That is +exactly the right instinct; this document names it and extends it. + +== What is leaky today + +Two gaps make the current seams insufficient for a second target: + +. *`TRtlPlatformPosix` hides a Linux dependency.* It collapses Linux and + FreeBSD into one adapter, but its `TStatBuf` / `TTm` record layouts and its + `O_CREAT` / `O_TRUNC` / `WNOHANG` / `CLOCK_REALTIME` / `S_IFDIR` constants + are hard-coded *Linux* values. A FreeBSD build reusing this adapter would + read garbage at the wrong struct offsets. This is the `include`-file + problem wearing a class's clothing. + +. *There is no port for the kernel leaf.* Today the runtime reaches the + kernel through `external name 'open'` libc bindings — the "syscall leaf" is + implicit in libc. Strategy B (direct syscalls, no libc — see + `freebsd-x86_64-backend-design.adoc`) needs that leaf to become an explicit, + swappable port. + +== The architecture + +=== Ports (Hexagonal / Dependency-Inversion) + +A *port* is an abstract base class plus a global instance pointer selected at +startup. Callers depend on the port, never on a concrete target. + +[cols="2,4",options="header"] +|=== +| Port | Responsibility + +| `TRtlPlatform` (exists) +| High-level OS operations (file/dir/process/time/console). Unchanged. + +| `TKernelABI` (new) +| The *syscall + startup leaf*: raw `Open/Read/Write/Close/Mmap/Munmap/Exit/ + …` for the target kernel, and the entry-stub contract (how `_start` reads + `argc/argv`, how `exit` is issued). Adapters: `TKernelLinuxX86_64`, + `TKernelFreeBSDX86_64`. In Strategy B, `TRtlPlatformPosix` calls + `GKernel.Open(...)` instead of `external name 'open'`. + +| `TPlatformLayout` (new) +| The target's *struct layouts and OS constants*: `struct stat` field + offsets, `struct tm` layout, `O_*` flag bits, signal numbers, `SEEK_*`, + `CLOCK_*`. This is where the Linux constants leaking into + `TRtlPlatformPosix` go to live properly, per target. +|=== + +[NOTE] +==== +`TKernelABI` and `TPlatformLayout` are kept *separate* deliberately. The +syscall ABI and the struct/constant layout vary on *different axes*: a future +`freebsd-arm64` may reuse FreeBSD's syscall numbers (`TKernelABI`) while +needing different struct packing (`TPlatformLayout`), and vice-versa. Two +ports let those vary independently instead of forcing a four-way adapter +matrix. The Abstract Factory (below) composes the correct pair so callers +never assemble an inconsistent combination by hand. +==== + +Compiler-side ports (already present, kept as ports): + +* `TNativeBackend` — instruction selection / ABI lowering / asm printing. +* `TLinkTarget` + the linker's container writer — ELF today (PE/Mach-O later). + +=== Which runtime units change, and which do not + +The runtime binds libc / syscalls across *eight* units today +(`rtl.platform.posix.pas`, `blaise_arc.pas`, `blaise_exc.pas`, +`blaise_mem.pas`, `blaise_thread.pas`, `blaise_weak.pas`, `blaise_float.pas`, +`blaise_str.pas`, plus `streams.pas` in stdlib). It is tempting to conclude +that all of them must be "made hexagonal." They must not. Refactoring each +unit into its own platform abstraction would produce eight mini-hexagons and +*recreate* the scattered-target-code problem this architecture exists to +avoid. + +The governing distinction is *not* "does this unit call libc" but *"does this +call vary by target"*: + +[cols="3,2,3",options="header"] +|=== +| Bound symbol(s) | Varies by OS? | Action + +| `open/read/write/close/lseek/fstat/stat/mkdir/unlink/rename/getcwd/chdir/ + getenv/exit/nanosleep/pipe/dup2/fork/execvp/waitpid` +| *Yes* — different syscall numbers; FreeBSD reports errors via the carry + flag, not a negative return. +| Route through `TKernelABI`. + +| `mmap/munmap/mremap` (`blaise_mem.pas`) +| *Yes* — and FreeBSD has no `mremap`. +| Route through `TKernelABI` (memory leaf, with a grow-path fallback). + +| `pthread_create/join/mutex_*` (`blaise_thread.pas`, and the mutexes used by + `blaise_exc.pas`/`blaise_weak.pas`) +| *Yes* — no pthreads without libc; FreeBSD uses `thr_new`/`_umtx_op`. +| Route through `TKernelABI` (threads leaf — the hardest, done last). + +| `localtime_r/gmtime_r/timegm` (`rtl.platform.posix.pas`) +| *Yes* — libc-dependent, no syscall equivalent. +| Replace with a *target-independent* pure-Pascal calendar/timezone unit + (no port needed — same code everywhere). + +| `memcpy/memset/memcmp` +| *No* — pure computation, never touches the kernel. +| Provide ONE freestanding implementation (CPU-keyed `.s`/Pascal primitive, + like the existing atomic stubs). *Not* a `TKernelABI` method — this is a + "no libc is present" concern, not a "target differs" concern. + +| `_AtomicAddInt32`/`_AtomicSubInt32` (`blaise_atomic_x86_64.s`), + `_blaise_longjmp` (`blaise_setjmp_x86_64.s`) +| *No* — same x86_64 instruction sequence on every OS. +| Leave as-is (CPU-keyed assembly, already correctly isolated). For FreeBSD, + only *verify* the `jmp_buf`/stack layout used by setjmp/longjmp. + +| `_IntToStr`/`_DoubleToStr`/string helpers (`blaise_str.pas`, + `blaise_float.pas`) +| *No* — pure computation over Blaise string headers. +| Leave as-is. +|=== + +So the refactor is *narrow and funnelled*, not unit-by-unit: + +. *One unit changes shape* — `rtl.platform.posix.pas`: its `external name + 'open'` bindings become `GKernel.Open(...)` calls. This is the bulk of the + kernel-facing surface and the natural caller of the port. +. *A few units lose only their kernel-touching leaves* — `blaise_mem`, + `blaise_thread`, `blaise_exc`, `blaise_weak` keep all their logic; only the + handful of `mmap`/`pthread_*` lines move behind `TKernelABI`. The units are + *not* re-architected. +. *Most units are left alone* — pure-computation and CPU-keyed-asm units + (`blaise_str`, `blaise_float`, the atomic/setjmp stubs) do not touch a port. + `localtime_r`/etc. become one target-independent Pascal unit. + +The principle: *every kernel-varying leaf funnels into the single `TKernelABI` +port; everything that does not vary by target stays where it is.* One port is +the one place "what does FreeBSD do differently" lives — adding a port per unit +would defeat that. Forcing target-invariant code (memcpy, string formatting, +atomics) through a port would be abstraction for its own sake — the +over-engineering this design explicitly rejects. + +The migration sequence below exploits this: Step 0c introduces `TKernelABI` +with a Linux adapter that *still delegates to the existing libc bindings*, so +every caller is repointed at the port with *zero behaviour change* (verified +against the current Linux suite) before a single syscall is rewritten. No unit +is touched twice. + +=== Abstract Factory — the Target Toolkit + +The single most important new type. A `TTargetToolkit` takes a `TTargetDesc` +and produces the *consistent family* of adapters for that target: + +[source,pascal] +---- +type + TTargetToolkit = class + public + function MakeBackend: TNativeBackend; virtual; abstract; + function MakeLinkTarget: TLinkTarget; virtual; abstract; + function MakeKernelABI: TKernelABI; virtual; abstract; { runtime side } + function MakeLayout: TPlatformLayout; virtual; abstract; { runtime side } + function TargetName: string; virtual; abstract; + end; +---- + +Why Abstract Factory and not free functions: the produced objects must be +*coherent as a set*. A FreeBSD backend paired with a Linux link target, or a +FreeBSD syscall ABI paired with Linux struct offsets, is a silent bug. A +factory guarantees the family is internally consistent — this is the textbook +Abstract Factory motivation (families of related products that must be used +together). + +Concrete factories: `TLinuxX86_64Toolkit`, `TFreeBSDX86_64Toolkit`. + +=== Registry — name → toolkit + +A `TTargetRegistry` maps a canonical name (`"freebsd-x86_64"`) to its +`TTargetToolkit`. Adding a target is *one registration call plus the new +adapter files* — no existing `case` statement is edited, satisfying the +Open/Closed Principle. + +[source,pascal] +---- +TargetRegistry.Register('linux-x86_64', TLinuxX86_64Toolkit.Create); +TargetRegistry.Register('freebsd-x86_64', TFreeBSDX86_64Toolkit.Create); + +{ --target resolution, and the test harness, both go through this: } +Toolkit := TargetRegistry.Resolve(ParseTargetName(Opts.Target)); +---- + +This replaces the current `TargetHasNativeBackend` / `LinuxX86_64Target()` +hard-wiring with a lookup that the CLI *and* the test runner share. + +=== Pattern summary + +[cols="2,3,3",options="header"] +|=== +| GoF pattern | Applied to | Why it earns its place + +| Strategy +| `TNativeBackend`, `TKernelABI`, `TRtlPlatform`, `TPlatformLayout` +| Interchangeable target behaviour chosen at run time. Already in use + (`TProfilerStrategy`, `TNativeBackend`). + +| Abstract Factory +| `TTargetToolkit` +| Produces a *consistent family* of target adapters; prevents + mismatched-target bugs. + +| Template Method +| `TNativeBackend` base (ARC walk, record classifier) +| Shared algorithm, target-specific leaves. Already started in the backend + base class; lift the ARC walkers here as planned. + +| Bridge +| Linker symbol/reloc core × container writer (ELF/PE/Mach-O) +| Decouples the OS-agnostic relocation core from the container format so the + `(container × CPU × OS)` axes vary independently instead of multiplying into + an `N×M×K` subclass explosion. The linker header already anticipates "a + sibling writer behind the same TLinker symbol/relocation core." + +| Registry / Service Locator +| `TTargetRegistry` +| Resolve a toolkit by name without a central dispatch every new target must + edit. +|=== + +== Two selection moments (a subtlety, not a contradiction) + +The same ports-and-adapters pattern applies in two binaries with *different* +selection timing. Being explicit about this avoids accidentally linking every +target's runtime into every produced binary. + +[cols="2,4",options="header"] +|=== +| Binary | Selection moment & rule + +| The *compiler* (`blaise`) +| Holds *all registered* compiler-side toolkits at once (`TNativeBackend`, + `TLinkTarget`, container writers), so one `blaise` can cross-compile to any + target. Selection is per-invocation: `--target` (default `HostTarget()`) + resolves one toolkit from the registry. + +| A *produced* binary (e.g. a FreeBSD app) +| Must contain *only its own* runtime adapter (`TKernelABI` / + `TPlatformLayout` / `TRtlPlatform`), selected at that binary's own startup. + The unused targets' runtime code must NOT be linked in. This falls out of + per-target RTL archives: a FreeBSD binary links + `blaise_rtl-freebsd-x86_64.a`, which contains only the FreeBSD adapters. +|=== + +So: the compiler chooses a target *to emit for*; the produced binary chooses a +platform *to run on*. Both use the registry/factory, but the produced +binary's choice is fixed at link time by which RTL archive it pulls — there is +no run-time multi-target dispatch inside an application binary, and therefore +no bloat. + +== Testing — the payoff of run-time selection + +Because a toolkit is resolved from a `TTargetDesc` rather than frozen by +`{$IFDEF}`, the test runner gains two modes the FPC model cannot offer: + +. *Native self-target.* On a FreeBSD host, `HostTarget()` resolves the + FreeBSD toolkit, so running the compiler's own test suite there exercises + the FreeBSD `TKernelABI`/`TPlatformLayout`/`TNativeBackend` adapters + natively — no rebuild, no special flag. + +. *Cross-target static coverage.* On any host, the e2e harness can loop over + *every registered toolkit*, compile a probe program for each, and assert the + emitted object/executable is correctly shaped for that target (ELF + `EI_OSABI`, `e_type`, entry symbol, absence/presence of `PT_INTERP`, + relocation kinds). This gives Linux CI real coverage of the FreeBSD + backend's *codegen and linking* even though the FreeBSD binary cannot + execute on the Linux host. Behavioural (run-the-binary) coverage still + needs an emulated/native FreeBSD lane, but the structural regressions are + caught everywhere. + +This is a strict improvement over conditional compilation, where the only +testable target is the one the test binary was built for. + +== Migration sequence (structural) + +This is the *structural* refactor; the FreeBSD-specific content (syscall +numbers, struct layouts, `_start` shape) lives in +`freebsd-x86_64-backend-design.adoc`. + +. Introduce `TTargetToolkit` + `TTargetRegistry`; register the existing Linux + target through them. Re-express `--target` resolution via the registry. + *No behaviour change* — verify the full suite still passes (pure refactor). +. Extract `TPlatformLayout` from `TRtlPlatformPosix`: move `TStatBuf`/`TTm` + layouts and the `O_*`/`S_*`/`CLOCK_*`/`WNOHANG`/`SEEK_*` constants behind the + port; the Linux adapter supplies today's values. Verify suite unchanged. +. Introduce `TKernelABI`; for Linux, the first adapter may still delegate to + the libc bindings (no behaviour change) so the seam lands before the + syscall rewrite. Verify suite unchanged. +. Lift the ARC field-kind walkers and record classifier into `TNativeBackend` + as template methods with abstract per-CPU leaves (as the base class already + plans), so a second CPU/OS adapter inherits them. +. Only then add the `TFreeBSDX86_64Toolkit` adapters (per the FreeBSD design + doc). Adding the target touches *no* existing target's code — it is new + files plus one registry line. + +Steps 1–4 are behaviour-preserving refactors verifiable against the existing +Linux suite and fixpoints; step 5 is the actual FreeBSD work. Sequencing the +refactor first means the FreeBSD adapter slots into clean seams rather than +forcing them open mid-implementation. + +== Summary + +Hexagonal architecture is not just viable in Blaise — it is already the chosen +idiom for the runtime platform layer and the codegen backend, explicitly +`{$IFDEF}`-free. The path to many targets is to *complete* that idiom: +promote the kernel leaf and the struct/constant layout to first-class ports +(`TKernelABI`, `TPlatformLayout`), compose each target's adapter family with an +Abstract Factory (`TTargetToolkit`), resolve toolkits by name through a +Registry, keep the linker's container writer behind a Bridge, and reuse the +Template-Method backend base. Target divergence then lives entirely in *which +object is instantiated*, decided once at startup — the compiler picks a target +to emit for, a produced binary picks a platform to run on — and the test runner +can select adapters at run time to test any target, natively or structurally.