chore(phase3): deprecate QBE in help text, guard internal linker, document toolchain
Phase 3 of the native-default drive — the locally-verifiable items (CI and rolling-bootstrap changes are deferred to Phase 4, where they become correct once v0.12.0 is the rolling-bootstrap anchor): - Help text now marks the backends explicitly: "qbe (deprecated) | native (default)". QBE is retained as the e2e parity oracle and QBE-fixpoint guard; removal is a later release. - fixpoint-native-internal.sh now drives the FULL internal pipeline (internal assembler + internal linker) against an all-external (gcc assemble + gcc link) reference, instead of only differing on the assembler. Both internal tools are now the default toolchain, so the conformance guard must exercise both. Header comments updated to reflect that it guards the internal linker too and why it stays a differential probe (the internal assembler buffers the whole object in memory and OOMs on the compiler's own ~631k-line .s). - language-rationale.adoc records the decision: native backend + internal assembler + internal linker as the default toolchain (no external tools beyond the C runtime startup objects), why QBE is kept-but-deprecated, and the per-invocation escape hatches (--assembler external, --linker external, --backend qbe, --incremental). Verified: FIXPOINT_OK, NATIVE_FIXPOINT_OK, NATIVE_INTERNAL_OK; full suite 3541 tests / 0 failures on both the QBE-built and native-built test runners.
This commit is contained in:
Graeme Geldenhuys
parent
91b7354c8f
commit
de0ea5defb
|
|
@ -61,8 +61,13 @@ begin
|
|||
if I > 0 then
|
||||
Result := Result + ' | ';
|
||||
Result := Result + Names.Strings[I];
|
||||
if ParseBackendName(Names.Strings[I], K) and (K = bkNative) then
|
||||
Result := Result + ' (default)';
|
||||
if ParseBackendName(Names.Strings[I], K) then
|
||||
begin
|
||||
if K = bkNative then
|
||||
Result := Result + ' (default)'
|
||||
else if K = bkQBE then
|
||||
Result := Result + ' (deprecated)';
|
||||
end;
|
||||
end;
|
||||
finally
|
||||
Names.Free();
|
||||
|
|
|
|||
|
|
@ -4897,3 +4897,64 @@ standard `TStaticArrayTypeDesc` with bounds `0..N-1`.
|
|||
descriptor erases the index origin after semantic analysis, storing only
|
||||
integer bounds. This simplifies the codegen and is consistent with how
|
||||
const-array enum indexing already works.
|
||||
|
||||
== Native Backend and Internal Toolchain as Default
|
||||
|
||||
=== Decision
|
||||
|
||||
As of v0.12.0, the *native* x86-64 backend is the default code generator, and
|
||||
the in-process *internal assembler* (`blaise.assembler.x86_64`) and *internal
|
||||
linker* (`blaise.linker.elf`) are the default toolchain. A complete
|
||||
source-to-executable build invokes no external tools — only the host's C
|
||||
runtime startup objects (`crt1.o`, `crti.o`, `crtn.o` from the distribution's
|
||||
libc development package) are read from disk.
|
||||
|
||||
The QBE backend (`--backend qbe`) is retained but *deprecated*. It continues to
|
||||
serve as a differential oracle for the native backend in the e2e test suite
|
||||
(every `AssertRunsOnAll` program is compiled and run under both backends, and
|
||||
their output is asserted equal), and the QBE self-hosting fixpoint
|
||||
(`scripts/fixpoint.sh`) remains a guard. QBE is scheduled for removal in a
|
||||
later release once the native backend has accumulated sufficient field exposure.
|
||||
|
||||
=== Rationale
|
||||
|
||||
* *Toolchain independence.* The original pipeline shelled out to `qbe`, then
|
||||
`cc` to assemble, then `cc`/`ld` to link. Each is an external dependency a
|
||||
user must install and keep version-compatible. The native backend plus
|
||||
internal assembler and linker reduce the hard dependency to the C runtime
|
||||
startup objects alone, which simplifies installation and makes the compiler's
|
||||
output reproducible independent of the host toolchain version.
|
||||
|
||||
* *Exact debug information.* Only the native backend records precise codegen
|
||||
facts (real frame-pointer offsets, per-statement line labels) consumed by the
|
||||
OPDF debugger. The QBE backend emits approximate AST-walk debug records.
|
||||
Making native the default aligns the default build with the debuggable build.
|
||||
|
||||
* *Self-hosting closure.* The native pipeline reaches a clean self-hosting
|
||||
fixpoint (`scripts/fixpoint-native.sh` — stage-2 and stage-3 native assembly
|
||||
are byte-identical), so the compiler can rebuild itself end to end without QBE.
|
||||
|
||||
=== Alternatives considered
|
||||
|
||||
* *Remove QBE immediately.* Rejected for v0.12.0. QBE's value as an
|
||||
independent second implementation — the parity oracle that surfaces native
|
||||
codegen bugs the single-backend tests cannot — is highest precisely while the
|
||||
native backend is still maturing. Removal is deferred to a later release.
|
||||
|
||||
* *Keep QBE as the default and ship native opt-in.* Rejected. This was the
|
||||
prior state; it left the default build path different from the debuggable
|
||||
build path and kept the external-tool dependency on the common path.
|
||||
|
||||
=== Escape hatches
|
||||
|
||||
The internal toolchain can be bypassed per invocation when diagnosing a
|
||||
suspected internal-assembler or internal-linker defect:
|
||||
|
||||
* `--assembler external` — assemble the native `.s` with the host `cc` instead
|
||||
of the internal assembler.
|
||||
* `--linker external` — link with the host `cc`/`ld` instead of the internal
|
||||
linker.
|
||||
* `--backend qbe` — fall back to the QBE pipeline entirely (deprecated).
|
||||
|
||||
The default native build emits a single whole-program object; `--incremental`
|
||||
opts in to per-unit object emission with a warm cache.
|
||||
|
|
|
|||
|
|
@ -1,28 +1,35 @@
|
|||
#!/bin/bash
|
||||
# Internal-assembler conformance check for the Blaise self-hosting toolchain.
|
||||
# Internal-toolchain conformance check for the Blaise self-hosting toolchain.
|
||||
#
|
||||
# WHY THIS EXISTS, separately from fixpoint-native.sh:
|
||||
# WHAT IT GUARDS:
|
||||
# The DEFAULT native pipeline since v0.12.0 is the in-process internal
|
||||
# assembler (blaise.assembler.x86_64) + internal linker (blaise.linker.elf):
|
||||
# source -> native .s -> internal assembler -> .o -> internal linker ->
|
||||
# executable, with zero external tools (only distro CRT objects are read).
|
||||
# This script asserts that the fully-internal pipeline produces a binary
|
||||
# that BEHAVES identically to the trusted all-external (gcc assemble + gcc
|
||||
# link) reference.
|
||||
#
|
||||
# WHY IT EXISTS, separately from fixpoint-native.sh:
|
||||
# fixpoint-native.sh validates native CODEGEN by diffing the emitted .s
|
||||
# text, but it assembles that text with the EXTERNAL assembler (gcc).
|
||||
# The in-process internal assembler (blaise.assembler.x86_64, reached via
|
||||
# --assembler internal) is a DIFFERENT .s -> object path that NEITHER
|
||||
# fixpoint exercises. A miscompilation that only corrupts the internal
|
||||
# assembler's object output therefore passes both fixpoints cleanly
|
||||
# (this is exactly how the sret-Result field-read bug hid; see bugs.txt
|
||||
# 2026-06-16).
|
||||
# text, but it assembles + links that text with EXTERNAL gcc. Neither the
|
||||
# internal assembler nor the internal linker is on that path, so a defect
|
||||
# in either silently passes both fixpoints (this is exactly how the
|
||||
# sret-Result field-read bug hid; see bugs.txt 2026-06-16). This script is
|
||||
# the only fixpoint-level guard that drives both internal tools end to end.
|
||||
#
|
||||
# WHY THIS IS A DIFFERENTIAL CHECK, NOT A SELF-HOSTING FIXPOINT:
|
||||
# WHY IT IS A DIFFERENTIAL CHECK, NOT A SELF-HOSTING FIXPOINT:
|
||||
# The internal assembler currently buffers the whole assembly + ELF
|
||||
# object in memory, so assembling the compiler's own ~631k-line .s needs
|
||||
# ~53 GB and OOM-kills. A true "compile the compiler with itself via
|
||||
# --assembler internal" fixpoint is therefore not feasible until the
|
||||
# internal assembler streams its output. Instead we compile a
|
||||
# representative program that exercises the bug-prone paths (record-return
|
||||
# sret-Result field reads, immutable string literals) BOTH ways and assert
|
||||
# the internal-assembler binary BEHAVES identically (stdout + exit code) to
|
||||
# the trusted external-assembler binary. Note: the two assemblers may emit
|
||||
# different-but-valid encodings, so a byte-level section compare would
|
||||
# false-positive; behavioural equivalence is the sound invariant.
|
||||
# ~53 GB and OOM-kills. A true "compile the compiler with itself via the
|
||||
# internal toolchain" fixpoint is therefore not feasible until the internal
|
||||
# assembler streams its output. Instead we compile a representative program
|
||||
# that exercises the bug-prone paths (record-return sret-Result field reads,
|
||||
# immutable string literals) BOTH ways and assert the internal-toolchain
|
||||
# binary BEHAVES identically (stdout + exit code) to the all-external
|
||||
# reference. Note: the internal and external tools may emit
|
||||
# different-but-valid encodings / section layouts, so a byte-level compare
|
||||
# would false-positive; behavioural equivalence is the sound invariant.
|
||||
#
|
||||
# Requires: a native compiler at compiler/target/blaise (run the QBE
|
||||
# fixpoint or `pasbuild compile` first) and compiler/target/blaise_rtl.a.
|
||||
|
|
@ -99,25 +106,27 @@ PROBE_EOF
|
|||
|
||||
echo "compiler: $COMPILER"
|
||||
|
||||
echo "[1/3] compile probe with INTERNAL assembler"
|
||||
echo "[1/3] compile probe with INTERNAL assembler + INTERNAL linker (default pipeline)"
|
||||
if ! "$COMPILER" --source "$PROBE" $UNIT_ARGS \
|
||||
--backend native --assembler internal --output /tmp/fpni_int 2>/tmp/fpni_int.err; then
|
||||
--backend native --assembler internal --linker internal \
|
||||
--output /tmp/fpni_int 2>/tmp/fpni_int.err; then
|
||||
echo "INTERNAL_COMPILE_FAIL"; head -5 /tmp/fpni_int.err; exit 2
|
||||
fi
|
||||
|
||||
echo "[2/3] compile probe with EXTERNAL assembler (reference)"
|
||||
echo "[2/3] compile probe with EXTERNAL assembler + EXTERNAL linker (reference)"
|
||||
if ! "$COMPILER" --source "$PROBE" $UNIT_ARGS \
|
||||
--backend native --assembler external --output /tmp/fpni_ext 2>/tmp/fpni_ext.err; then
|
||||
--backend native --assembler external --linker external \
|
||||
--output /tmp/fpni_ext 2>/tmp/fpni_ext.err; then
|
||||
echo "EXTERNAL_COMPILE_FAIL"; head -5 /tmp/fpni_ext.err; exit 3
|
||||
fi
|
||||
|
||||
echo "[3/3] run both and compare stdout + exit code"
|
||||
# Behavioural equivalence is the sound invariant: the internal and external
|
||||
# assemblers may emit DIFFERENT-but-valid encodings for the same .s (so a
|
||||
# toolchains may emit DIFFERENT-but-valid encodings / section layouts (so a
|
||||
# byte-level .text/.rodata compare would false-positive), but a binary built
|
||||
# by the internal assembler must BEHAVE identically to the external-assembler
|
||||
# reference. The sret-Result bug broke exactly this (corrupted string
|
||||
# headers => wrong / garbage output), so stdout + exit code catch it.
|
||||
# by the internal assembler + internal linker must BEHAVE identically to the
|
||||
# all-external reference. The sret-Result bug broke exactly this (corrupted
|
||||
# string headers => wrong / garbage output), so stdout + exit code catch it.
|
||||
# Disable -e around the runs: a miscompiled internal binary may crash
|
||||
# (e.g. SIGSEGV from a corrupted string header), and we must REPORT that as
|
||||
# a guard failure rather than let -e abort the script mid-diagnostic.
|
||||
|
|
|
|||
Loading…
Reference in a new issue