The four-line "load the address of a named local-or-global into a
register" selector
if Self.IsLocal(Name) then
Self.Emit('leaq <VarOperand>, <reg>')
else
Self.EmitLeaqGlobal(Name, <reg>)
recurred verbatim at 20 sites across the field-access ladders, the
record-method receiver ladders (the Phase-2 item-12 target), the static-
array base loads, and the @-address-of paths. Collapse them into a single
EmitVarAddr(AName, ADstReg) helper.
Pure dedup, no behaviour change: EmitVarAddr emits exactly the same two
instruction forms the inlined selector did (and EmitLeaqGlobal already
routes threadvars through TLS, per the preceding fix). Verified byte-
identical native .s across threadvar and non-threadvar record/array/
method-call programs; all four fixpoints and the full suite (both QBE-
built and native-built test runners) unchanged.
A threadvar of record (or any aggregate whose base address is taken via
leaq) was addressed with a bare `leaq Name(%rip)` for both direct field
access and method-call receivers, with no %fs:0/@tpoff. The symbol was
correctly placed in .tbss, but a %rip-relative reference to a TLS symbol
does not resolve to the per-thread slot, so all threads shared one static
slot and clobbered each other. Scalar threadvars were already correct
(VarOperand emits %fs:Name@tpoff); QBE was unaffected.
Route every global record/array base load through the TLS-aware
EmitLeaqGlobal helper instead of bare `leaq Name(%rip)`:
- 16 field-access base ladders (FAE/FA/AFA.RecordName)
- 4 record-method receiver sites (the >6-slot and <=6-slot ladders)
- the offset-0 scalar-field-read fast path, which read the field
directly off Name(%rip); now branches on IsThreadVarGlobal and falls
through to EmitLeaqGlobal + indirect load for threadvars.
EmitLeaqGlobal emits the identical bare leaq for non-threadvar globals,
so codegen for ordinary globals is byte-identical (verified by .s diff)
and all four fixpoints are unchanged.
Tests: cp.test.e2e.threading gains
TestRun_ThreadVar_RecordField_PerThreadIsolation and
TestRun_ThreadVar_RecordMethod_PerThreadIsolation — 3 threads each stamp
a distinct id into a record threadvar (via field access / via method
call), spin re-reading, and assert no cross-thread clobber. Both fail
before the fix (native prints CORRUPT) and pass after.
Phase 2 item 11 of the de-duplication plan — the largest single dedup. The
">6-slot store-into-stack-slots" loop (write each logical argument to its
(I+1)*8(%rsp) slot, dispatching per kind: reload a hoisted akRecCall buffer
from its hoist depth / store a var-param address / store a width-adjusted xmm
bit pattern for a by-value float / store %rax for a plain scalar) was copy-
pasted verbatim at 6 call sites — the regular, metaclass-ctor, implicit-Self,
and proc-field >6-slot method paths. Collapse into EmitArgsToSlots(AArgs,
AParams, AAllocSz, AHoistTotal, AHoistDepths, AHoistKinds); 7 call sites now
share it.
Risk gates preserved as the helper contract: all 6 sites used the same Self-at-
slot-0 base offset (Dest = (I+1)*8) and the same push-phase-free akRecCall
reload formula (AAllocSz + AHoistTotal - depth), so they parameterize cleanly —
no phase/index divergence to thread.
Behaviour-reconciling, but verified behaviour-preserving: native .s for a
>6-arg program (Self+8 int args, and a 7-slot mixed int/double/var-param call)
is byte-identical to pre-refactor HEAD; the native self-host fixpoint (which
compiles the whole compiler, exercising every >6-slot path including metaclass
and proc-field) is unchanged. All four fixpoints pass; full suite OK (3853
tests) on both QBE-built and native-built test runners.
Phase 1 item 7 of the de-duplication plan. The post-call sret return-buffer
slide-down (`if HTotal > 0 then` reclaim the arg-hoist region and re-push the
fat pointer at (%rsp)) was byte-identical at the tail of all three
interface-sret return paths: EmitIntfSretCall, EmitIntfSretMethodCall, and
EmitClassIntfSretMethodCall. Collapse into EmitSretBufferSlideDown(AHoistTotal).
Proven byte-identical: native .s for an interface-returning program (direct and
chained `MakeGreeter('Hi ').Greet('there')`, exercising the sret-return-with-
hoisted-arg slide) is identical to pre-refactor HEAD. All four self-host
fixpoints pass; full suite OK (3853 tests) on both QBE-built and native-built
test runners.
Phase 1 item 5 of the de-duplication plan. The read-path static-array
subscript scaling — `if SAT.LowBound <> 0 then subq $lowbound; imulq $elemsize`
applied to an index already in %rax — was spelled out at 7 sites, each casting
a different expression to TStaticArrayTypeDesc. Collapse the body into
EmitStaticElemScale(ASAT) and pass the resolved type.
Only the read sites are touched: the static-array STORE paths compute the
element size from a precomputed DAElemType local rather than
ASAT.ElementType.RawSize(), so they are deliberately left as-is (the imulq
operand differs).
Proven byte-identical: native .s for a static-array program (non-zero low
bounds, integer + double + record-nested arrays, read and write) is identical
to pre-refactor HEAD. All four self-host fixpoints pass; full suite OK (3853
tests) on both QBE-built and native-built test runners.
Phase 1 (byte-identical) de-duplication of the native x86-64 backend, items
1-3 of Andrew's plan:
- EmitStmtList(AList): the compound-body inline idiom
`for I := 0 to L.Count-1 do EmitStmt(TASTStmt(L.Items[I]))` was copy-pasted
at 14 sites (try/finally/except bodies, loop bodies, program block, unit
init). Now one helper; nil-safe. The element cast stays internal — the AST
stores statements in TObjectList (raw-pointer slots), so the cast is
inherent, just no longer repeated 14×.
- EmitTryFramePrologue(AFinallyBody, ALblExc, ALblTry): the setjmp try-frame
entry (slot alloc, _PushExcFrame, FExcDepth bump, FFinallyStack push,
_blaise_setjmp, testl/jnz/jmp) was byte-identical between EmitTryFinallyStmt
and EmitTryExceptStmt, varying only in the FFinallyStack payload (FinallyBody
vs nil) and the jnz target — now both parameters.
- EmitPopExcFrame: the frame-pop bookkeeping triplet (_PopExcFrame +
Dec(FExcDepth) + FFinallyStack.Delete) appeared at 5 sites; the Dec and
Delete must always travel with the pop, so they are emitted together. The
EmitExcUnwind loop deliberately keeps its bare _PopExcFrame (it must not
touch the codegen-time depth bookkeeping).
Proven byte-identical: native .s for an EH-heavy program (try/finally,
try/except with handlers + else, nested try, repeat/continue, compound) is
identical to pre-refactor HEAD. All four self-host fixpoints pass; full suite
OK (3853 tests) on both QBE-built and native-built test runners.
The SysV/Win64 record-return classifier — ClassifyRecordReturn plus its five
leaf predicates (IsRecordManagedClean, IsRecordAllIntegerLeaves,
IsRecordAllFloatLeaves, IsRecordAllIntOrFloatLeaves, EightbyteIsSSE) — existed
as byte-identical twins in TCodeGenQBE and TNativeBackend (~145 lines each).
This is a drift-prone ABI decision table that both backends must keep in
lockstep; a divergence would silently misclassify a record return on one
backend only.
Move the logic to free functions in blaise.codegen (Recret* — renamed to avoid
colliding with the backend methods, since Blaise resolves an unqualified call
inside a method to the method, not the unit function). Only the top-level
classifier consults the target OS (the Win64 aggregate rule), so it takes a
TTargetDesc parameter; the leaf predicates are target-independent pure type
walks. Each backend keeps its method as a one-line delegator, so every
existing Self.X call site is unchanged.
Validated behaviour-preserving: native .s AND QBE IR are byte-identical
pre/post for a record-return program exercising every return class
(rcInt1/rcInt2/rcSSE1/rcSSE2/rcIntSSE/rcSSEInt/rcSret/rcWin64-clean). All four
self-host fixpoints (QBE, native, internal-assembler, warm-cache) pass; full
suite OK (3853 tests) on both QBE-built and native-built test runners.
Implements item F4 of Andrew's native-codegen de-duplication plan — the
highest-value, lowest-risk cross-backend hoist.
A deep field chain feeding a method/property call — MakeIt().A.B.Method() —
crashed the native backend. The chained field read released the transient
function result immediately after the first field hop, but _ClassRelease at
refcount 0 runs _FieldCleanup, which frees the intermediate objects (A, then B)
that the rest of the chain still dereferences. The terminal call then read
freed memory: a use-after-free that segfaulted (or, when the freed slot was not
yet reused, returned the stale value — non-deterministic).
This was the root cause of the TProcTypesOfObjectTests crash that killed the
TestRunner process and blocked ~975 alphabetically-later tests in the full
sequential suite (all of which pass individually). QBE never released the
transient there, so it leaked instead of crashing — the bug was native-only and
only reproduced under a native-built test runner.
Fix: when the loaded field value is a class/interface reference borrowed from
the transient base, AddRef-pin it before the release so the cleanup's matching
release nets out and the object survives the chain. This intentionally leaks +1
on the result, matching the QBE deep-chain transient leak already documented in
bugs.txt; correctness over a crash until the chain gains a deferred-release
mechanism.
Adds TE2EArcTests.TestRun_TransientDeepChainMethodCall_NoUseAfterFree and a
RunUnderValgrindNative base helper (the QBE-only RunUnderValgrind cannot see a
native UAF; the stdout value check alone is unreliable because the read is
non-deterministic, so native valgrind is the real guard).
A function/getter result used as a field-access receiver was never
released, leaking +1 refcount per access on both QBE and native.
MakeCreature().HitPoints left the TCreature alive; TList<T>[I].Field
leaked via the Get getter.
Both backends now emit _ClassRelease for the transient base after the
field value is loaded. ExprOwnsRef/NativeExprOwnsRef extended to
recognise indexed-property subscripts (TStringSubscriptExpr wrapping a
ReadMethod-backed TFieldAccessExpr) as owning +1.
Dual-backend E2E regression test added
(TestDebug_ReceiverFieldAccess_NoLeak).
All 4 fixpoints verified (FIXPOINT_OK, NATIVE_FIXPOINT_OK,
NATIVE_INTERNAL_OK, WARMCACHE_FIXPOINT_OK).
A function or method returning a class leaves its Result at refcount +1: the
callee's `Result := x` AddRef'd and the epilogue did not release Result. The
caller's assignment therefore consumes that transferred reference and must NOT
AddRef again. The native backend's plain class-variable assignment emitted
_ClassAddRef unconditionally, so `C := MakeIt()` (any object-returning call)
leaked exactly one reference per call.
Guard the AddRef with NativeExprOwnsRef, mirroring the tyString/tyDynArray
branches directly above it and the QBE backend's ExprOwnsRef elision (the QBE
path has always been leak-free here). Eliding is safe even for a borrowed return
(`Result := G`): the callee's Result assignment still AddRef'd, so the
transient genuinely owns +1 — verified no use-after-free when the source object
outlives the assignment.
Found via a TList<TCreature> + `L[I].HitPoints := ...` program reporting leaks
under --debug; reduced to a non-generic `C := MakeIt(); C := nil` minimal case.
NOTE: this fixes the assignment-site half. Object-returning call results used as
a field/method *receiver* or as a subscript getter result (e.g. L[I].Field) are
a separate, deeper gap — the native backend lacks the QBE backend's
statement-level deferred-release (PendingRelease/Flush) — addressed next.
cp.test.e2e.leakcheck.TestDebug_FuncReturnAssign_NoLeak pins both backends.
Full suite 3850 green; native fixpoint reproduces (and shrank the compiler's own
emitted asm by ~1400 lines of redundant retains).
Add the FreeBSD entry stub for a static, libc-free ET_EXEC:
- runtime.start.static.freebsd.pas — the FreeBSD sibling of
runtime.start.static.linux. _start captures %rsp, aligns the stack, and calls
the Pascal _BlaiseStartC, which parses argc/argv/envp off the kernel's initial
stack (same layout as Linux), captures environ, calls main(argc, argv), and
exits via the FreeBSD exit syscall. Deliberately minimal: no TLS/auxv-walk
(a trivial program does no threadvar access) — that lands with the threads
work in Step 4, mirroring how the Linux static start gained TLS.
- runtime.syscall.freebsd.pas — the minimum kernel leaf _start needs: _exit
(SYS_exit = 1), write (SYS_write = 4, aliased from _sys_write), and the
environ global. The full file/process/thread leaf grows here in Step 4. ABI
notes record the FreeBSD differences from Linux: different syscall numbers and
the carry-flag error convention (only relevant for the error-translating
wrappers that arrive in Step 4).
Both units are standalone — nothing in the default Linux build graph uses them,
so the Linux RTL and self-hosting are unaffected; the FreeBSD RTL composition
links them at Step 5.
TLinkerE2ETests.TestLink_FreeBSDStart_StaticExecShape links the _start fixture
with the FreeBSD target and asserts the Strategy-B shape: e_type = ET_EXEC,
entry == _start, no PT_INTERP. (Uses LinkToBytes, not ReadFile — the latter
truncates at the ELF header's first NUL byte.)
Step 3 of docs/freebsd-x86_64-backend-design.adoc. Full suite 3850 green;
FIXPOINT_OK.
The RTL-unification work relocated the RTL units into the compiler tree and
replaced the per-host blaise_rtl.a archive with in-process source compilation;
the runtime now ships its own _start (commit 626ee4c9). Update the FreeBSD
backend design doc to match current reality:
- runtime/src/main/pascal/* -> compiler/src/main/pascal/* (unit relocation).
- blaise_thread.pas/blaise_mem.pas -> runtime.thread.pas/runtime.mem.pas; the
setjmp .s file is now inline-asm runtime.setjmp.pas (all .s migrated).
- Startup section: runtime owns _start; the linker no longer scans host CRT
objects (Scrt1.o/crti.o/crtn.o).
- Toolchain section: FindRTLArchive/blaise_rtl.a is gone; EnsureRTLObjects
source-builds the RTL; the internal linker honours --target (Step 1).
- Step 4 syscall trampoline reframed as inline-asm Pascal (no .s files),
mirroring the shipped Linux runtime.syscall.linux leaf.
- Step 5 reframed: end-state (no per-target .a) has landed, so it is a unit-list
change in EnsureRTLObjects, not an archive change.
- Mark Steps 1 and 2 done with NOTE admonitions; fix the Step 2 class name
(TPlatformLayoutFreeBSDX86_64) and pin it to FreeBSD 14.x; fix a Step 9 ->
Step 8 cross-reference (there is no Step 9).
Add rtl.platform.layout.freebsd with TPlatformLayoutFreeBSDX86_64, the
FreeBSD sibling of the Linux layout adapter. It supplies the FreeBSD 14.x
amd64 struct stat field offsets (st_mode@24, st_mtim.tv_sec@64, st_size@112,
sizeof=224 — all differing from Linux) and the O_CREAT/O_TRUNC/O_APPEND flag
bits (0x200/0x400/0x008, also differing); the remaining constants (S_*,
SEEK_*, CLOCK_REALTIME, WNOHANG) share Linux's values. Layout pinned to
FreeBSD 14.x and valid for 13.x (ino_t/dev_t widening landed in 12, stable
since).
The unit is a standalone sibling: nothing in the default (Linux) build graph
uses it, so the Linux RTL and self-hosting are unaffected. The FreeBSD RTL
archive composes it in place of rtl.platform.layout.linux (Step 5).
cp.test.platformlayout.freebsd adds 11 tests that instantiate the adapter
directly and assert each constant plus the three struct-stat accessors, the
latter by planting sentinels at the FreeBSD offsets in a 224-byte buffer and
reading them back — catching an offset typo on the Linux CI host without
needing FreeBSD emulation.
Step 2 of docs/freebsd-x86_64-backend-design.adoc. Full suite 3849 tests
green; FIXPOINT_OK.
streams.pas bound malloc/free/realloc to libc by name, which left an undefined
reference under --static (libc-free) — the memory streams are reachable from the
compiler, so a static self-host link failed on 'free'.
Bind them to the RTL's own mmap-backed allocator
(_BlaiseGetMem/_BlaiseFreeMem/_BlaiseReallocMem) instead. Same signatures, so
the call sites are unchanged; memcpy stays the C name (libc, or runtime.cstub
under --static). The streams now allocate from the same heap as everything else
and carry no libc dependency.
With this, free()/malloc()/realloc() are resolved and the compiler links fully
--static. Verified: blaise --static builds the compiler itself into a libc-free
ET_EXEC (no PT_INTERP, not a dynamic executable) that compiles + runs a program;
FIXPOINT_OK, NATIVE_FIXPOINT_OK, NATIVE_INTERNAL_OK; full suite 0 assertion
failures (residual EUnitNotFound errors are a local search-path artifact).
Closes the gaps that blocked a --static (libc-free) build of the compiler itself:
- runtime.syscall.linux: add the chmod(2) syscall stub (SYS_chmod=90) defining
the bare 'chmod' the internal linker's MakeFileExecutable imports.
- uSemantic: route RawStrToDouble through the RTL's pure-Pascal _StrToDouble
(runtime.float) instead of libc's strtod, dropping the last libc dependency in
the compiler's float-literal constant folding. Mirrors the existing
_DoubleToStr routing and is correct for both static and dynamic builds.
- blaise.linker.elf: align FTlsSize to FTlsAlign in both the static and dynamic
TLS layout so TPOFF32 (sym - FTlsAddr - FTlsSize) matches the thread pointer
the freestanding _start places at AlignUp(memsz, align). Previously these
diverged when the raw .tdata+.tbss sum was not a multiple of the TLS alignment,
shifting every threadvar read by the padding (latent: today's 104-byte/8-align
TLS happens to coincide).
With these, `blaise --static` can link the compiler itself; the resulting
binary is a fully static, libc-free ET_EXEC.
The asm brace-comment strip (added with the threads leaf to remove {...}
comments the lexer leaves inside asm blocks) ran over the WHOLE line, including
inside .ascii string literals. The native backend emits string-const data as
.ascii directives, and the compiler's own codegen contains template strings with
literal braces (e.g. codegen.qbe.pas's 'data $__s%d = { w -1, w %d, w %d, b
"%s", b 0 }'). When the compiler was assembled by the internal assembler, the
strip ate the '{ ... }' span out of those .ascii literals, corrupting the
template strings.
The damage only showed when the compiler ran --emit-ir / GenerateIR: the
corrupted template produced garbage data lines (data $__s0 = <garbage>), so the
~1240 IR-assertion tests failed (TestHelloWorld_HasStrLitData etc.). Compiled
user programs were unaffected (their own Format/strings come from freshly built
RTL), and the QBE-built compiler was unaffected (gcc assembles its .ascii), which
is why local QBE fixpoints stayed green while the CI native-internal TestRunner
went red.
Fix: skip over quoted strings in the brace strip, exactly as the adjacent
'#'-comment strip already does, so a brace inside a .ascii literal is left
intact.
Root-caused from the CI failure on ci/linux-direct-syscalls; the corruption is
visible as -clean but -garbled string data from a
native-internal-assembled compiler.
Pre-landing review fixes, all in the direct-syscall / threads-leaf series:
- runtime.thread.static.linux + runtime.start.static.linux: reclaim the
per-thread TLS block when clone(2) fails after BuildThreadTLS (previously only
the stack mapping was unmapped, leaking one TLS mapping per failed
pthread_create). Add a symmetric FreeThreadTLS that recovers the mmap base
from the thread pointer using the same alignment math BuildThreadTLS used.
- runtime.libc2.linux: system() now forwards `environ` to execve instead of
nil, so the spawned /bin/sh sees PATH/HOME/etc. (was running with an empty
environment under --static).
- runtime.libc2.linux: mkstemp loops getrandom until all 6 bytes are filled
rather than ignoring a short read that would weaken the random suffix.
- runtime.libc.linux: drop execvp's dead HasSlash branch (both arms called the
identical execve); keep the single call + the no-PATH-search note.
- runtime.thread.static.linux: correct the stale _clone_thread comment that
claimed args pass through callee-saved regs (they are seeded on the child
stack and popped).
- blaise.assembler.x86_64: in the asm brace-comment strip, splice without
trimming mid-scan (a left-trim shifted the string under the scan index) and
trim once at the end, so a second brace comment on the same line is still
found.
- blaise.codegen.native.driver: replace a non-ASCII ellipsis in an added
comment with '...' (ASCII-only source rule).
Verified: FIXPOINT_OK, NATIVE_FIXPOINT_OK, NATIVE_INTERNAL_OK; self-hosted
compiler builds clean (0 spurious lock prefixes); static threads binary runs
counter=80000 PASS (deterministic across runs); no unit/codegen/assembler/linker
test regressions.
Replaces the single-threaded no-op pthread_mutex stubs with a real, libc-free
threading leaf built directly on clone(2) and futex(2), so a static binary can
spawn worker threads — the last RTL piece needed for a libc-free static program
to do real multithreaded work.
runtime.thread.static.linux:
- pthread_create via raw clone(2): each thread gets its own mmap'd stack and a
fresh per-thread TLS block (CLONE_SETTLS) so threadvar access (exception
frames, the allocator) is per-thread. A small asm trampoline seeds the start
routine + arg onto the child stack, calls it, then exit(2) (thread-only).
- pthread_join via the CLONE_CHILD_CLEARTID futex word: the kernel clears it on
thread exit and futex-wakes; join futex-waits on it, then munmaps the stack.
- 3-state futex mutex (Drepper): the caller's pthread_mutex_t buffer's first
Integer is the futex word (0=unlocked, 1=locked, 2=contended); fast-path CAS,
slow-path FUTEX_WAIT/FUTEX_WAKE. sysconf is left to runtime.libc.linux.
runtime.syscall.linux: add the futex(2) syscall (SYS_futex, 6-arg, %rcx->%r10).
runtime.start.static.linux: capture the PT_TLS template at startup and expose
BuildThreadTLS so pthread_create can build each thread's TLS block the same way
_start builds the main thread's (variant II: TLS data then TCB self-pointer).
blaise.assembler.x86_64: add the xchg/cmpxchg encoders (xchgl/xchgq/cmpxchgl/
cmpxchgq, reg<->reg and reg<->mem) the futex mutex needs; strip brace comments
inside asm blocks in ParseLine (whole-line and trailing); and initialise
TParsedLine.HasLock to False so the lock prefix never carries between lines.
Verified: thrtest (8 worker threads x 10000 increments under the futex mutex)
prints counter=80000 PASS; QBE self-host FIXPOINT_OK.
The --static link path is currently blocked by a pre-existing self-host codegen
bug (spurious lock prefixes in large binaries built by a stage-2 compiler),
which is independent of this change and tracked separately.
Completes the runnable-static-binary milestone: `blaise --static` now produces a
fully static, libc-free ELF (statically linked, no NEEDED libc.so.6, no
PT_INTERP) that EXECUTES correctly — `hello` prints, arithmetic/WriteLn work.
Static TLS (the last blocker — threadvar access via %fs faulted because %fs was
unset and static mode emitted no PT_TLS):
- linker LayoutSections now lays out .tdata/.tbss as a TLS block and sets
FTlsAddr/FTlsSize/FTlsFileSize/FTlsAlign (previously only LayoutDynamic did,
so static-mode TPOFF relocs resolved against 0).
- EmitExecutable emits a PT_TLS program header when the program has TLS, and
LayoutSections reserves a 3rd phdr slot so the first section does not overlap
and overwrite it.
- runtime.start.static.linux: rewritten with a Pascal core (_BlaiseStartC) that
parses argc/argv/envp + the auxv, reads PT_TLS via AT_PHDR, mmaps + inits the
TLS block, sets the thread pointer with arch_prctl(ARCH_SET_FS) (variant II:
TP past the block, TCB self-pointer at %fs:0), then calls main.
runtime.thread.static.linux: no-op single-threaded pthread_mutex_* stubs
(correct for a single-threaded process). pthread_create is intentionally
unprovided, so a thread-using static program (e.g. `uses classes`) fails to
link — the honest signal that the threads leaf (clone/futex) is the one
remaining piece.
Also fixed: runtime.start.static.linux had its `uses` in the implementation
section (Blaise only honours interface-section uses) which made syscall symbols
invisible; and em-dashes in comments tripped the lexer (known UTF-8 issue).
Full suite green (3829); dynamic native/QBE (incl. TLS + classes) unaffected.
runtime.thread.static.linux: no-op pthread_mutex_init/lock/unlock/destroy
(correct for a single-threaded process; real clone/futex threads are deferred).
pthread_create is intentionally absent so a thread-spawning static program fails
to link rather than silently misbehaving.
With this the LAST libc leaf is resolved: a --static hello now links to a fully
static ELF (statically linked, no NEEDED libc.so.6, no PT_INTERP). It does not
yet RUN — it faults on the first threadvar access because %fs (the thread
pointer) is unset and the linker emits no PT_TLS for static mode. Static TLS
setup (PT_TLS + arch_prctl(ARCH_SET_FS) in _start + correct TPOFF) is the next
piece.
Full suite green (3829); dynamic native/QBE unaffected.
The logic-bearing (non-syscall) leaves. A --static link now resolves
abort/__cxa_atexit/exit/gmtime_r/localtime_r/timegm/mkstemp/system; the only
remaining gap is the pthread_mutex_* stubs (Tier 3).
New runtime.libc2.linux:
- atexit registry: __cxa_atexit records (func,arg) LIFO; the bare `exit`
(main's epilogue) runs them then exit_group. `exit`/`_exit` moved out of
runtime.syscall.linux (which keeps only the raw _exit that runs no handlers).
- abort: kill(getpid, SIGABRT) then exit_group(134).
- gmtime_r/localtime_r/timegm: freestanding civil-calendar math (UTC;
days_from_civil / civil_from_days). No timezone DB yet — localtime == UTC.
- mkstemp: getrandom-named O_CREAT|O_EXCL|O_RDWR 0600 file, retry on EEXIST.
- system: fork + execve /bin/sh -c + wait4.
Full suite green (3829); dynamic native/QBE unaffected.
Adds the syscall-backed half of the libc replacement. A --static link now
resolves mremap/getcwd/fork/execvp/waitpid/time/getenv/sysconf; the remaining
gap is the freestanding-logic leaves (abort/__cxa_atexit/gmtime_r/localtime_r/
timegm/mkstemp/system, Tier 2) and the mutex stubs (Tier 3).
- runtime.syscall.linux: + raw mremap/fork/execve/wait4/sched_getaffinity/
getrandom/arch_prctl/kill/sys_time/sys_getcwd (4th syscall arg %rcx->%r10 for
the >=4-arg ones).
- runtime.libc.linux (new): the libc-shaped wrappers that need more than a raw
syscall — getcwd (return buf/nil), getenv (walk environ), time, waitpid
(wait4 + NULL rusage), execvp (execve; direct path, PATH search deferred to
when a bare-name exec appears), sysconf(_SC_NPROCESSORS_ONLN) via
sched_getaffinity + popcount. Owns the `environ` global.
- runtime.start.static.linux: capture environ = &argv[argc+1] from the initial
process stack into the global before calling main.
- EnsureRTLObjects: link runtime.libc.linux in --static builds.
Full suite green (3829); dynamic native/QBE unaffected.
First increment of the libc-free direct-syscall RTL (Linux as the proving
ground for FreeBSD; docs/linux-syscall-migration.adoc to follow). The default
build is unchanged — libc-backed dynamic ELF; everything here is behind the new
--static flag.
New kernel-leaf units (selected by EnsureRTLObjects when --static):
- runtime.syscall.linux — raw `syscall` stubs DEFINING the bare POSIX names the
RTL imports (open/read/write/lseek/close/fstat/stat/mkdir/rmdir/unlink/rename/
chdir/getpid/dup2/pipe/mmap/munmap/nanosleep/clock_gettime/exit/_exit).
`write`/`exit` are Pascal-reserved, so their Pascal names are _sys_* and the
asm body emits a bare label alias. 4th syscall arg moved %rcx->%r10.
- runtime.cstub — freestanding memcpy/memset/memcmp/strlen (no Char type; PChar
byte indexing).
- runtime.start.static.linux — freestanding _start (argc/argv off %rsp, call
main, exit_group) replacing the libc __libc_start_main entry.
Wiring: --static -> TBackendOpts.Static; EnsureRTLObjects swaps runtime.start
for runtime.start.static.linux and adds the syscall+cstub leaves; the native
internal linker calls SetDynamic(False) for a non-PIE ET_EXEC.
Internal linker static-mode fixes (these are correct in any ET_EXEC, not just
PIE — they were wrongly gated to dynamic-only):
- R_X86_64_64 against a defined symbol resolves to the symbol's absolute address
(no dynamic reloc); only PIE/dynamic mode also emits the .rela.dyn RELATIVE.
- R_X86_64_TPOFF32 (Local-Exec TLS) resolves at link time — it IS the static TLS
model. Updated TestReloc_Quad64_* to assert the now-correct resolution.
A --static link currently reaches symbol resolution and reports the remaining
libc leaves to implement (mremap/getcwd/fork/execvp/waitpid/time/getenv/sysconf/
gmtime_r/localtime_r/timegm/__cxa_atexit/abort/mkstemp/system/pthread_mutex_*) —
the next increments. Full suite green (3829); dynamic native/QBE unaffected.
A record passed BY VALUE to a function (e.g. `const AParsed: TParsedLine`)
lowers to a QBE `:_ffi_<Name>` aggregate type whose field list was built by
FlattenRecordToAggLetters inlining each field's type letter back-to-back, with
nested records expanded letter-by-letter. That dropped the per-field and nested-
record tail padding, so QBE laid out the struct SMALLER than Blaise's record and
with shifted field offsets.
For TParsedLine this put the trailing HasLock byte at QBE offset 244 (struct size
248) while the function body reads it at the real Pascal offset 252 (size 256).
A by-value read at +252 then lands 4 bytes PAST the passed struct, in caller
stack garbage. When that garbage was non-zero, the assembler's
`if AParsed.HasLock then CBEmit($F0)` fired on EVERY instruction, prepending a
spurious lock prefix — `lock ret` is illegal, so the produced binary SIGILL'd.
This silently corrupted large native binaries built by a self-hosted compiler
(the assembler unit is where it bit hardest), and it was the real cause of the
"--static binaries are poisoned" and "stage-2 TestRunner SIGILLs" symptoms.
It hid from all four fixpoints: the QBE fixpoint compares IR text (the IR was
always correct — `loadub +252` is right; QBE miscompiled the by-value param
layout), and the native fixpoints pipe through the EXTERNAL assembler.
Fix: FlattenRecordToAggLetters now walks fields by their real TFieldInfo.Offset,
inserting explicit `b` byte padding before each field and after the last one up
to TotalSize, so the QBE aggregate's field offsets and size match the Pascal
record exactly. Nested records / static arrays / sets are emitted as their whole
byte span (the by-value ABI only needs correct size and field positions).
Verified: QBE-built compiler now produces 0 spurious lock prefixes; static
threads binary runs (counter=80000 PASS); compiler/target/blaise self-compiles
clean; FIXPOINT_OK, NATIVE_FIXPOINT_OK, NATIVE_INTERNAL_OK.
Calling a record-returning method as a statement (result discarded) omitted
the hidden sret pointer required by the SysV x86-64 ABI. Self was passed
in %rdi where the sret buffer should be, so the callee wrote the returned
record through the object pointer, corrupting its fields.
Semantic pass: set ResolvedReturnTypeDesc on TMethodCallStmt at all three
class-method resolution sites so both backends can detect the record return.
QBE backend: allocate a temporary buffer via alloc8, zero it, route through
EmitRecordReturnCallSite, then release any managed fields (strings, classes,
interfaces, dynamic arrays). Both the regular and ObjExpr receiver paths.
Native backend: reserve an aligned stack buffer, shift Self from %rdi to
%rsi, place the sret pointer in %rdi, and clean up managed fields after the
call. Handles the <=6-slot path.
A `#`-prefixed numeric literal now denotes a Unicode codepoint and contributes
its UTF-8 encoding to the surrounding string constant — the coherent semantics
for Blaise's Char-less, UTF-8-native string type:
#65 -> 'A' (1 byte $41)
#$20AC -> the euro sign (3 bytes E2 82 AC)
#$1F600 -> the grinning face(4 bytes F0 9F 98 80)
The literal is always a string, and adjacent `#` and '...' literals merge into
one compile-time constant (#72#73'!' -> 'HI!'), matching Pascal's literal-
merging tradition. Decimal (#nnnn) and hexadecimal (#$hhhh) are both accepted;
the underlying tokeniser already captured #$-hex runs, so the change is confined
to UnescapeString + a new CodepointToUtf8.
Codepoints outside 0..U+10FFFF and the surrogate range U+D800..U+DFFF (not valid
Unicode scalar values) are a compile error, so emitted string constants are
always valid UTF-8.
Tests cover decimal ASCII, hex BMP (3-byte), astral (4-byte), and #+string
merging. All four fixpoints green; full suite 3835.
After the RTL-unification move, stdlib units (classes, contnrs) explicitly
`uses runtime.arc` and friends. The compiler driver already source-builds the
RTL from compiler/src/main/pascal at link time, but the FRONT-END unit loader
was never told where that source lives — so compiling any program that uses a
stdlib unit (or any RTL unit transitively) failed with
"Unit 'runtime.arc' not found in search paths" unless the user manually passed
--unit-path compiler/src/main/pascal.
Add the RTL source directory to the loader's search paths automatically,
resolved the same way EnsureRTLObjects resolves it for linking: --rtl-src, then
$BLAISE_RTL_SRC, then the binary/CWD-relative RTLSourceDir default. (stdlib
itself stays an explicit --unit-path, as before — it is an opt-in library, not
implicit like the RTL.)
Regression introduced by the RTL-unification land; reported by a user.
Verified: a `uses classes` program now compiles + runs with only the stdlib
unit-path, from any CWD. All four fixpoints green; full suite 3831.
An implicit-Self method call — a bare `Method(...)` inside another method that
resolves to a method of the current class — loaded Self plus every argument
into the System V integer argument registers unconditionally. When Self and the
arguments together needed more than the six available integer registers,
codegen aborted with "System V integer arg register index 6 out of range".
The explicit-receiver path (obj.Method() / Self.Method()) already split on slot
count and spilled the overflow to the stack; only the implicit-Self expression
path was missing it. Add the same split: keep the push/pop-into-registers fast
path for <= 6 slots, and for more, evaluate the arguments into a stack slot
block, load the first six slots into registers, and spill the rest per the ABI
(EmitImplicitSelfCallOverflow, mirroring EmitMethodCallExpr's overflow branch).
Tests: an assembly-level test asserts the slot-block spill shape and dispatch,
and an end-to-end test runs a program with an 8-argument non-virtual and a
7-argument virtual implicit-Self call and verifies every argument — including
the spilled ones — arrives intact.
Partial Stage-3 teardown — the safe removals that do not affect the bootstrap
chain (the runtime Makefile/ar stay until a release ships a source-build-capable
binary, since the v0.12.0-rooted rolling-bootstrap replay needs them for its
pre-source-build commits):
- Remove FindRTLArchive + TToolchain.RTLPath from uToolchain — dead since the
link paths source-build the RTL; the field was set but never read.
- TInternalLinkerE2ETests gated on blaise_rtl.a existing, so it silently skipped
once the archive was gone; gate on the RTL source instead. It now runs (5
tests) — the last automated test still tied to the archive.
- Refresh the runtime mem test/bench build-instruction comments to the
source-built, archive-free invocation (blaise --output, no qbe+gcc+.a).
Full suite green (3829 tests, 1 ignored) — TInternalLinkerE2ETests no longer
skipped.
The rolling-bootstrap smoke probe ran `$cc --source s.pas --output s` with no
unit-path, relying on the binary finding the RTL beside itself. That worked
while the RTL was an archive (FindRTL looks beside the binary), but once a
commit makes the compiler source-build the RTL, the boot/carried binary is not
beside its compiler/ source tree, so RTLSourceDir (binary-relative, then
CWD-relative) misses and the probe fails with "RTL source directory not found".
Point the probe at the worktree's RTL source via $BLAISE_RTL_SRC and the unit
path. Backward-compatible: pre-source-build commits ignore $BLAISE_RTL_SRC and
still link the archive copied beside the binary; the extra unit paths are
harmless for a program that uses no units.
The E2E test helpers linked Blaise programs against the shipped blaise_rtl.a.
With the archive on its way out (RTL-unification Stage 3), migrate them to the
source-built RTL:
- cp.test.e2e.base gains LinkWithRTL: assembles the program object, builds the
RTL from source via scripts/build-rtl-objects.sh (--exclude-defined-by drops
the RTL units the whole-program assembly already inlined), and links the loose
objects. Its five cc-link sites and the ToolchainAvailable gate now use it /
the RTL source instead of the archive.
- New cp.test.rtllink unit factors the same link helper for the standalone
helpers (publishedrtti, proctypes_ofobject, attributes) that lower programs
themselves rather than via the base class.
- cp.test.cli and cp.test.assembler drive an external blaise binary that
source-builds its own RTL, so their gates now require the binary + RTL source,
not the archive.
- cp.test.linker's TestArchive_ParsesRTL kept its archive-parser coverage
(including the GNU long-name member) by building a throwaway .a from the
source-built objects with ar — a test-time-only ar use, not a product one.
build-rtl-objects.sh resolves its RTL source relative to its own location (so it
works from any CWD — the fixpoint scripts run from the project root, the test
runner from compiler/), caches a per-object .syms list to keep repeated
--exclude-defined-by calls cheap, and skips rebuilding fresh objects.
Full suite green (3829 tests) on both QBE- and native-built runners; the
previously-skipped TInternalAsmE2ETests / TCLIContractTests and the archive
parser test now run.
Builds on the source-built-RTL link path. Two correctness fixes plus the
fixpoint-script migration that takes the self-host pipeline off blaise_rtl.a.
1. Symbol-based dedup. A program that uses an RTL unit (e.g. `uses classes`
pulls runtime.arc) compiles that unit into its own object cache, so
EnsureRTLObjects must not supply a second copy or the link double-defines it.
The dedup now compares the DEFINED symbols of the caller's objects against
each RTL object (via the ELF reader) and skips one only when fully redundant
— matching the old archive's member selection. An earlier filename-only
check wrongly suppressed the real bare-symbol provider when a same-named but
differently-mangled stale object was present.
2. Race-free build. EnsureRTLObjects now builds into a per-process private
directory (rtl/build-<pid>/) and atomically renames each finished object
into the shared cache. Parallel builders (e.g. e2e suites warming a cold
cache) no longer corrupt each other: a reader never sees a half-written
object, and two builders just publish identical results. The private dir is
cleaned up afterwards. getpid/rmdir are declared directly rather than via
GRtlPlatform to avoid a unit-init-order crash in the driver.
3. Fixpoint scripts off the archive. New scripts/build-rtl-objects.sh builds
the RTL from source (the script-level equivalent of EnsureRTLObjects), with
--exclude-defined-by so a whole-program --emit-ir/--emit-asm dump (which
inlines the RTL units the compiler uses) is not handed a duplicate copy.
fixpoint.sh and fixpoint-native.sh link via this helper instead of
blaise_rtl.a; fixpoint-native-internal.sh and fixpoint-warmcache.sh no longer
copy the archive beside the compiler (the driver source-builds the RTL).
All four fixpoints green (FIXPOINT_OK, NATIVE_FIXPOINT_OK, NATIVE_INTERNAL_OK,
WARMCACHE_FIXPOINT_OK); full suite green (3829 tests) under QBE- and
native-built runners; four cold-cache parallel suite runs pass.
The runtime Makefile / ar / make install are still present; removing them is the
final Stage-3 step.
The leak tracker registered its end-of-run report via glibc's bare `atexit`.
That symbol lives only in the static libc_nonshared.a; it is NOT a dynamic
export of libc.so.6. The QBE/cc path linked fine because gcc auto-pulls
libc_nonshared.a, but the native backend's linkers (internal ELF linker and
the native external-assembler cc invocation) do not, so a --debug build aborted
at load time with `symbol lookup error: undefined symbol: atexit`.
Register via __cxa_atexit instead — a genuine dynamic libc export that resolves
on every link path. Its third argument is the DSO handle; nil registers the
handler against the main program. The handler takes one (ignored) argument.
Pre-existing bug (reproduces on v0.12.0); affects only --debug builds compiled
with --backend native. Verified on all three link paths (QBE/cc, native
internal linker, native external assembler) plus the full suite and all four
fixpoints.
Stage 3 of the RTL unification: the cc/QBE link line now source-builds the
RTL the same way the native internal linker already did, via a single shared
TBackendDriver.EnsureRTLObjects. Both paths compile the implicit RTL units
(runtime.* + rtl.platform.*) into a per-compiler object cache and link those
.o files directly, instead of linking the pre-built blaise_rtl.a. Verified
by linking a program with the archive moved aside on all three paths
(qbe+cc, native internal linker, native external assembler).
Moving from an archive to explicit .o files on the link line exposed two
pre-existing separate-compilation bugs that the archive's member selection
had masked (an archive member is only pulled to satisfy an undefined symbol,
so duplicate definitions across members never collide):
1. A unit compiled standalone via --source never had SourceFile set (only
loader-loaded dependency units did), so its exported interface carried an
empty SourceHash. A later compile depending on that unit could not
validate the cached .o and silently recompiled it from source, re-inlining
the dependency body. Set TopUnit.SourceFile from the --source path.
2. In unit-as-top-level mode the compiler never told the backend which
dependencies were imported (only the program path called NoteDepInitUnit),
so the backend re-defined imported units' globals (e.g. GPlatformLayout,
owned by rtl.platform) instead of referencing them externally. Mirror the
program path: note prebuilt-iface deps and SkipDepCodegen source deps so
their globals stay external.
EnsureRTLObjects builds the RTL leaf-first through a --unit-cache (so each
unit references its deps' globals externally and intermediate deps such as
rtl.platform are built once) using the native backend + internal assembler
(RTL units carry inline asm). An AIncludeStartup flag controls runtime.start:
the native internal linker includes its bare _start (Blaise owns the entry),
the cc link line omits it (libc's startup provides _start and calls main).
The transitional blaise_rtl.a is still produced by the runtime Makefile and
still consumed by the fixpoint scripts and the driver-bypassing e2e test
helpers; removing those is the remaining Stage 3 work.
All four fixpoints pass (FIXPOINT_OK, NATIVE_FIXPOINT_OK, NATIVE_INTERNAL_OK,
WARMCACHE_FIXPOINT_OK); full suite green (3829 tests) under both the QBE-built
and native-built test runner.
RTL-unification Stage 2 (docs/rtl-unification-plan.adoc): the native internal
linker no longer links a pre-built blaise_rtl.a. Instead it compiles the
implicit RTL units (runtime.* + rtl.platform.*) from source into an object cache
beside the compiler (compiler/target/rtl/*.o) and links those, recompiling a
unit only when its cached .o is missing or older than the source.
- EnsureRTLObjects (native driver) drives the per-unit compile via self-invoke
(blaise --no-incremental --assembler internal --source U.pas), the proven
per-unit path; LinkViaInternalLinker adds the cached .o via AddOwnedObject
instead of AddArchive.
- uToolchain.RTLSourceDir resolves the RTL source robustly: binary-relative
(compiler/target -> ../src/main/pascal), then CWD-relative (handles a binary
copied to /tmp but invoked from the project root, as the fixpoints do).
- New --rtl-src DIR flag (TBackendOpts.RTLSrcDir) + $BLAISE_RTL_SRC for a
relocated/release binary; the driver fails loudly when the RTL source is
unreachable rather than emitting a broken binary.
Performance: warm cache adds ~0ms (RTL skipped); a cold full RTL recompile is
~0.4s. Scope: this changes the NATIVE internal-linker path only — the QBE
backend and cc-link paths still use blaise_rtl.a, so the archive remains a build
artifact (the fixpoint scripts and QBE-based tests still build it).
TestLink_MissingRTL_FailsLoudly rewritten for the new contract (--rtl-src at a
nonexistent dir must fail naming the RTL source). All four fixpoints OK; full
suite 3829 green; cold-bootstrap from the existing release works.
Execution plan for folding the runtime into the compiler: single
pasbuild compile -m blaise-compiler, no separate runtime module, no
make install, no ar-built blaise_rtl.a.
Records the constraining facts (programs emit RTL calls as undefined externals
satisfied by the archive; the archive is a Blaise-only ar artifact pasbuild
does not produce; RTL is now pure Pascal), the chosen target shape (RTL units
relocated into the compiler source tree, compiler produces a cached rtl/*.o dir
the internal linker consumes), and the full blast radius — project.xml,
uToolchain/FindRTLArchive, native driver AddArchive, all four fixpoint scripts,
rolling-bootstrap, CI bootstrap.yml, ~12 tests, release artifacts, and docs.
Recommends DOTTED-FLAT unit names (runtime.atomic, runtime.str, …) over a
subdirectory: the unit loader is filename-based (dotted name -> dotted file, no
subdir traversal), so dotted-flat keeps the units on the compiler's existing
unit-path with zero new path entry, at the cost of a one-time uses-clause
rename sweep; emitted runtime symbols are unchanged either way.
Sequences the migration archive-compatible-first (old release keeps cold-
bootstrapping) and only drops the archive once a stage-2 binary produces/
consumes the .o cache, with a full four-fixpoint + cold-bootstrap re-verify
after each stage and a build-time measurement gate before keeping the
compile-per-target follow-up.
blaise_atomic exported a PInteger = ^Integer type that collides with the same
alias in blaise_arc when the RTL units are compiled together into one program
(the pasbuild library / whole-program path), giving 'Duplicate type name
PInteger'. The per-unit Makefile build never saw it. Type the params as
Pointer (the asm body reads through %rdi regardless); the external-name callers
in blaise_arc are unaffected (link is by symbol). Prerequisite for compiling
the RTL units together (RTL-unification).
Records the runtime end-state: blaise embeds the RTL Pascal source once and
compiles it per --target on demand (selecting that target's TPlatformLayout +
kernel-stub adapter set), linking in-process — so a single binary cross-compiles
to any registered target with no blaise_rtl.a, no ar, no make install, and no
per-target .a files. The archive is documented as the transitional mechanism.
Adds an RTL-unification track to the migration sequence (gated on the .s→inline-
asm migration, which is in progress: blaise_atomic + blaise_setjmp done,
blaise_start + blaise_utf8 remaining). Compile-per-target adopted first;
in-memory/unit-cache RTL object cache deferred until build time is measured.
Reconciles the 'two selection moments' and FreeBSD Step 5 with the no-archive
end-state.
LinkViaInternalLinker always built the linker with the default (Linux)
TLinkTarget, so --target freebsd-x86_64 still emitted a SYSV/Linux ELF. Resolve
the target's toolkit and build TLinker with its MakeLinkTarget, so the emitted
ELF carries the right EI_OSABI / machine / load base for the requested target.
--target freebsd-x86_64 now stamps EI_OSABI = 9 (FreeBSD); the default Linux
target is unchanged (EI_OSABI = 0, still compiles+runs). The FreeBSD binary is
not yet runnable (Strategy-B syscall _start stub lands later) — this is the
OSABI/link-target wiring (Step 1 of the FreeBSD design).
Adds TLinkerE2ETests.TestLink_FreeBSDTarget_StampsOSABI locking in that the
FreeBSD link target produces a static ET_EXEC with EI_OSABI=9.
All four fixpoints OK; full suite 3813 green.
Records the Step 0c outcome: a runtime TKernelABI virtual port (30 abstract
methods + a GKernel global, routing all rtl.platform.posix kernel calls through
it with a libc-delegating Linux adapter) was implemented and reverted. It
destabilised self-hosting — binaries built against the modified RTL segfaulted
at startup with an unwindable/overflowed stack, and the failure compounded
across bootstrap stages rather than converging. The Linux adapter was pure libc
delegation, so it carried real self-hosting risk for zero present benefit.
Replacement design: the kernel leaf is a LINK-TIME symbol swap (the pattern
already used for blaise_setjmp/atomic .s objects). The RTL keeps its
'external name' bindings; the FreeBSD archive links a raw-syscall stub object
exporting the same symbols. No runtime virtual dispatch, no new global, no
change to the layout-sensitive RTL units. The kernel mechanism is a build-time
property of a target (libc-linked vs freestanding, never switched at runtime),
so a link-time swap models it exactly while a runtime port does not.
Both docs updated throughout: ports table, units-that-change table, migration
sequence (0a/0b done, kernel-leaf revised, ARC-walker lift deferred as arm64
groundwork), toolkit sketch, summaries. Adds a [#kernel-leaf] rationale note.
