Diagnosis credit: Andrew Haines (issue #89) correctly identified that
the subscript-assign path loads through a var/out parameter's slot only
once — the slot holds the ADDRESS of the caller's variable, so element
writes landed in the wrong memory and corrupted the heap. His patch
covered the QBE dyn-array and class cases; the class half had already
landed independently (248dccf). This implements the remaining cases
across BOTH backends in the current tree.
Subscript writes through var/out params (TStaticSubscriptAssign gains
IsVarParam, set by the semantic pass; bif-coverage status updated):
- dynamic arrays: one extra dereference to reach the data pointer
(writes were lost and stray stores corrupted the heap; SetLength and
element reads already worked),
- static arrays: load the array address from the slot instead of
offsetting the slot itself (QBE wrote into the parameter slot region;
the native ident READ also produced garbage — pmVar now treated like
the other by-ref param modes),
- PChar: extra dereference before the byte store (writes were lost).
Interface var/out parameters (previously did not even compile: QBE
emitted loads from a non-existent %_var_G_obj; native mis-spilled the
single incoming pointer as a two-register fat pointer):
- interface variables now occupy ONE contiguous 16-byte fat-pointer
block (obj at +0, itab at +8). QBE locals: a single alloc8 16 with
the _itab name derived at +8; QBE globals: a single 16-byte data item
$Name_obj with the itab half addressed as $Name_obj + 8 (the separate
$Name_itab item could not be guaranteed adjacent). Native locals and
globals were already contiguous.
- new IntfObjAddr/IntfItabAddr helpers route every QBE obj/itab access
(assignment variants incl. weak, dispatch, expr-pair reads, as-out
binding); var/out params dereference the slot first.
- native: var-param-aware receiver load in EmitInterfaceCall, var-param
LHS in EmitInterfaceAssign (shares the sret-Result pointer path),
interface globals usable as var args (leaq Name_obj), and the call
slot counter treats a var interface arg as ONE pointer slot (it was
counted as two, desynchronising the argument register pops).
IR assertions updated for the new global fat-pointer layout. New e2e
tests: TestRun_VarParamDynArray_WriteAndGrow,
TestRun_VarParamStaticArray_PChar,
TestRun_VarParamInterface_DispatchAndReassign.
Closes#89.
Every recLineInfo record carried the program's main source file, so
'break unit.pas:NN' failed with 'No code found' for any line inside a
unit, and callstack frames attributed unit code to the program file
(found trying to break inside kanban.ui.pas in the kanban tool).
TDbgFunc gains a SourceFile field; the native backend stamps it from
TUnit.SourceFile while emitting each unit (empty for the main program,
which keeps the emitter's own fallback). The facts-mode line emission
writes each function's own file into its records. The recLineInfo
format already carries a per-record filename and pdr matches by
basename, so no format or debugger change is needed.
Verified: breakpoints by unit-file:line resolve and fire, parameters
print at the stop, and callstacks name the correct file per frame.
compiler/project.xml now passes --backend native --debug-opdf for the
blaise-compiler module, so local builds of the compiler and TestRunner
exercise the native x86-64 backend end-to-end and are debuggable with
pdr. The shipped default backend remains QBE.
Compiling the compiler itself natively (never done before) exposed two
pre-existing native codegen bugs, both fixed:
- Local dynamic-array variables were not zero-initialised. SetLength
reads the old data pointer and the epilogue releases it, so stack
garbage in the slot corrupted the heap — the unit→.o iface-embed
path crashed in the allocator. Dyn-array locals now start nil like
string/class/record locals. New e2e test
TestRun_Native_LocalDynArray_DirtyStack pins this with a
dirty-stack helper.
- P[I] := Chr(N) stored the low byte of the _Chr-allocated STRING
POINTER instead of N: the native backend lacked the QBE backend's
EmitByteRhs short-circuit. Added EmitByteRhsToEax (Chr folds to its
argument, single-char literals to their ordinal) and applied it at
every byte-sized store site (PChar subscript, dyn/static array
byte elements, P^ byte writes, byte array-field elements). This
corrupted every ELF header patch in uElfObject, producing .o files
with garbage e_shoff/e_shnum.
With both fixes the full separate-compilation round trip works under
the all-native toolchain and the complete suite passes (2959 tests)
with a natively-built compiler and TestRunner.
uDebugOPDF: interface-typed globals are two labels (Name_obj/_itab);
recGlobalVar now references Name_obj so --debug-opdf links for
programs with interface globals (e.g. the compiler's own CG).
pdr now resolves the ASLR slide correctly (load base from the
binary's offset-0 mapping), so the -no-pie guard in both native link
paths is no longer needed. Debug binaries are position-independent
again, matching the platform default.
Verified under live ASLR: breakpoints, var-param drilldown, captured
vars, dynamic arrays, TList<T> inspection, callstack and stepping all
work against PIE binaries.
Remove the 'PIE (ASLR) support in PDR' section from
future-improvements.adoc — implemented.
OPDF/native debugging:
- uDebugFacts: TDbgVar.Indirect — the frame slot holds the value's
ADDRESS rather than the value (var/out params, captured outer locals).
- uDebugOPDF: emit LocationExpr=3 'RBP-relative indirect' for such
slots; mangle generic instance names in class/property records
(vtable_TList_Integer — raw 'TList<Integer>' broke the assembler);
set OPDF_FLAG_DYNARRAY_LEN32 in the header so debuggers read the
[refcount:Int32][length:Int32] dynamic-array header correctly.
- native: DbgMarkParams marks var/out params and _cap_ capture slots
indirect; captured vars are typed from the enclosing declaration
(OuterVarType via FDbgOuterDecl).
Verified with pdr: var-param record AND class field drilldown inside
the callee, captured-var inspection in nested procedures, dynamic
array printing and subscripting, TList<T> instance inspection.
Codegen bugs uncovered by the debug probes, fixed in both backends:
- var-param class field read/write used a single deref: the slot holds
the caller variable's address, so code treated that address as the
instance pointer — reads returned garbage and writes corrupted the
caller's frame. All field-access leaf sites now load twice when
IsClassAccess and IsVarParam (QBE: EmitInstancePtr/EmitLValueAddr/
EmitFieldAssignment/EmitExpr leaves; native: the parallel
RecordName-based receiver loads).
- QBE nested-proc signatures emitted a double comma when a proc has
captured vars AND regular params (l %_cap_X, , w %_par_K).
- native: assignments to captured string/dynarray/class/record/float
vars wrote to a global of the same name instead of through the
capture pointer; captured float reads did the same.
- native: 'var D: Double' params were classified as xmm args on both
sides of a call — the pointer arrived in an xmm register and the
callee stored through garbage. Var-param floats now travel in
integer registers like every other by-ref param.
New e2e tests: TestRun_VarParamClass_FieldReadWrite,
TestRun_NestedProc_CaptureAndParams, TestRun_VarParamFloat_CapturedFloat.
Three gaps between the OPDF data and a realistic multi-unit program:
- Type records for unit-defined classes referenced bare symbols
(vtable_TStringList, TStringList_SetText) while the emitted symbols
carry the owning-unit prefix — any program using TStringList failed
to LINK under --debug-opdf. New MangledClassSym mirrors the
codegen's ClassSymName allowlist (System / rtl.* / blaise_* /
program-scope stay bare) for VMTAddress and property accessor
addresses.
- Self was recorded untyped (its slot is allocated as a raw pointer),
so 'print Self.FField' failed. DbgMarkParams now types Self as the
owning class — pdr dereferences class-typed variables, so instance
field drilldown works. The unit-less driver path now passes the
program's symbol table to the backend (the analyser, which the
table's uses-chain provider points at, is still alive there; the
in-test fallback variant dangled and was reverted).
- Value record parameters present their '_data' shadow slot (the
callee's inline copy) as the parameter instead of the raw ABI
pointer slot — record fields are inline at known offsets, which is
exactly what field access needs.
Verified with pdr against a program using classes+sysutils units:
print P.X / P.Y (record param fields), Self.FCount / Self.FName
(instance fields, string rendering), Msg (local string) all correct.
Suite: 2955 OK; FIXPOINT_OK; NATIVE_FIXPOINT_OK.
The OPDF emitter was a standalone AST walk that GUESSED runtime facts:
local offsets ignored parameter slots and native slot sizing, every line
record pointed at the function entry, and HighPC was a next-function
approximation. The native backend owns all of those exactly — so it now
produces them.
New uDebugFacts unit: under --debug-opdf the native backend records, per
emitted function, the assembly symbol (LowPC), an end label after the
final ret (exact HighPC), every frame slot with its real %rbp offset
(parameters flagged with var/const, internal bookkeeping slots
filtered), and a .Ldbg_N local label per statement with its source
line/column. TOPDFEmitter gains a facts mode that emits scopes, params
(plus a locatable recLocalVar per parameter), locals and per-STATEMENT
recLineInfo from those facts; without facts (QBE backend — QBE assigns
frames and addresses itself) it keeps the approximate AST walk.
The driver appends the OPDF section to the SAME assembly file for the
native backend: local labels resolve within one object, so nothing needs
exporting. Debug links pass -no-pie (OPDF stores absolute addresses;
the pdr loader does not yet apply a PIE slide).
Verified end to end with the pdr reference debugger against a native
binary: 'break file.pas:line' hits mid-function, 'step' advances one
source line at a time, 'print <param>/<local>' reads correct values via
the real RBP offsets, and 'callstack' shows named frames with lines
(TThing_Bump at dbg.pas:11 <- main at dbg.pas:25). None of these worked
before. Normal builds are byte-identical (labels only under OPDF mode;
both fixpoints clean).
Tests: facts-mode emitter assertions (exact HighPC, real offsets,
statement-label line records) + native backend facts collection
(.Ldbg labels, param/local offsets, label-free normal builds).
Suite: 2954 OK; FIXPOINT_OK; NATIVE_FIXPOINT_OK.
Three OPDF emitter gaps from the debugger-feedback backlog:
- Class and record METHODS (destructors included) never got
recFunctionScope records: after semantic analysis the implementation
bodies live on the type declarations' method lists, while
EmitFunctionScopes only walked Block.ProcDecls (whose impl entries are
body-less matching stubs). The walk now gathers both sources; method
records carry the qualified label (TThing_Bump) as the record name —
the symbol a debugger resolves for break-by-function-name — and as
LowPC. Plain functions are unchanged (label = name).
- Under --debug-opdf every function symbol is now exported (ExportPrefix
and EmitFuncDef honour FOpdfMode): the .opdf companion is assembled as
a separate object whose scope records reference function labels, so
local symbols failed to link the moment a method got a scope record.
Same rationale as the existing VTableDataPrefix export.
- Generic-instance vtables bypassed VTableDataPrefix and stayed local
under --debug-opdf.
Verified end to end: a program with a virtual-method class compiles,
links and runs with --debug-opdf (previously the documented blocker).
Still open (bugs.txt): per-statement line records need codegen-emitted
statement labels — the line table remains function-start granularity.
Tests: TestOPDF_MethodScope_Emitted + TestOPDF_DestructorScope_Emitted.
Suite: 2950 OK; FIXPOINT_OK; NATIVE_FIXPOINT_OK.
obj.Field := MakeClass() leaked one reference per store: class-returning
functions and methods uniformly transfer +1 (the callee retains into
Result and never releases it — verified for constructor results, locals,
globals, fields and chained calls on both backends), but the class-field
store sites retained unconditionally on top of that transfer.
The ExprOwnsRef guard is now applied to every class-field store:
QBE EmitFieldAssignment, native TFieldAssignment (ObjExpr, sret-Result,
RecordName branches) and the native implicit-Self assignment — matching
the long-guarded string/dyn-array paths and QBE's implicit-Self path.
The historical blocker (a stage-2 compiler built with this guard
corrupted its own heap) was a TAIL of pre-existing over-releases that
the tighter refcounts exposed. Found and fixed with a temporary
negative-refcount abort in _ClassRelease:
- Typed exception handlers injected one synthetic local PER 'on E:'
clause: two same-named handlers double-allocated the slot and emitted
one epilogue release each, while only the matched body bound (+1) —
the exception's refcount went negative on scope exit. The synthetic
local is now deduplicated by name (uSemantic).
- Handler re-binding leaked the previous binding: the shared slot is now
released before each bind (both backends).
Verified shapes: ExprOwnsRef=True can only be user functions/methods/
getters/indirect calls — inlining excludes class returns and no external
returns a class, so every owning report is a genuine +1.
Compiler self-compile leak count drops 14 -> 6 objects. Tests:
TestDebug_ClassFieldFromCall_NoLeak + TestDebug_MultiHandlerVar_
NoOverRelease (leak-tracker asserted, both backends). Suite: 2948 OK on
working and fixpoint binaries (stage-2); FIXPOINT_OK; NATIVE_FIXPOINT_OK;
full native TestRunner OK.
The FTypeXxx caches (FTypeInteger, FTypeString, ...) held an extra
retained reference on top of FAllTypes' owning reference — NewType
returns +1 and the strong field store retained again, so every builtin
TTypeDesc ran one refcount high for the whole compilation.
With the [Unretained] store lowering in place (the owned +1 from the
NewType call is released at the assignment site), the caches are now
plain borrowed views into the type pool, which matches the ownership
story: FAllTypes owns the single reference, everything else borrows.
A previous attempt at this marking (pre-dating the Unretained
temp-release lowering) regressed leak counts; with the prerequisite in
place the leak report is unchanged (14 residual objects, none of them
type-pool related) and all gates pass on a stage-2 rebuild.
Suite: 2946 OK (stage-2 binary); FIXPOINT_OK; NATIVE_FIXPOINT_OK;
full natively-compiled TestRunner OK (2946).
Resolves the TStringList.Objects / TList<T> retention question from the
bug backlog as an explicit design decision (language-rationale,
'Collection Ownership'):
- TList<T>/TStack<T>/TQueue<T> managed elements ARE retained on store —
this already works (generic stores lower to ARC-aware pointer writes);
TE2ETListTests.TestRun_TList_ClassElements_RetainedAcrossScope now
pins it on both backends (object survives its creating scope).
- TStringList.Objects[] stays NON-OWNING by design: the API type is
Pointer and the integer-cast idiom (TObject(PtrUInt(N)), used in 27
places in the compiler itself) makes blind retention impossible.
Convention documented at the declaration and in the rationale.
- Known limitation recorded: generic Clear/Destroy do not yet release
remaining managed elements (needs a Default(T)-style zero-store);
tracked in the leak backlog.
Suite: 2946 OK; FIXPOINT_OK; NATIVE_FIXPOINT_OK.
Two native miscompiles, found chasing the generic-template in-memory
bif round-trip crash (TIfaceIOTests.TestRoundTrip_Generic*_TemplateParams):
1. .Free() on an l-value receiver expression (Def.ClassDef.Free()) only
released — the slot was not nilled (the QBE lowering stores 0). The
stale pointer aliased the next same-size allocation, so the following
ARC field store double-released the NEW object; its owned lists died
while still referenced (ReadGenericClassPayload's Free+Create+assign
dance left ClassDef.Fields freed → nil deref in the test). Field and
ident receivers now release AND nil via EmitLValueSlotAddr; other
expression receivers keep the release-only behaviour.
2. A record-returning call assigned into an array element
(t[i] := FLexer.Next()) evaluated the RHS via EmitExprToEax, which
cannot pass the hidden sret pointer — the callee wrote its Result
record through garbage (TLexerTests crashed; the suite had never run
natively because the round-trip crash aborted earlier). Both
subscript-assign branches now sret record-returning calls directly
into the element after releasing the old element's managed fields.
The FULL natively-compiled TestRunner passes for the first time:
2944 tests, 0 failures (previously aborted at TIfaceIOTests).
Tests: TNativeConstArgTests.TestFree_FieldReceiver_NilsSlot (asm-level)
+ TE2ERecordsTests.TestRun_RecordCallResult_IntoArrayElement (both
backends). Suite: 2945 OK; FIXPOINT_OK; NATIVE_FIXPOINT_OK.
H.S.Note(); failed semantic analysis with "Class 'IShape' has no method
'Note'" — AnalyseMethodCall's receiver-expression path accepted
interface-typed receivers in its kind check but then resolved the method
against TRecordTypeDesc unconditionally. Interface receivers now get
their own branch (mirror of the implicit-Self interface path): validate
against the interface's method list and mark the call for itab dispatch.
The QBE backend already lowered the shape; the native EmitMethodCallStmt
site additionally failed to pass ObjExpr to EmitInterfaceCall, emitting
bogus _obj/_itab operands from the empty ObjectName — it now forwards
the receiver expression like EmitMethodCallExpr does.
Test: TE2EClasses2Tests.TestRun_InterfaceFieldCall_StatementPosition
(both backends). Suite: 2943 OK; FIXPOINT_OK; NATIVE_FIXPOINT_OK.
Open-array arguments were lowered correctly only at standalone
procedure/function call sites. Method calls (statement and expression
paths, static and virtual) and constructor calls emitted just the data
pointer — the callee read its (data, high) pair shifted, so High(A) and
the elements were garbage (F.Sum([10, 20, 30]) printed junk; the
statement form printed a wrong count).
New OpenArrayArgFragment helper emits the 'l data, l high' pair for
inline array literals, static-array coercions, and open-array parameter
forwarding; the four method/constructor argument loops now use it.
The native backend already handled all these shapes.
Test: TE2EOpenArrayTests.TestRun_OpenArray_MethodAndCtorParams —
constructor, function and procedure (statement) open-array params on a
class, both backends. Suite: 2942 OK; FIXPOINT_OK; NATIVE_FIXPOINT_OK.
A function-of-object variable called in EXPRESSION position went through
EmitCallIndirect, which loads only the Code pointer and passes the first
user argument in %rdi — Data (Self) from offset +8 of the TMethod block
was never loaded and the argument registers were not shifted. The
STATEMENT path already dispatched on IsMethodPtr to EmitMethodPtrCall;
the expression path now does the same.
Repro: any 'function(N: Integer): Integer of object' variable call in an
expression — QBE was correct, native segfaulted.
Test: TE2EMiscTests.TestRun_FunctionOfObject_IndirectCall (both
backends). Suite: 2941 OK; FIXPOINT_OK; NATIVE_FIXPOINT_OK.
Interfaces may declare properties (FPC/Delphi parity):
IValued = interface
function GetValue(): Integer;
procedure SetValue(AValue: Integer);
property Value: Integer read GetValue write SetValue;
end;
Accessors must be methods of the interface or an inherited parent
(interfaces have no fields), validated at registration. I.Value reads
lower to the existing zero-arg getter itab dispatch; I.Value := X
lowers to the setter dispatch with X as the single argument — pure
compile-time sugar, no itab slots, no layout change. Child interfaces
see inherited properties via the parent chain. Wired end to end:
parser, TInterfaceTypeDef.Properties (AST + clone), TInterfaceTypeDesc
property registry, semantic read/write resolution, both backends, .bif
serialisation and import registration. v1 limits (recorded in
language-rationale): plain interface-typed receivers; no indexed/
default array properties.
Two pre-existing linking bugs surfaced by the dogfood program and are
fixed alongside:
- Property accessor names written in a different case than the method
declaration (read getValue for GetValue) produced unresolved symbols —
accessor names are now normalised to the declared casing at
registration (classes and interfaces, compile and import paths).
- A program-level class implementing an interface failed to link
whenever the program had a uses clause: program-scope methods carry
bare symbol names (uSemantic.CurrentUnitPrefix) but itabs and
property-setter call sites prefixed them with the program name via
Sym.OwningUnit. ClassUnitPrefix (QBE) / ClassSymName (native) now
skip the prefix for program-owned classes (new FProgramName field).
Tests: 7 in cp.test.interfaces (parse, registration, accessor
validation, read-only enforcement, inheritance, IR dispatch), bif
round-trip in cp.test.unitinterface, 2 e2e suites on both backends in
cp.test.e2e.classes2 (interface read/write incl. compound assignment
and inherited dispatch; case-mismatch + uses regression). Suite: 2940
OK on working and fixpoint binaries; FIXPOINT_OK; NATIVE_FIXPOINT_OK.
Sin(12) and Tanh(I) now compile: the float compiler builtins (Sqrt,
Ceil/Floor/Round/Trunc, Ln/Log2/Log10, Power, the trig family, IsNaN/
IsInfinite) accept integer-family arguments and widen them to Double —
matching FPC/Delphi and Blaise's own implicit int→float assignment
rule. The trig builtins return Double for integer arguments (still
Single→Single / Double→Double for float arguments). Power previously
type-checked nothing and emitted invalid IR for integer (and Single)
arguments; it now validates and coerces both to double.
Double(I) / Single(I) typecasts were lowered as bit copies: QBE emitted
an integer temp into 'stored' (rejected by qbe), native stored an
integer register — so the Tanh(Single(I)) workaround failed too. Float
casts now emit real conversions on both backends (swtof/sltof/ultof/
uwtof + exts/truncd on QBE; cvtsi2sd/ss + cvtss2sd/cvtsd2ss natively),
including float→float width changes.
Fixing this exposed two pre-existing native float-width bugs: a Single
RHS assigned to a Double variable was stored without cvtss2sd, and
mixed-width binary operands (s * 1000 — integer literals are emitted as
.double constants) ran the Single binary path at the wrong width.
Added EmitXmm0WidthAdjust and applied it at assignment and to both
binary operands.
Decision recorded in docs/language-rationale.adoc (Implicit
Integer→Float Widening). Tests: 8 IR/semantic tests in cp.test.math
(the two RejectInteger tests inverted to acceptance) + 2 e2e tests on
both backends in cp.test.e2e.math. Suite: 2930 OK on working and
fixpoint binaries; FIXPOINT_OK; NATIVE_FIXPOINT_OK.
Array-typed FIELDS were second-class citizens for element access; this
lands the full variation family on both backends:
- Semantic: r.A[i] := v dropped the subscript from the LHS type — the
parser stores it in TFieldAssignment.PropIndexExpr (the indexed-
property slot) and semantic ignored it for real fields, demanding the
whole array type on the RHS. A subscript on a real dyn/static-array
field is now an ELEMENT write (new semantic-set IsElemWrite flag);
both backends emit the element store with the standard ARC and
record-copy rules.
- Parser: c.N.A[i] := v failed with "Expected 'end' but got '['" —
the chained L-value walker now accepts a terminating Field[idx] :=,
and the subscript-chain path accepts arr[i].A[j] := v (subscript
directly over another subscript stays a clear parse error).
- Bare implicit-Self: A[i] := v inside a method raised "Undeclared
variable 'A'" — TStaticSubscriptAssign now resolves array-typed
fields of Self (IsImplicitSelf + ImplicitFieldInfo).
- SetLength(r.A, n): QBE refused ("first argument must be a
variable"); native silently emitted NO code for field receivers and
mis-stored through var-param receivers. QBE routes through
EmitLValueAddr; native gains EmitLValueSlotAddr covering field,
var-param and implicit-Self receivers for dyn-array and string
SetLength.
- Read side: c.A[i] through a class variable computed the element base
as if c were an inline record (missing object-pointer load) and
segfaulted; implicit-Self bases had the same gap; native missed
chained reads (c.N.A[i]) entirely. All base shapes are handled in
the IsArrayAccess read paths of both backends now.
bif-coverage.status regenerated for the new semantic-set AST fields
(safe). Tests: 4 IR tests (cp.test.dynarray) + 6 e2e tests on both
backends (cp.test.e2e.records) covering record/class/implicit-Self
receivers, nested chains, static-array fields and string-element ARC.
Suite: 2922 OK on working and fixpoint binaries; FIXPOINT_OK (single
round); NATIVE_FIXPOINT_OK.
Dynamic arrays of records were broken in three interlocking ways:
- Parser: a[i].Field := v (and a[i].Method, chained a[i].F.G := v) on
the statement LHS raised "Expected ':=' but got '.'" — the subscript
statement branch only accepted ':=' directly after ']'. It now
builds a subscript-rooted postfix chain ending in a TFieldAssignment
(via ObjExpr) or TMethodCallStmt (via ObjExpr). grammar.ebnf gains
SubscriptFieldAssign / SubscriptMethodCall rules.
- Both backends: a[i] := r stored the ADDRESS of r into the element
instead of copying the record, and element reads loaded the first
8 bytes of the element as if it were a pointer. The two bugs masked
each other (elements aliased r — the TElfSection workaround comment
in uElfObject.pas documents the symptom). Record-element subscript
reads now yield the element address (dyn/open/static arrays) and
writes do an ARC-aware fieldwise copy: EmitRecordCopy on QBE,
retain-src/release-dest/memcpy on native. Native static arrays of
records had the same read/write bug and are fixed too.
- QBE backend: Exit inside the SECOND (or later) try block of a
function skipped _PopExcFrame, leaving a stale g_exc_top that
corrupted later raises/pops. EmitTryFinallyStmt/EmitTryExceptStmt
emitted normal and exception paths sequentially but decremented the
codegen-time FExcDepth on both (net -1 per try statement). Ported
the native backend's rebalancing (restore depth before emitting the
exception path). This is the likely root cause of the historical
'avoid bare Exit inside try' convention.
Tests: 3 IR tests (cp.test.dynarray), 4 e2e tests on both backends
(cp.test.e2e.records), IR pop-count + e2e regression for the exc-frame
bug (cp.test.exceptions, cp.test.e2e.exceptions). Suite: 2912 OK on
working and fixpoint binaries; FIXPOINT_OK; NATIVE_FIXPOINT_OK.
Records how the landed implementation differs from the plan: a single
unified hoist region (EmitArgHoist) for open-array literals,
record-call sret buffers and string pin slots, and shape-based
protection at unknown-signature sites instead of a TInterfaceTypeDesc
const-flag extension.
Port the QBE backend's shape-aware const-string argument convention to
the native x86-64 backend: const params of any kind no longer get the
callee entry-retain/exit-release pair; the caller protects string
arguments by shape (borrowed / consume / pin — see
blaise.codegen.arcshapes.TConstArgMode). Both backends now share one
convention, which also makes native calls into the QBE-compiled RTL
consistent for const params.
The pin slots live in a unified pre-call hoist region (EmitArgHoist)
shared with open-array literal blocks and — new — the sret buffers of
record-returning call arguments. Every call path emits the same
region + post-call epilogue (string releases, record-temp field
releases, reclaim) with the call's return registers preserved:
direct calls, method calls (both strategies), constructor calls,
interface dispatch (var flags from TInterfaceTypeDesc mark positions
to skip; shape protection is applied to every string-typed value arg
since const-ness is invisible there), proc-pointer and method-pointer
calls (TProcParamInfo const flags), inherited calls, and the three
sret call paths (dest pointer saved below the region so %rsp-relative
dest operands cannot drift).
The address-taken walkers move to the shared unit
blaise.codegen.arcshapes (QBE IR output is byte-identical); the native
backend rebuilds its borrowed-argument blocklist per function from
them, mirroring the QBE backend's FConstArgUnsafe.
Bugs found by the new tests and the natively-compiled TestRunner,
fixed here (native backend, all pre-existing):
- record-returning calls as arguments interleaved their sret buffer
between pushed argument slots (garbage args); the hoist fixes this
and EmitSretArgReleases is gone
- callee prologue copied record/static-array value params via memcpy
before spilling later argument registers (memcpy clobbered them);
spills now run in two phases
- >6-slot method calls forwarded var/out params by slot address
instead of the held pointer (no ParamMode check)
- TStringLiteral ignored IsCharCoerce: S[I] = ',' compared the
literal's data pointer against the byte
- EmitNarrowToType truncated pointer-shaped values (class casts) to
32 bits
- IntByteSize defaulted nil/unmapped kinds to 4 bytes (QBE uses 'l');
forward-class fields loaded half a pointer
- EmitCallIndirect/EmitMethodPtrCall loaded the target into %r10/%r11
before argument evaluation (caller-saved clobber window)
- EmitCall's push/pop loop used a stale ParamType from the counting
loop
Tests: cp.test.nativeconstarg (12 asm-level shape tests incl. the
callee-pair flip) and cp.test.e2e.constarg (21 runtime parity tests on
both backends). Full suite 2903 OK on the working and fixpoint
binaries; FIXPOINT_OK; NATIVE_FIXPOINT_OK; natively-compiled
TestRunner green except two pre-existing generic-template round-trip
failures (bugs.txt).
With the default-argument statement bug fixed, the borrowed-local
elision blocked in 91f44ec is sound and enabled: a const-string argument
that is a plain local or by-value/const parameter variable emits no
caller pin at all — the frame's own reference outlives the call.
Exclusions guard the aliasing edges: address-taken locals (explicit @,
passed to var/out params), locals captured by nested procedures (both
via a per-function blocklist rebuilt in EmitVarAllocs), and calls whose
signature has a var/out string param (F(L, L) lets the callee release
L's buffer through the alias). Nine IR contract tests cover every
shape, including the new exclusion guards.
Tail wins on the remaining profile:
- QBEMangle: fast path returns the input unchanged when it contains no
mangled characters (1.3M calls were rebuilding clean names one concat
per character).
- TStringList.FindSorted: compares the probe slot in place through the
CompareStr/CompareText builtins instead of copying it into a local
and dispatching through Compare — several ARC ops per binary-search
step on the hottest lookup path.
- _StringFormatN: the sizing pass uses a pure DecimalWidth count
instead of rendering every %d argument twice.
Compiler self-compile: 0.78 s -> 0.60 s wall, 8.60G -> 6.45G
instructions. Verified through two self-hosted generations, gen-2
native emission, both fixpoints and the full suite (2870 tests).
AnalyseMethodCall (TMethodCallStmt) never called AppendDefaultArgs, so a
method-call statement that omitted defaulted parameters was emitted with
fewer arguments than the callee's parameter list — the callee then read
whatever the missing slots happened to contain. For parameters past the
six SysV integer registers that meant caller-frame stack junk: the
native backend's own EmitInterfaceCall (defaulted 7th param AObjExpr)
read garbage instead of nil, and only survived because the old register
allocation upstream happened to leave benign values there. Expression
calls, proc-call statements and implicit-Self calls already appended
defaults; the three resolution sites in AnalyseMethodCall now do too
(AppendDefaultArgs is idempotent).
Pinned by cp.test.defaultargs.pas: explicit-receiver statement,
seventh-slot default, implicit-Self statement, and proc statement.
Const-string args were pinned (AddRef before the call, Release after)
regardless of shape. ConstArgMode now classifies them: string literals
and named consts are immortal and emit no ARC ops at all; +1 owned
temps (function/method/getter returns) emit a single post-call Release
that consumes the transferred reference — previously these leaked one
buffer per call (the pin pair netted to zero and EmitOwnedArgReleases
deliberately skipped const-string params); every other shape keeps the
pin. The hot TStringList lookup signatures (Compare, KeyEquals,
HashOf, Find, FindSorted, IndexOf) become const.
Borrowed elision for plain local variables — the larger win — is
deliberately NOT enabled: it is sound at the ARC level, but removing
the balanced pin pairs changes caller register allocation and exposes a
latent use-before-def of a callee-saved register in the native
backend's EmitInterfaceCall (a second-generation compiler then reads
the caller's leftover %r13 as an AST pointer and crashes). The full
repro and bisect trail are recorded in the bug log; the camPin branch
carries a pointer to it.
Verified through two self-hosted generations plus both fixpoints;
contract pinned by 7 IR tests in cp.test.constarg.pas.
WriteDecimal/WriteDecimalU heap-allocated a 22-byte digit scratch on
every call, the IntToStr family allocated a second per-call scratch on
top, and _StringFormatN allocated and freed a buffer per %d argument in
both of its passes — four allocator round-trips per formatted integer
on one of the hottest runtime paths (every emitted temp name goes
through IntToStr). All nine sites now use fixed stack arrays; one
shared scratch serves both _StringFormatN passes.
Compiler self-compile: 8.73G -> 8.65G instructions.
The generic keyed containers used documented linear key scans. They now
build the same lazy open-addressing index as TStringList: threshold 16,
kept at <= 50% load, maintained on Add, invalidated by Remove/Clear/
Destroy (storage stays insertion-ordered, so TOrderedDictionary indexed
access is unchanged).
Hashing an arbitrary key type works through monomorphisation: the
generic bodies call GCHashOf(Key), which resolves per instantiation
against new overloads (string hashes its bytes FNV-1a, matching the
content semantics of '=' on strings; Integer/Int64/Pointer use a
multiplicative mix). Instantiating a keyed container with a type that
has no GCHashOf overload is a compile-time error; the threshold const
lives in the interface because generic bodies are analysed at their
instantiation site.
Neutral on the compiler self-compile (its dictionaries are small); this
is stdlib quality for user programs. Contract pinned by 7 new
in-process tests (cp.test.maphash.pas) written against the linear
implementation first.
_StringReleaseCheck was a separate 4-argument call on every string
release (~350M per compiler self-compile) and showed up as 12.7% of all
instructions on its own. The three sanity checks (double-free, length,
capacity) now run inline in _StringRelease with DiagAbort as the cold
path; the diagnostics remain always-on.
Compiler self-compile: 9.2G -> 8.7G instructions, 0.80 s -> 0.78 s.
Every standalone call resolved overloads by scanning the whole
FProcIndex with SameText (twice: IndexOf + ResolveStandaloneOverload),
and every method call scanned FMethodIndex once per inheritance hop
(2.35% of the compile after the TStringList hash landed). FProcGroups/
FMethodGroups now map each key to a TObjectList holding every decl
registered under it, so resolution, the registration-time duplicate
check and impl-signature matching fetch all candidates with one hashed
IndexOf. FProcIndex.Objects[] rewrites go through
ReplaceProcIndexObject so the groups see iface->impl swaps.
The groups are additionally held in an owning TObjectList keep-alive:
TStringList.Objects stores raw, non-retained pointers, so a group whose
only strong reference was a local would be released at the creating
routine's exit and its memory recycled (manifested as a crash resolving
Exception.Create during self-compile). TObjectList itself retains on
Add and releases on Put/Clear/Destroy. The non-retaining behaviour of
TStringList.Objects and generic TList<T> class elements is recorded in
the bug log.
Compiler self-compile: 0.91 s -> 0.80 s, 10.5G -> 9.2G instructions.
Profiling the compiler self-compile (--emit-ir, callgrind) showed 86% of
all instructions inside linear scans of five unsorted TStringLists
(FUnitSymbols 68%, FMethodIndex 12%, FStrLits 4%, FUnitIfaces 2%);
TStringList.Find averaged 145 Compare calls per lookup, and each
iteration paid six ARC ops through Compare's by-value string params.
Unsorted lists with >= 16 entries now build a lazy open-addressing hash
index on first Find: FNV-1a over the bytes, case-folded when the list is
case-insensitive, kept at <= 50% load so an empty slot terminates every
probe. Duplicate keys keep the first-added index, preserving the
IndexOf contract that overload registration relies on. Appends update
the table in place; Delete/Insert/Put/Clear/CustomSort and the
Sorted/CaseSensitive setters invalidate it for lazy rebuild. Sorted
lists keep using binary search; small lists keep the linear scan.
Compiler self-compile: 4.75 s -> 0.91 s wall (5.2x), 71.6G -> 10.5G
instructions (6.8x). Behaviour contract pinned by 12 new in-process
tests in cp.test.stringlistfind.pas, written against the old linear
implementation first.
Five native-backend fixes found while bringing up the natively-compiled
TestRunner:
- Bare class-name expressions (IsMetaclassRef) now emit the typeinfo
address (leaq typeinfo_<T>(%rip)), matching the QBE backend's
'copy $typeinfo_<T>'. Previously they fell through to a global
variable load of an undefined symbol, breaking RegisterTest(TFoo).
Also: tyMetaClass and tySet accepted in the function param/return
type guards, and ForceDirectories added to the TProcCall handler.
- ClassCreate: the constructor symbol now goes through
MethodEmitNameNative (unit-prefixed mangling) instead of raw
OwnerTypeName_Create, and the _ClassCreate result is kept in %rbx
across the constructor call — constructors are plain procedures and
do not return Self in %rax.
- Class-name string blobs (__cn_*) now carry the bare class name as
content while keeping the unit-prefixed symbol name. Previously the
prefixed name was embedded, so ClassName/TestClassName returned
'unit_TClass' and the runner's --suite filters never matched.
- Const string params now get the callee-side entry-retain/exit-release
pair. _StringConcat returns an rc=0 transient; QBE pins it caller
side (EnsureConstStringRef), the native backend had no protection, so
any retaining callee further down the chain freed the transient while
still in use — heap corruption that crashed the semantic pass on
'class of' expressions.
- Static-array locals are memset to zero on entry, mirroring QBE's
EmitVarAllocs. Element assignment into arrays of managed types
releases the old element; without the memset that release read stack
garbage (TProcess slots in RunFilteredTests).
- Open-array literal arguments are hoisted before the argument pushes
(HoistOALitArgs + BeginCallArgs/PushCallArg/EndCallArgs). The old
in-loop EmitOpenArrayLiteral subq'd its element block in the middle
of the push sequence, so the popq loop that loads the argument
registers popped from inside the block — Outer('gcc', ['-o', B, A])
received AExe='-o'. FMethodArgExtraStack is gone (its only writer
was the in-loop literal emission, and it was reset by nested call
sites mid-argument-evaluation).
Three new e2e tests cover metaclass values on both backends (including
the multi-unit RegisterTest pattern) and an open-array literal that is
not the first argument.
The native backend rejected MethodAddress calls where the second
argument was not a string literal. Evaluate the expression normally
and pass the result (already a data pointer) to _MethodAddress, matching
the QBE backend.
Also convert remaining old-style single-quote concatenation source
constants in e2e tests to triple-quote text blocks for readability.
scripts/fixpoint-native.sh verifies that the native backend is fully
self-hosting: stage-0 (QBE-compiled compiler) emits native assembly,
the linked stage-1 binary recompiles itself via --backend native to
produce stage-2, and stage-2 recompiles itself to produce stage-3.
A clean fixpoint means stage-2 and stage-3 assembly are identical.
Both fixpoints (QBE and native) are now required before every commit.
EmitExprToEax uses %rcx as a scratch register for binary expressions,
but the open-array, dynamic-array, static-array, and PChar element
access paths all saved the base address in %rcx before calling
EmitExprToEax for the index expression. A computed index like [I + 1]
would clobber %rcx, causing the element address computation to
dereference a garbage pointer.
Use pushq/popq to protect the base address across the EmitExprToEax
call, matching the pattern already used by the string-subscript path.
Same fix applied to the @Array[I] address-of paths.
This was the last blocker for the native fixpoint — the native-compiled
compiler can now fully recompile itself with --backend native and
produce identical assembly at stage-2 and stage-3.
The native backend's EmitWrite hardcoded fd=1 for all _SysWrite* calls,
ignoring the StdErr file argument. WriteLn(StdErr, 'msg') emitted the
integer 2 as a literal followed by the message on stdout instead of
routing to file descriptor 2.
Mirror the QBE backend's StdErr detection: if the first argument is the
StdErr identifier, set fd=2 and skip it in the argument loop.
Three related bugs in the native x86-64 backend where record data was
written to the pointer slot (leaq = address-of-slot) rather than through
the pointer (movq = load-pointer-then-deref):
1. EmitSretCall / EmitMethodSretCall: when the LHS of a record-returning
function call is a var/out parameter, pass ASretIsIndirect=True so the
memset and reg-return capture target the caller's buffer, not the
local stack slot holding the pointer.
2. EmitRecordRegReturnCapture: honour a new AIsIndirect flag — emit
movq (load pointer) instead of leaq (address-of-slot) for all
classification shapes (rcInt1..rcSSEInt).
3. EmitExprToEax for TIdentExpr with tyRecord/tyStaticArray: when the
identifier is a const/value record parameter (ParamMode = pmRecordValue
or pmStaticArrayValue), the frame slot holds a pointer to the data
copy, not inline data — emit movq instead of leaq.
Bug 1+2 caused out-param record assignments to overwrite the pointer
with the return value. Bug 3 caused const-record field assignments
(e.g. FTarget := ATarget in SetTarget) to copy a stack address into the
field instead of the record data. Both crashed the native-compiled
compiler when using --backend native.
Added E2E test: TestRun_Native_RecordSret_OutParam.
Replace the hardcoded two-backend AssertRunsOnBoth with a set-driven
design that scales to any number of backends:
- TBackends = set of TBackend; AllBackends constant
- AssertRunsOnAll: iterates AllBackends (replaces AssertRunsOnBoth)
- AssertRunsOn(ABackends: TBackends; ...): run on a specific subset
- BackendName(TBackend): string for assertion labels
When a new backend (e.g. LLVM) is added to TBackend and AllBackends,
all 232 AssertRunsOnAll call sites automatically exercise it.
Align the QBE backend unit names with the dotted naming convention
already used by the native backend (blaise.codegen.native.*).
uCodeGen.pas → blaise.codegen.pas
uCodeGenQBE.pas → blaise.codegen.qbe.pas
Updated all uses clauses (~70 test files, main compiler units),
documentation references (README + 5 docs/*.adoc files), and
comment references in runtime/stdlib.
EmitMethodDef and EmitFuncDef called ClassifyRecordReturn four times on
the same return type (signature, declaration, prologue, epilogue).
Classify once into a local RC variable and pass it to each helper.
The per-call-site classification in EmitRecordReturnCallSite remains
unchanged — it classifies a different type (the callee's) each time.
FIXPOINT_OK, 2831 tests pass.
Implement SysV AMD64 ABI register-return classification in the native
x86-64 backend, matching the QBE backend's record-return support.
Small POD records (≤16 bytes, no managed fields) now return in registers
(rax/rdx for INTEGER, xmm0/xmm1 for SSE) instead of always using the
hidden sret pointer. Classification covers all seven SysV shapes:
rcInt1, rcInt2, rcSSE1, rcSSE2, rcIntSSE, rcSSEInt, and rcWin64Agg.
Changes:
- Move TRecReturnClass enum to uCodeGen.pas (shared by both backends)
- Add classification helpers to TNativeBackend (IsRecordManagedClean,
IsRecordAllIntegerLeaves, ClassifyRecordReturn, etc.)
- BuildFrame: reg-return records allocate Result as a local buffer
(not an sret pointer slot)
- EmitRecordReturnEpilogue: load Result buffer into return registers
- EmitRecordRegReturnCapture: store return registers into dest buffer
- EmitSretCall/EmitMethodSretCall: dispatch through reg-return path
when callee qualifies
- EmitFieldAddrToRcx: new helper for computing record/class field
addresses (needed by float field reads)
- EmitExprToXmm0: handle TFieldAccessExpr and TIntLiteral
- Fix float field assignment in TFieldAssignment (was using integer
load/store path for Single/Double fields)
- Fix Single argument passing (add cvtsd2ss when Double→Single
conversion needed)
Also adds 3 missing QBE E2E tests (nested-record, two-Single, managed-
field sret) and 9 native E2E tests covering all register-return shapes.
FIXPOINT_OK, 2831 tests pass.
Classify record returns per SysV x86-64 §3.2.3 instead of using sret
unconditionally. Small POD records (1-16 bytes, integer and/or float
leaves) now travel in registers (rax/rdx, xmm0/xmm1), matching how C
compilers return small structs. Records with managed fields (string,
class, interface, dynarray — checked recursively) always stay on sret.
Win64 delegates to QBE's -t amd64_win ABI pass via uniform aggregate
type declarations.
Based on Andrew Haines' qbe_record_byval_return branch, rebased onto
master and adapted. Added FlattenRecordToAggLetters to correctly
classify nested-record field types in QBE aggregate declarations.
39 IR tests (cp.test.recordret) + 12 E2E tests (cp.test.e2e.recordret).
2819 tests, FIXPOINT_OK.
Track string and dynamic-array buffer leaks in --debug builds.
Strings are registered in _StringAddRef (on the 0→1 transition)
and removed in _StringRelease; likewise for _DynArrayAddRef/Release.
The leak report prints "string" or "dynarray" as the type tag
instead of a class name.
Sentinel-based type tags (GLTTagString=Pointer(2), GLTTagDynArray=
Pointer(3)) distinguish buffer entries from class entries in the
hash map. The report reads the correct refcount header offset
for each type (12-byte string header, 8-byte dynarray header,
CLASS_HDR for classes).
LT_BUCKETS grows from 1024 to 4096 to accommodate the higher
entry count when strings/dynarrays are tracked. Report wording
changes from "object(s) not released" to "leak(s) not released".
2768 tests pass (2 new e2e tests), FIXPOINT_OK.
Extend _LeakTrackerRegister from 2 to 4 args (UserPtr, ClassName,
UnitName, Line) so the --debug leak report prints where each leaked
object was allocated. Both QBE and native backends now emit the
extra unit-name string literal and AST line number at every
_ClassAlloc site. Leak report output changes from:
- TBox (rc=1)
to:
- TBox (rc=1) at LeakSite:14
Runtime: bucket size grows from 16 to 32 bytes; LTInsert stores
unit-name ptr + line; _LeakTrackerReport prints the " at unit:line"
suffix.
Native backend: plumb FDebugMode from TCodeGenNative through
TNativeBackend; emit _LeakTrackerEnable in $main; emit
_LeakTrackerRegister at both constructor-call paths.
QBE backend: add FCurrentUnitName, set in Generate/GenerateUnit/
AppendUnit/AppendProgram; extend both _LeakTrackerRegister call
sites with EmitStrLit(FCurrentUnitName) + node.Line.
2766 tests pass (2 new e2e tests), FIXPOINT_OK.
When a record-returning function call is passed directly as an argument
to another call (e.g. Consume(MakeRec('hello'))), the caller allocates
an sret buffer on the stack. After the outer call returns, the managed
fields (strings, classes, dyn-arrays, interfaces) inside that buffer
must be released — otherwise they leak.
The QBE backend already handled this via EmitOwnedArgReleases; the
native backend lacked the equivalent. Add EmitSretArgReleases which
iterates args in reverse, computes each sret buffer's stack offset,
and calls EmitRecordFieldReleases to release every managed leaf.
Wire it into all call emission paths: EmitCall, EmitCallIndirect,
EmitMethodCallStmt, EmitMethodCallExpr, EmitInheritedCall,
EmitInterfaceCall, EmitInterfaceFieldCall, EmitMethodPtrCall, and
the implicit-self inline call sites.
Weak interface variables (both backends):
- Native: EmitInterfaceAssign routes [Weak] interface locals/globals
through _WeakAssign (no refcount) instead of _ClassAddRef/_ClassRelease.
Scope-exit cleanup emits _WeakClear for both interface and class weak
locals. EmitGlobalReleases checks IsWeakGlobal for program globals.
- QBE: already had full weak support (no changes needed).
Type-cast of interface variable (both backends):
- TVal(S) where S is an interface var must load from S_obj (the obj
half of the fat pointer), not bare S (which doesn't exist as a symbol).
QBE: EmitExpr type-cast path loads from VarRef_obj.
Native: EmitExprToEax type-cast path uses IntfObjOperand.
Previously crashed at link time with "undefined reference to S".
Two new e2e tests: IntfFieldFromFunc (sret into class field) and
WeakInterfaceVar (weak assignment + scope cleanup + destruction ordering).
Both backends now handle storing the result of an interface-returning
function into a class field:
H.F := MakeIntf(42); { explicit field assignment }
F := MakeIntf(42); { implicit-Self field inside a method }
QBE backend: EmitInterfaceToFieldSlots gains an IsInterfaceCall check
that allocates a 16-byte sret buffer, calls EmitRecordCallSret, loads
obj+itab from the buffer, releases the old field obj, and stores new —
no extra AddRef since the callee already retains into the sret buffer.
Native backend: both EmitInterfaceAssign (implicit-Self path) and
EmitInterfaceToFieldSlotsAt (explicit field path) gain TFuncCallExpr
branches that call EmitIntfSretCall, release the old obj from the
field, and copy obj+itab from the stack buffer into the field slots.
Previously both backends raised an error for this combination — the
sret convention for interface returns (commit c8ce6f7) only covered
local/global variable targets.
Both QBE and native backends now support functions/methods that return
an interface type. The return uses sret (struct-return) convention:
the caller allocates a 16-byte buffer (obj+itab), passes its address as
a hidden first argument, and the callee writes both fat-pointer slots
into the buffer.
QBE backend:
- Function signature adds hidden l %_par__sret, function becomes void
- Result_obj aliases sret+0, Result_itab aliases sret+8
- Epilogue is plain ret (no return value)
- Caller-side: alloc 16-byte buffer, EmitRecordCallSret, load obj+itab
- New IsInterfaceCall() helper and assignment handler for the sret path
Native backend:
- BuildFrame marks interface returns as FSretFunc (same as records)
- EmitIntfSretCall: allocates stack buffer, passes as %rdi, calls func
- EmitInterfaceAssign: new sret-Result path dereferences the Result
pointer to write obj+itab into the caller's buffer
- Assignment handler loads obj+itab from sret buffer after the call
Previously, functions returning interfaces crashed the QBE backend
(undeclared split-slot names in IR) and were unsupported in native.
Extends the ARC element-assignment pattern to static arrays: when
assigning to A[I] where the element type is string or class, emit
AddRef(new)/Release(old) before the store, matching the QBE backend
and the dynarray path added in the previous commit.
When assigning to a dynamic array element where the element type is
managed (string or class), the native backend now emits AddRef on the
new value and Release on the old value before storing, matching the
QBE backend's behaviour.
Previously A[I] := 'new' would store without releasing the old string
at A[I], leaking one reference per overwrite.
E2E test TestRun_Native_DynArrayElemArc_String verifies string element
assignment with replacement on both backends.
