diff --git a/.gitignore b/.gitignore index 87dbc0b..f7a8762 100644 --- a/.gitignore +++ b/.gitignore @@ -77,3 +77,4 @@ CLAUDE.md # Local bug-tracking notes — not committed bugs.txt .gstack/ +progress-native-backend.txt diff --git a/progress-native-backend.txt b/progress-native-backend.txt deleted file mode 100644 index 66c4eb5..0000000 --- a/progress-native-backend.txt +++ /dev/null @@ -1,238 +0,0 @@ -Blaise Native Codegen Backend — Phase 5 Progress -================================================ - -Plan: ~/.claude/plans/read-docs-toolchain-independence-adoc-an-parallel-rabin.md -Branch: native-codegen - -[x] M0a Declare ICodeGen (uCodeGen.pas); make TCodeGenQBE implement it - (signature-only); collapse Blaise.pas dispatch to run the codegen - sequence once via an ICodeGen var. Full test suite + fixpoint green. - DONE 2026-06-02 — 2293 tests pass; FIXPOINT_OK; valgrind clean. - Also fixed a pre-existing codegen bug found en route: interface-typed - GLOBAL := nil emitted an undefined-symbol store (see bugs.txt). -[x] M0b Port uToolchain.pas (trimmed); add TTargetDesc + TNativeBackend + - TCodeGenNative shell (implements ICodeGen, walks AST); wire - --backend native / --target into Blaise.pas (no codegen yet). - DONE 2026-06-02 — units: blaise.codegen.target / .native / .native.backend - / .native.x86_64 + uToolchain. --backend qbe|native, --target - - (linux-{x86_64,i386,arm64}, freebsd-{x86_64,arm64}, windows-x86_64, - macos-arm64; only linux-x86_64 has a backend). Native compile fails - loudly. 2295 tests pass; FIXPOINT_OK. Also fixed a 2nd pre-existing - codegen bug: whole-record assignment to a var/out record param (bugs.txt). - FreeBSD added as a target OS per request (shares SysV ABI with Linux; - only the link line differs). -[x] M1 program; begin end. compiles+runs via --backend native, exit 0. - DONE 2026-06-02 — TX86_64Backend.EmitProgram emits the $main entry - (frame + _SetArgs + return 0) in AT&T asm; TCodeGenNative.Generate drives - it; CompileAndRunNative added to e2e base (no QBE step). Verified via - strace (only cc invoked, no qbe) and objdump of main. 2296 tests pass; - FIXPOINT_OK. FAsm uses TStringBuilder (stdlib) for append-only assembly - construction (per perf review). -[x] M2 integer arithmetic + WriteLn parity with QBE. - DONE 2026-06-02 — EmitExprToEax (int literals, + - * div mod via - %eax/%ecx, push/pop for nesting, idivl/cltd for div/mod) + EmitStmt / - EmitWrite (_SysWriteInt fd=1, _SysWriteNewline). Output matches QBE - oracle on arithmetic, div/mod, nesting, precedence, negatives, Write - without newline. 2299 tests pass; FIXPOINT_OK. -[x] M3 control flow parity — if/else, while, repeat. DONE 2026-06-02. - Comparison ops (= <> < > <= >=) via cmpl+setcc; EmitCondBranch - (testl + jne/jmp); NewLabel for .L-locals; if/while/repeat lowering. - (`for` landed in M4 with locals.) Matches QBE oracle. -[x] M4 memory/locals — integer program-globals + for loop. DONE 2026-06-02. - Program-level int vars emitted as RIP-relative .data globals (matching - the QBE backend's model for a program's top-level var block); declare/ - assign/read; EmitDataSection collects slots (named globals exported, - hidden .L for-end slots file-local). for to/downto with end evaluated - once (hidden slot), nesting, counter-driven while/repeat. Matches QBE - oracle. 2306 tests pass; FIXPOINT_OK. - Narrow loads/stores already done in M5 (wider integer family). - Records DONE 2026-06-03 — TFieldAssignment (write) and TFieldAccessExpr - (read) for integer-family record fields; local and global record vars. - AddSlot allocates TotalSize (rounded to 8) bytes for record locals. - EmitDataSection emits .skip N / .balign 8 for global records. - EmitProgram registers record-typed globals alongside integer globals. - Field address = base + FieldInfo.Offset: offset=0 uses VarOperand/RIP - directly; non-zero offsets leaq the base into %rcx then use N(%rcx). - 3 new e2e tests (RecordGlobal, RecordLocal, RecordParam) on both backends. - Static arrays DONE 2026-06-03 — TStaticSubscriptAssign (write A[I]:=V) - and TStringSubscriptExpr/tyStaticArray (read A[I]) for integer-family - elements; zero and non-zero LowBound; local and global arrays. - AddSlot uses AType.RawSize for tyStaticArray (rounds to 8). - EmitDataSection emits .skip N for array globals. - EmitProgram registers array-typed globals. - TIdentExpr/tyStaticArray returns leaq base address (not a load). - Element address = base + (I − LowBnd) × ElemSize via imulq/addq. - 3 new e2e tests (GlobalReadWrite, LocalReadWrite, NonZeroLow). - M4 COMPLETE. -[x] M5 function calls — direct calls to user procs/functions. DONE 2026-06-02. - Stack-frame model (params+Result+locals as -N(%rbp) slots; FFrame maps - name->offset); EmitFunctionDef (prologue, spill arg regs to param slots, - body, Result in %eax, epilogue); EmitCall (args in edi/esi/edx/ecx/r8d/ - r9d via push-then-pop, result in %eax); var read/assign route through - VarOperand (local slot vs RIP global); for-loop var also uses VarOperand - so loops work inside functions. Integer value params + integer/void - return. Frame layout mirrors FPC -O- (verified vs 3.2.2 and 3.3.1) and - the QBE oracle. Recursion works. 2309 tests pass; FIXPOINT_OK. - Wider integer family + conversions DONE 2026-06-02 — Byte/Word/SmallInt/ - Int64 (and UInt32/UInt64) as globals, locals, value params and return. - All integer values now flow 64-bit-extended in %rax (sign/zero-extend by - width+signedness on load via movsx/movzx/movslq/movl; re-narrow on store - via %al/%ax/%eax/%rax sub-views). Arithmetic/compare in 64-bit (addq/ - imulq/idivq+cqto, divq+xor %edx for unsigned div/mod). Explicit casts - Byte(X)/Word(X)/Int64(X) lower to EmitNarrowToType (truncate+re-extend). - WriteLn dispatches _SysWriteInt / _SysWriteInt64 / _SysWriteUInt64 by - width+signedness (matches QBE oracle). Frame slots 8-byte-aligned, sized - by type; globals emit .byte/.word/.long/.quad. Per-slot type held in a - TDictionary map (briefly encoded as a small Integer to - dodge a generics defect; the defect is now fixed — bugs.txt "retain - generic templates" — and the map is back to a plain TTypeDesc). 2318 - tests pass; FIXPOINT_OK. New e2e: WiderIntGlobals, Int64Arithmetic, - WiderIntParamsAndReturn, TypeCastConversions, SignednessAndWraparound, - WriteUnsigned32 (Cardinal > 2^31 prints unsigned). - Var/out parameters DONE 2026-06-03 — var/out params pass the caller's - address (leaq for locals/globals); the param slot holds a pointer - (always movq spill); reads dereference through the pointer, writes - store through it; pass-through (var param forwarded to another var - param) loads the pointer from the slot. Uses AST IsVarParam flags - (set by the semantic analyser) rather than a local tracking set — - simpler and consistent with the QBE backend. Wider-int var params - (Int64, Byte) work correctly (the dereference uses the declared - type's width). Existing QBE-only e2e tests upgraded to - AssertRunsOnBoth (SrcVarParamSwap, SrcConstParam). 4 new native e2e - tests: VarParamSwap, VarParamPassThrough, VarParamWiderInt, OutParam. - 2347 tests pass; FIXPOINT_OK. - Break/continue/exit DONE 2026-06-03 — break jumps to the loop-end - label; continue jumps to the increment (for) or condition (while/repeat) - label; FBreakLabels/FContinueLabels stacks pushed before loop body, - popped after. Exit jumps to the function epilogue via FExitLabel - (set per EmitFunctionDef); bare Exit in $main emits leave+ret directly. - Exit(Value) works via the synthesised ResultAssign from the semantic - pass. Existing QBE-only e2e tests upgraded to AssertRunsOnBoth - (ForBreak, ExitFromFunction, ConstInt, ConstNeg, DefaultParam, - TypeCastIntByte). 4 new native e2e tests: ForBreak, WhileContinue, - ExitFromFunction, ExitValueShorthand. 2351 tests pass; FIXPOINT_OK. - >6 args DONE 2026-06-03 — SysV AMD64: args 0-5 in rdi/rsi/rdx/rcx/r8/r9; - args 6-7 on the stack at +16(%rbp)/+24(%rbp) in callee. EmitCall - evaluates all args left-to-right onto the stack, pops stack args into - %r10/%r11 (caller-saved scratch), pops reg args into rdi..r9, then - re-pushes stack args in ABI order (arg N-1 deepest, arg6 on top at callq). - Caller cleans up stack args with addq after the call. BuildFrame assigns - positive %rbp offsets (+16, +24, ...) to stack params so VarOperand emits - N(%rbp) for them. Supports up to 8 total args; more raises a clear error. - AddSlot now stores offsets as negatives for local slots; VarOperand checks - sign to emit -N(%rbp) vs +N(%rbp). 2 new e2e tests (SevenArgs, EightArgs) - run on both backends. 2363 tests pass; FIXPOINT_OK. - Indirect (bare proc/func pointer) calls DONE 2026-06-03 — EmitCallIndirect - loads the function pointer from the variable slot into %r10 (caller-saved - scratch that survives arg evaluation in %rax), pushes/pops args into - registers as for EmitCall, then dispatches via callq *%r10. @FuncName in - expression context emits leaq funcname(%rip), %rax. IntByteSize extended - to return 8 for tyProcedural/tyPointer/tyClass/etc. so proc-pointer globals - get a .quad/.balign 8 slot and are stored/loaded with movq (was movl/long, - truncating the 64-bit address). 2 new e2e tests: BareProc, BareFunc. - Method pointers (of object, 16-byte Code+Data block) deferred — requires - TFieldAssignment support for M.Code/M.Data record fields. - Method-pointer calls (of object) deferred to M7 — requires class - allocation, TFieldAssignment for class/pointer fields, and call dispatch - (Code→%r10, Data→%rdi, shift remaining args). Placeholder test - TestRun_Native_IndirectCall_MethodPtr added: QBE path verified, - native path Ignored with TODO M7. - M5 COMPLETE (float/record/string params+return deferred to M6/M7). - Bugs found en route (bugs.txt): Exit(Value) shorthand unsupported; - nested (local) procedures emit an undefined external; two distinct - instantiations of one generic in a single unit drop a vtable; - TDictionary instantiation fails to link (vtable - not emitted). -[x] M6 floats (s/d) parity. DONE 2026-06-03. - Double/Single literals, globals, locals, arithmetic (+,-,*,/), comparisons - (< > = <> <= >=), WriteLn(D), value params, and function return. - EmitExprToXmm0 for float expressions; EmitLoadFloat/EmitStoreFloat; - float globals in .data (.double/.float directives); float literals in - .rodata (RIP-relative movsd/movss); cvtsd2ss for Double-literal → Single - assignment; EmitCondBranch dispatches ucomisd/ucomiss + CF/ZF jumps for - float comparisons; EmitFunctionDef prologue uses XmmIdx counter for - %xmm0/%xmm1/... spill (independent of IntIdx for int regs); epilogue - loads Result into %xmm0 for float return; EmitCall detects mixed/float - params and uses pre-allocated stack slots + two-pass register loading. - 8 new e2e tests (all AssertRunsOnBoth). 2383 tests; FIXPOINT_OK. - Bugs fixed en route: WriteLn(Double/Single) crashed QBE (wrong arg type - in _SysWriteInt call); boSlash (not boDiv) is the `/` operator for - floats; float literals default to tyDouble so Single assignments need - cvtsd2ss. -[x] M7a sret — record-returning functions. DONE 2026-06-03. - FSretFunc flag in BuildFrame when ResolvedReturnType.Kind = tyRecord; - Result slot is an 8-byte pointer slot (sret buffer addr from %rdi); - prologue spills hidden %rdi → Result slot (IntIdx starts at 1 so - normal params get %rsi/%rdx/...); no Result initialisation (caller - zeroes); epilogue emits plain ret. TFieldAssignment for RecordName='Result' - loads sret ptr from slot and writes through %rcx + field offset. - EmitSretCall: leaq dest → %r10; memset(%r10,0,TotalSize); reload %r10; - push/pop normal args into %rsi...; movq %r10,%rdi; callq func. - TAssignment detects record LHS + record-returning RHS and dispatches to - EmitSretCall. TestRun_Native_RecordReturnFunction promoted from Ignore. - 2383 tests pass; FIXPOINT_OK. -[x] M7b class system — vtables, typeinfo, class allocation, method dispatch, - field access through class pointer, RTTI (MethodAddress). DONE 2026-06-03. - EmitClassMethods iterates AProg.Block.TypeDecls (not ProcDecls) so class - method bodies in CD.Methods are emitted. EmitClassSection emits .data vtables - (typeinfo + virtual method ptrs), typeinfo blocks (8-quad: parent, itab, - name, methods, size, cleanup, vtable, attrs), class-name strings (ARC header - + ASCII null-terminated), method tables (count + name-ptr/code pairs), and - field-cleanup stubs. NativeMangle strips the $ prefix from vtable ImplName - entries (StrAt/StrCopyTail) to match QBE-backend output. Constructor calls - allocate via _ClassAlloc, wire the vtable ptr, then call the named ctor body. - Class variable assignment uses ARC (_ClassAddRef new, _ClassRelease old). - TMethod record fields (Code/Data) accessed via struct-field read/write with - leaq+offset. Method-pointer (of-object) calls dispatch via EmitMethodPtrCall - (leaq slot→%rcx; movq 0(%rcx)→%r10 Code; movq 8(%rcx)→%r11 Data; args into - %rsi/%rdx/...; movq %r11,%rdi; callq *%r10). MethodAddress emits a call to - _MethodAddress(instance, name_ptr) with the class-name string +12 offset. - Method-pointer cast assignment (TFoo(M)) copies 16 bytes via memcpy. - tyProcedural IsMethodPtr globals emit .quad 0 × 2 (16 bytes) in EmitDataSection. - tyClass globals registered and emitted as .quad 0 (8 bytes). FSretFunc flag - unchanged from M7a. TestRun_Native_IndirectCall_MethodPtr promoted from - Ignore to AssertRunsOnBoth. 2383 tests pass. -[ ] M8 full cp.test.e2e suite parity (stdout+exit), QBE vs native -[ ] M9 self-hosting: native-built Blaise --emit-ir == QBE-built Blaise -[ ] M10 (opt) linear-scan regalloc / direct ELF .o emission -[ ] M11 (opt) i386 and arm64 backends (new TTargetDesc + backend subclass each) - -Architecture invariants: -- --emit-ir always uses TCodeGenQBE; its output is byte-identical regardless of - --backend (fixpoint protected). -- uCodeGenQBE.pas text-emit core untouched (only the class declaration adds the - ICodeGen interface). -- Driver holds an ICodeGen and runs the codegen sequence ONCE (no per-backend - duplication of SetSymbolTable/AppendUnit/AppendProgram/Generate). ARC frees it. -- ICodeGen is an interface (Blaise is fully ARC; no lifetime penalty). Backend- - specific setters stay off the interface. -- Native backend generates machine code from the AST directly (no QBE-text parse). -- All pointer-size logic routes through TTargetDesc.PtrSize (no hard-coded 8). -- Target-specific code only in blaise.codegen.native..pas + TTargetDesc. -- Default backend = qbe; default target = host (x86_64-linux). - -Deferred / backlog: -- Units -> TList (Generics.Collections) to drop the TUnit() cast in - Blaise.pas (`for U in Units do`). DEFERRED 2026-06-02: the language constructs - bootstrap fine under v0.9.0, but TObjectList.Create(True) currently OWNS the - TUnit items (freed via Units.Free at Blaise.pas:606) whereas TList is - non-owning — switching shifts TUnit lifetime onto ARC in bootstrap-critical - code, an ownership ripple. Per user guidance, revisit later (e.g. once a fixed - release is stage-1, or alongside a deliberate ownership pass). Cosmetic cast - removal is not worth the double-free/leak risk now. -- Test fixtures via compiler/src/test/resources/ — real .pas unit files instead - of inline heredoc strings for multi-unit e2e tests. DEFERRED 2026-06-02 (own - change): add FTestResourcesPath (ProjectRoot + 'compiler/src/test/resources') - to TE2ETestCase and a CompileAndRunWithResourceUnit helper that puts that dir - on the unit-loader search path; migrate a couple of heredoc tests as examples. - Mirrors the existing real-file convention in compiler/src/it/. Useful as M6+ - programs (records/classes/strings) outgrow inline strings. The current inline - CompileAndRunWithUnit helper (writes a unit to scratch) covers today's needs. - -Notes / residuals: -- Bootstrap workaround in Blaise.pas: no explicit `CG := nil` on the ICodeGen - global (stage-1 v0.9.0 binary mis-compiles interface-global nil stores). The - underlying codegen bug is FIXED in this tree; once a release built from this - tree becomes stage-1, restore the explicit nil-assignments. See bugs.txt. -- releases/v0.10.0-pre updated 2026-06-02 with the fixpoint-verified binary+RTL - from this tree (so subsequent stage-1 builds carry the interface-nil fix).