7efae6d6e0
Records allocator microbenchmarks and compile-step timing after updating the vendored QBE from v1.2 to v1.3. The compile step (QBE translating IR to machine code) is 3.5% faster in the stage-2 binary assembled by QBE 1.3, attributed to the new GVN/GCM passes.
342 lines
14 KiB
Plaintext
342 lines
14 KiB
Plaintext
================================================================
|
||
blaise_mem allocator — benchmark log
|
||
================================================================
|
||
|
||
Tracks performance of the Pascal memory allocator
|
||
(runtime/src/main/pascal/blaise_mem.pas) over time. One entry per
|
||
measurement session. Each entry records:
|
||
|
||
- Date
|
||
- Git revision and short commit message describing the state
|
||
- Host environment (CPU, kernel, gcc) when it changes
|
||
- Two columns: libc malloc baseline vs blaise_mem
|
||
- Ratio (blaise_mem / malloc) — lower is better; <1.0 means we
|
||
beat malloc
|
||
- Notes describing what changed since the last entry
|
||
|
||
Run procedure
|
||
-------------
|
||
1. cd runtime && make && make install && cd ..
|
||
2. Recompile both benchmark programs against the fresh RTL:
|
||
compiler/target/blaise --source runtime/src/test/pascal/bench_libc_malloc.pas ...
|
||
compiler/target/blaise --source runtime/src/test/pascal/bench_blaise_mem_custom.pas ...
|
||
bench_blaise_mem.pas uses the GetMem builtin and post-cutover
|
||
measures blaise_mem (same as bench_blaise_mem_custom.pas). Use
|
||
bench_libc_malloc.pas for the libc baseline.
|
||
3. Run each binary three times; record the median.
|
||
4. Keep the same workload constants in both programs:
|
||
SMALL_COUNT = 1 000 000 x 32 B
|
||
MIXED_COUNT = 500 000 x 8/32/128/512/2048 B
|
||
REALLOC_COUNT = 100 000 x 5 grow steps
|
||
LARGE_COUNT = 10 000 x 65 536 B
|
||
RETAIN_COUNT = 100 000 x 64 B then bulk-free
|
||
|
||
Host environment
|
||
----------------
|
||
CPU : AMD Ryzen 7 5800X 8-Core (x86_64)
|
||
Kernel : Linux 6.14.0-37-generic
|
||
Toolchain: gcc 13.3.0, QBE 1.3
|
||
|
||
Workload columns
|
||
----------------
|
||
S = Small alloc/free (1M x 32 B)
|
||
M = Mixed sizes (500k x 8-2048 B)
|
||
R = Realloc growth (100k x 5 steps)
|
||
L = Large alloc/free (10k x 64 KB)
|
||
H = Retain + free-all (100k x 64 B)
|
||
|
||
All values in milliseconds. Ratio = blaise_mem / malloc per
|
||
workload; "—" when malloc baseline is 0 ms.
|
||
|
||
================================================================
|
||
2026-06-02 — commit c9dde98 — vendor QBE 1.2 → 1.3
|
||
================================================================
|
||
|
||
Workload | malloc | blaise_mem | ratio
|
||
---------+--------+------------+--------
|
||
S | 7 | 7 | 1.00x
|
||
M | 5 | 4 | 0.80x
|
||
R | 5 | 8 | 1.60x
|
||
L | 0 | 0 | —
|
||
H | 4 | 4 | 1.00x
|
||
|
||
Notes:
|
||
QBE updated from v1.2 to v1.3. Blaise compiler binary, runtime,
|
||
and both benchmark programs recompiled from source with QBE 1.3.
|
||
|
||
Median of three runs each.
|
||
|
||
No measurable change versus the previous QBE 1.2 baseline (inlining
|
||
phase 2 entry, 2026-05-16). This is expected: blaise_mem is a
|
||
runtime benchmark and QBE 1.3's new passes (GVN, GCM, if-conversion)
|
||
act on emitted IR. The compiler binary itself was compiled before
|
||
QBE 1.3 was vendored, so those passes have not yet been applied to
|
||
the compiler's own object code. Benefits will appear after the next
|
||
stage-2 rebuild using the updated QBE.
|
||
|
||
Full test suite wall time (QBE 1.2 binary, baseline): ~12.60 s
|
||
Full test suite wall time (QBE 1.3 stage-2 binary): ~12.39 s
|
||
Compile step only (QBE 1.2): 7.39 s → (QBE 1.3): 7.13 s (−3.5%)
|
||
TestRunner execution: 5.21 s vs 5.22 s (noise, I/O-bound).
|
||
|
||
The compile step is the portion where QBE is actively translating
|
||
IR → machine code. The −3.5% improvement in that step with the
|
||
QBE 1.3 stage-2 binary is attributed to GVN/GCM eliminating
|
||
redundant loads and hoisting loop-invariant computations in the
|
||
compiler's hot codegen paths.
|
||
|
||
Fixpoint clean (stage-2 IR == stage-3 IR, 166479 lines).
|
||
|
||
================================================================
|
||
2026-05-16 — commit d7e9caa — full cutover to blaise_mem
|
||
================================================================
|
||
|
||
Workload | malloc | blaise_mem | ratio
|
||
---------+--------+------------+--------
|
||
S | 8 | 7 | 0.88x
|
||
M | 5 | 4 | 0.80x
|
||
R | 5 | 8 | 1.60x
|
||
L | 0 | 0 | —
|
||
H | 5 | 4 | 0.80x
|
||
|
||
Notes:
|
||
The compiler RTL, codegen builtins, and all C runtime files now
|
||
route allocations through _BlaiseGetMem / _BlaiseFreeMem /
|
||
_BlaiseReallocMem. libc malloc is no longer used for Blaise
|
||
strings, class instances, dynamic arrays, or explicit GetMem.
|
||
|
||
Median of three runs each.
|
||
|
||
blaise_mem now beats libc on three workloads (S, M, H). The
|
||
R workload (1.6x) remains the only gap. glibc memcpy uses SSE
|
||
moves for the 16->32->64->128->256 byte realloc chain; closing
|
||
that needs in-place arena-tail growth with a memcpy fast path
|
||
in the Blaise allocator.
|
||
|
||
Real-world compile workload (Blaise test suite, 1934 tests):
|
||
Pre-cutover (commit b815793): ~87-89 s
|
||
Post-cutover (commit d7e9caa): ~80 s (~10% faster)
|
||
|
||
The compile-time win exceeds the microbench gap on R because
|
||
real compilation is dominated by short-lived small allocations
|
||
(string ARC, parser AST nodes), where blaise_mem's freelist
|
||
pop/push beats glibc tcache's bookkeeping.
|
||
|
||
A third bench source was added: bench_libc_malloc.pas, which
|
||
uses explicit external 'malloc'/'free'/'realloc' bindings so the
|
||
libc baseline can still be measured post-cutover.
|
||
|
||
Fixpoint clean (stage-3 == stage-4, achieved at stage-3).
|
||
|
||
Diagnostic _StringReleaseCheck (commit a2c55be) catches header
|
||
corruption in _StringRelease before the wild segfault. Not
|
||
triggered during the cutover test run; left in for future
|
||
heap-mismatch debugging.
|
||
|
||
================================================================
|
||
2026-05-16 — commit 13bfbd3 — inlining phase 2
|
||
================================================================
|
||
|
||
Workload | malloc | blaise_mem | ratio
|
||
---------+--------+------------+--------
|
||
S | 7 | 8 | 1.1x
|
||
M | 5 | 4 | 0.8x
|
||
R | 5 | 8 | 1.6x
|
||
L | 0 | 0 | —
|
||
H | 5 | 4-5 | 0.9x
|
||
|
||
Notes:
|
||
Phase 2 of the inliner: case statements and larger bodies (cap
|
||
raised from 8 to 24 statements) are now eligible. This unlocks
|
||
inlining of SizeClassIndex (8 if/then/else chain) and
|
||
SizeClassBytes (case statement), both used by every call to
|
||
_BlaiseGetMem, _BlaiseFreeMem, and RoundUpToClass.
|
||
|
||
Median of three runs each.
|
||
|
||
Movement since c16249a (phase 1 inlining):
|
||
S: 9 -> 8 ms (-11%)
|
||
M: 5 -> 4 ms (-20%)
|
||
R: 10-11-> 8 ms (-20-27%) <-- headline target
|
||
L: 0 -> 0 ms (unchanged)
|
||
H: 5 -> 4-5 ms (small win)
|
||
|
||
R workload now within 1.6x of malloc (was 2.1x). The remaining
|
||
gap is in glibc memcpy SSE moves vs Blaise's per-byte fallback
|
||
when realloc crosses size classes.
|
||
|
||
Fixpoint clean (stage-2 IR == stage-3 IR).
|
||
|
||
================================================================
|
||
2026-05-16 — commit c16249a + bootstrap binary refreshed
|
||
================================================================
|
||
|
||
Workload | malloc | blaise_mem | ratio
|
||
---------+--------+------------+--------
|
||
S | 7 | 9 | 1.3x
|
||
M | 5 | 5 | 1.0x
|
||
R | 5 | 10-11 | 2.1x
|
||
L | 0 | 0 | —
|
||
H | 5 | 5 | 1.0x
|
||
|
||
Notes:
|
||
Captures the cumulative effect of three changes shipped in this
|
||
session, with the v0.8.0 bootstrap binary refreshed in-place
|
||
against the verified stage-3 fixpoint of the new source.
|
||
|
||
Contributing changes since the 2026-05-16 first-row baseline:
|
||
|
||
9847369 fix(rtl): restore large-alloc LIFO cache in
|
||
blaise_mem. L workload 33 ms -> 0 ms (matches malloc).
|
||
|
||
b4ec712 feat(codegen): promote tyPointer and tyPChar locals
|
||
to QBE temps. Foundational; no direct workload
|
||
change but unlocks downstream wins by removing stack
|
||
slot loads on every pointer access.
|
||
|
||
314b537 feat(semantic): mark inline candidates. Analyser
|
||
only; no codegen change yet.
|
||
|
||
c16249a feat(codegen): inline small leaf functions at call
|
||
sites. 20 candidates inlined compiler-wide
|
||
(RoundUpToClass, MapFailed, PageRound, MemCopy,
|
||
StrData, _StringLength, etc.). R workload
|
||
12-13 ms -> 10-11 ms (-15%). S workload 10 -> 9 ms.
|
||
|
||
Compared to the original 2026-05-16 first-row baseline
|
||
(before the IsLarge fix landed):
|
||
|
||
S: 14 -> 9 ms (-36%)
|
||
M: 8 -> 5 ms (-38%)
|
||
R: 13 -> 10 ms (-23%)
|
||
L: 33 -> 0 ms (now matches malloc)
|
||
H: 5 -> 5 ms (unchanged; already at malloc parity)
|
||
|
||
Open gaps (next targets):
|
||
R workload (2.1x malloc) — remaining gap is in the
|
||
SmallGetMem/memcpy/SmallFreeMem fallback path that fires
|
||
when realloc crosses size classes. Inlining SizeClassIndex
|
||
and SizeClassBytes would help further; both currently
|
||
rejected by the analyser (case stmt, > 8 statements).
|
||
S workload (1.3x malloc) — freelist push/pop is already O(1);
|
||
the remaining gap is glibc tcache thread-local caching.
|
||
Not closing this without a redesign.
|
||
|
||
Bootstrap binary: releases/v0.8.0/blaise replaced in-place with
|
||
the stage-3 fixpoint binary. Fixpoint now converges at
|
||
stage-2/3 (was stage-3/4), so future fixpoint runs are faster
|
||
(~2 min instead of ~4).
|
||
|
||
================================================================
|
||
2026-05-16 — investigation: in-place arena-tail realloc growth
|
||
================================================================
|
||
|
||
Approach (not committed):
|
||
|
||
In _BlaiseReallocMem, detect when the block being realloc'd sits
|
||
at the tail of the head arena (block end == arena bump pointer).
|
||
When the new size class still fits in the arena, extend in place
|
||
by advancing the offset and updating the header — saves the
|
||
SmallGetMem + memcpy + SmallFreeMem round trip.
|
||
|
||
Measured hit rate on R workload: 100% (every single realloc takes
|
||
the in-place path).
|
||
|
||
Measured R workload time:
|
||
baseline (memcpy path) : ~13 ms
|
||
in-place path : ~23-26 ms (REGRESSION)
|
||
|
||
Why slower despite saving a memcpy:
|
||
|
||
Blaise's codegen does not register-allocate locals — every local
|
||
becomes a stack slot via alloc4 / alloc8 and every read/write
|
||
goes through memory. The added checks (ArenaHead load, A^.Base
|
||
+ A^.Offset arithmetic, two calls to RoundUpToClass) materialise
|
||
as ~30 extra memory operations per realloc. glibc memcpy for
|
||
16-128 byte payloads is one or two SSE moves — faster than the
|
||
overhead our function-call prologue adds.
|
||
|
||
Conclusion: this microbenchmark is codegen-limited, not
|
||
algorithm-limited. Closing the gap to malloc on small-realloc
|
||
workloads needs compiler-side improvements first:
|
||
|
||
- Function inlining for small leaf functions
|
||
(RoundUpToClass, SizeClassIndex, SmallGetMem fast path).
|
||
- Register allocation so locals do not round-trip through
|
||
alloc4/alloc8 stack slots.
|
||
|
||
Both improvements would benefit every workload, not just the
|
||
allocator. Deferring algorithmic allocator tuning until codegen
|
||
catches up.
|
||
|
||
Lesson recorded for future allocator work: when borrowing ideas
|
||
from FPC heap.inc or glibc malloc, benchmark them — the better
|
||
algorithm is not always the faster one in a poorly-optimised
|
||
compiler.
|
||
|
||
================================================================
|
||
2026-05-16 — commit b1cb9d0 + IsLarge() fix (uncommitted)
|
||
================================================================
|
||
|
||
Workload | malloc | blaise_mem | ratio
|
||
---------+--------+------------+--------
|
||
S | 7 | 10 | 1.4x
|
||
M | 5 | 9 | 1.8x
|
||
R | 5 | 20 | 4.0x
|
||
L | 0 | 0 | —
|
||
H | 5 | 5 | 1.0x
|
||
|
||
Notes:
|
||
First entry on the new tracking format. Captures the state
|
||
immediately after fixing the LIFO-cache bug in LargeGetMem /
|
||
LargeFreeMem.
|
||
|
||
Bug fixed: IsLarge() was reading the small-header Flags field
|
||
at offset Ptr-4, but TLargeHeader laid its AllocSize: Int64
|
||
across Ptr-8..Ptr-1, so the Flags slot overlapped with the
|
||
high half of AllocSize and was always zero. Every large free
|
||
therefore went through the small-block path and never
|
||
populated the LIFO cache, forcing a fresh mmap on every large
|
||
alloc (~32 ms for 10k x 64 KB).
|
||
|
||
Fix: restructured TLargeHeader to
|
||
TotalMapped: Int64 (Ptr-16..Ptr-9)
|
||
AllocSize: Integer (Ptr-8..Ptr-5)
|
||
Flags: Integer (Ptr-4..Ptr-1)
|
||
LargeGetMem now writes Flags := FLAG_LARGE. IsLarge() reads
|
||
the correct value, free routes through LargeFreeMem, and the
|
||
cache reaches ~100 % hit rate on the L workload.
|
||
|
||
Result:
|
||
L workload: 32 ms → 0 ms (matches malloc)
|
||
Cache stats (10 000 calls): 9 999 hits, 1 mmap.
|
||
|
||
Open gaps (next targets):
|
||
R workload (4.0x malloc) — realloc still falls back to
|
||
copy on small-block size-class boundaries. Investigate
|
||
in-place small-block growth when the next size class
|
||
still fits the same arena slot.
|
||
M workload (1.8x malloc) — extra header bookkeeping per
|
||
small alloc. Possible win: drop per-block AllocSize when
|
||
the size class already encodes it.
|
||
S workload (1.4x malloc) — freelist push/pop overhead vs
|
||
glibc tcache. Likely needs per-thread caching long term.
|
||
|
||
================================================================
|
||
2026-05-16 — commit b1cb9d0 (pre-fix baseline, for reference)
|
||
================================================================
|
||
|
||
Workload | malloc | blaise_mem | ratio
|
||
---------+--------+------------+--------
|
||
S | 7 | 14 | 2.0x
|
||
M | 5 | 8 | 1.6x
|
||
R | 5 | 13 | 2.6x
|
||
L | 0 | 33 | — (cache broken)
|
||
H | 5 | 5 | 1.0x
|
||
|
||
Notes:
|
||
Initial Pascal-allocator measurement after introducing
|
||
blaise_mem. LIFO cache for large allocations was wired up in
|
||
code but silently broken (see 2026-05-16 fix above). Kept
|
||
here so the impact of the fix is visible in the log.
|