feat(lexer): conditional compilation with predefined BLAISE (issue #131)

Implements real symbol-presence conditional compilation in the lexer.
Previously {$IFDEF} was hardcoded false (always took {$ELSE}) and
{$DEFINE}/{$UNDEF} were silently consumed, with no define table and no
command-line flag.

- A case-insensitive define table on TLexer.  {$DEFINE sym} / {$UNDEF sym}
  add and remove symbols; {$IFDEF sym} keeps its body when defined (and
  {$IFNDEF sym} when not), with an optional {$ELSE} and a closing {$ENDIF}.
  IFDEF/IFNDEF blocks nest (the existing depth-tracking skip helpers are
  reused, now driven by the real truth value).
- Predefined symbols, seeded in every lexer: BLAISE (the headline
  cross-compiler use case — {$IFDEF BLAISE} ... {$ELSE} ... {$ENDIF}) plus
  the target CPU/OS symbols CPUX86_64, CPUAMD64, LINUX, UNIX.  No version
  macro yet.
- A -d / --define <sym> command-line flag (FPC -dSYM / Delphi -D), carried
  on TFrontEndOpts and threaded to the program's lexer AND to every unit the
  TUnitLoader compiles, so {$IFDEF} resolves consistently across the program
  and its units.

Tests:
- cp.test.lexer.pas: predefined BLAISE keeps the body, undefined takes ELSE,
  IFNDEF, DEFINE-then-IFDEF, UNDEF-then-IFDEF, and AddDefine (the -d path).
- cp.test.e2e.misc.pas (dual-backend): the {$IFDEF BLAISE} cross-compiler
  pattern, and a combined DEFINE/UNDEF/IFNDEF/CPU-OS/nested program.

Docs: grammar.ebnf documents the directives and predefines; language-
rationale.adoc records the decision (symbol-presence only, no {$IF} expr
form yet; BLAISE + CPU/OS predefined, no version macro) and alternatives.

Verified: FIXPOINT_OK, NATIVE_FIXPOINT_OK, NATIVE_INTERNAL_OK; full suite
green on both the QBE-built and native-built runners (3715 tests).
This commit is contained in:
Graeme Geldenhuys 2026-06-20 20:16:16 +01:00
parent 2a6d0c5912
commit d91d9ee65a
8 changed files with 307 additions and 12 deletions

View file

@ -93,6 +93,8 @@ begin
WriteLn(FormatFlagLine('--output <path>', 'Output binary path'));
WriteLn(FormatFlagLine('--unit-path <dir>',
'Add directory to unit search path (repeatable)'));
WriteLn(FormatFlagLine('--define <sym> | -d <sym>',
'Define a conditional-compilation symbol (repeatable)'));
WriteLn(FormatFlagLine('--backend <id>', BackendUsageLine()));
WriteLn(FormatFlagLine('--target <os>-<cpu>',
'linux-x86_64 (default), linux-i386, linux-arm64,'));
@ -143,6 +145,17 @@ end;
cross-cutting knobs a backend driver reads (Target, OPDFEnabled,
DebugMode, UseInternalAsm). Returns False (and writes a diagnostic) on a
bad flag; the caller owns and frees both objects regardless. }
{ Apply each -d/--define symbol in ADefines to ALexer's conditional-compilation
table. No-op when ADefines is nil/empty. }
procedure AddDefinesTo(ALexer: TLexer; ADefines: TStringList);
var
I: Integer;
begin
if ADefines = nil then Exit;
for I := 0 to ADefines.Count - 1 do
ALexer.AddDefine(ADefines.Strings[I]);
end;
function ParseArgs(AFront: TFrontEndOpts; AOpts: TBackendOpts;
APendingFlags, APendingArgs: TStringList): Boolean;
var
@ -185,6 +198,13 @@ begin
Inc(I);
AFront.SearchPaths.Add(ParamStr(I));
end
else if ((Arg = '--define') or (Arg = '-d')) and (I < ParamCount()) then
begin
{ Define a conditional-compilation symbol (FPC -dSYM / Delphi -D).
Visible to IFDEF directives in the program and every unit it compiles. }
Inc(I);
AFront.Defines.Add(ParamStr(I));
end
else if Arg = '--skip-dep-codegen' then
{ Omit dep unit bodies from the main codegen pass — every cross-
unit call becomes an extern reference. Caller is responsible
@ -622,6 +642,7 @@ begin
try
try
Lexer := TLexer.Create(Source.Text, SourceFile);
AddDefinesTo(Lexer, Front.Defines);
Parser := TParser.Create(Lexer);
IsUnitMode := Parser.IsUnitTopLevel();
if IsUnitMode then
@ -642,12 +663,12 @@ begin
begin
if IsUnitMode and (TopUnit.UsedUnits.Count > 0) then
begin
Loader := TUnitLoader.Create(SearchPaths);
Loader := TUnitLoader.Create(SearchPaths, Front.Defines);
Units := Loader.LoadAll(TopUnit.UsedUnits);
end
else if (Prog <> nil) and (Prog.UsedUnits.Count > 0) then
begin
Loader := TUnitLoader.Create(SearchPaths);
Loader := TUnitLoader.Create(SearchPaths, Front.Defines);
Units := Loader.LoadAll(Prog.UsedUnits);
end;
if Units <> nil then

View file

@ -41,6 +41,7 @@ type
SourceFile: string;
OutputFile: string;
SearchPaths: TStringList; { owned }
Defines: TStringList; { owned — conditional-compilation symbols from -d }
EmitIfaceDir: string;
Incremental: Boolean;
UnitCacheDir: string;
@ -69,10 +70,12 @@ implementation
constructor TFrontEndOpts.Create;
begin
Self.SearchPaths := TStringList.Create();
Self.Defines := TStringList.Create();
end;
destructor TFrontEndOpts.Destroy;
begin
Self.Defines.Free();
Self.SearchPaths.Free();
inherited Destroy();
end;

View file

@ -15,7 +15,7 @@ unit uLexer;
interface
uses
SysUtils, uPasTokeniser, uStrCompat;
SysUtils, Classes, uPasTokeniser, uStrCompat;
type
TTokenKind = (
@ -125,16 +125,25 @@ type
private
FTok: TFpgPascalTokeniser;
FFilename: string;
FDefines: TStringList; { conditional-compilation symbols, case-insensitive }
function MapKeyword(const AUpper: string): TTokenKind;
function UnescapeString(const ARaw: string): string;
function ProcessTextBlock(const ARaw: string): string;
function DirectiveName(const AText: string): string;
function DirectiveArg(const AText: string): string;
function IsDefined(const ASym: string): Boolean;
procedure DefineSymbol(const ASym: string);
procedure UndefSymbol(const ASym: string);
procedure SeedPredefines;
procedure SkipToElseOrEndif;
procedure SkipToEndif;
public
constructor Create(const ASource: string; const AFilename: string = '');
destructor Destroy; override;
property Filename: string read FFilename;
{ Define a conditional-compilation symbol before lexing (e.g. from the
-d command-line flag). Case-insensitive. }
procedure AddDefine(const ASym: string);
function Next: TToken;
end;
@ -157,14 +166,56 @@ begin
FTok := TFpgPascalTokeniser.Create();
FTok.SetSource(ASource);
FFilename := AFilename;
FDefines := TStringList.Create();
FDefines.CaseSensitive := False; { conditional symbols are case-insensitive }
Self.SeedPredefines();
end;
destructor TLexer.Destroy;
begin
FDefines.Free();
FTok.Free();
inherited Destroy();
end;
{ Seed the always-on predefined conditional symbols. BLAISE identifies this
compiler (the headline use case: cross-compiler IFDEF BLAISE / ELSE / ENDIF);
the CPU/OS symbols mirror the FPC family so portable source compiles
unchanged. Targets beyond Linux/x86-64 will extend this set. }
procedure TLexer.SeedPredefines;
begin
Self.DefineSymbol('BLAISE');
Self.DefineSymbol('CPUX86_64');
Self.DefineSymbol('CPUAMD64'); { FPC's alias for the same target }
Self.DefineSymbol('LINUX');
Self.DefineSymbol('UNIX');
end;
function TLexer.IsDefined(const ASym: string): Boolean;
begin
Result := FDefines.IndexOf(ASym) >= 0;
end;
procedure TLexer.DefineSymbol(const ASym: string);
begin
if (ASym <> '') and (FDefines.IndexOf(ASym) < 0) then
FDefines.Add(ASym);
end;
procedure TLexer.UndefSymbol(const ASym: string);
var
Idx: Integer;
begin
Idx := FDefines.IndexOf(ASym);
if Idx >= 0 then
FDefines.Delete(Idx);
end;
procedure TLexer.AddDefine(const ASym: string);
begin
Self.DefineSymbol(UpperCase(ASym));
end;
function TLexer.MapKeyword(const AUpper: string): TTokenKind;
begin
if AUpper = 'PROGRAM' then Result := tkProgram
@ -392,6 +443,35 @@ begin
end;
end;
{ The argument after the directive name, upper-cased: for an IFDEF Foo directive
returns 'FOO'. Skips the name and any spaces, stops at the closing brace or
whitespace. Returns '' when there is no argument. }
function TLexer.DirectiveArg(const AText: string): string;
var
I, C: Integer;
begin
Result := '';
I := 2; { skip '{$' }
{ skip the directive name }
while I < Length(AText) do
begin
C := OrdAt(AText, I);
if (C = Ord('}')) or (C = Ord(' ')) then Break;
I := I + 1;
end;
{ skip spaces between name and argument }
while (I < Length(AText)) and (OrdAt(AText, I) = Ord(' ')) do
I := I + 1;
{ read the argument up to the closing brace or whitespace }
while I < Length(AText) do
begin
C := OrdAt(AText, I);
if (C = Ord('}')) or (C = Ord(' ')) then Break;
Result := Result + UpCase(Chr(C));
I := I + 1;
end;
end;
// Skip tokens from inside a FALSE conditional block until the matching
// ELSE or ENDIF at depth 1 (handles nesting). Stops after consuming the token.
procedure TLexer.SkipToElseOrEndif;
@ -459,19 +539,31 @@ begin
begin
text := FTok.TokenText();
dname := DirectiveName(text);
if dname = 'IFDEF' then
if dname = 'DEFINE' then
Self.DefineSymbol(DirectiveArg(text))
else if dname = 'UNDEF' then
Self.UndefSymbol(DirectiveArg(text))
else if dname = 'IFDEF' then
begin
{ Blaise defines no FPC-specific symbols — always skip the body }
SkipToElseOrEndif();
{ Keep the body when the symbol is defined; otherwise skip to the
matching ELSE/ENDIF. }
if not Self.IsDefined(DirectiveArg(text)) then
SkipToElseOrEndif();
end
else if dname = 'IFNDEF' then
begin
{ Symbol is never defined in Blaise — always execute the body }
{ Keep the body when the symbol is NOT defined. }
if Self.IsDefined(DirectiveArg(text)) then
SkipToElseOrEndif();
end
else if dname = 'ELSE' then
SkipToEndif() { we were in the true branch; skip the else }
{ Reaching an ELSE in normal token flow means the preceding IFDEF/IFNDEF
branch was taken (we only get here from inside an active branch), so
skip the else body to the matching ENDIF. }
SkipToEndif()
;
{ ENDIF and all other directives: silently consumed }
{ ENDIF and all other directives (e.g. H+, mode ...): silently
consumed. }
Continue;
end;
Break;

View file

@ -29,7 +29,8 @@ type
TUnitLoader = class
public
constructor Create(const ASearchPaths: TStringList);
constructor Create(const ASearchPaths: TStringList;
const ADefines: TStringList = nil);
destructor Destroy; override;
{ Returns an owned TObjectList of TUnit in dependency order (leaves
first). The caller is responsible for freeing the list.
@ -54,6 +55,7 @@ type
property PrebuiltObjectPaths: TStringList read FPrebuiltObjectPaths;
private
FSearchPaths: TStringList; { not owned }
FDefines: TStringList; { not owned — conditional symbols for each unit's lexer }
FLoading: TStringList; { units currently on the load stack — cycle detection }
FLoadedNames: TStringList; { units already fully loaded }
FResult: TObjectList; { the in-progress output list (not owned here) }
@ -214,11 +216,17 @@ var
SL: TStringList;
L: TLexer;
P: TParser;
DI: Integer;
begin
SL := TStringList.Create();
try
SL.LoadFromFile(APath);
L := TLexer.Create(SL.Text, APath);
{ Apply the same -d/--define symbols the main program got, so IFDEF
directives resolve consistently across the program and all its units. }
if FDefines <> nil then
for DI := 0 to FDefines.Count - 1 do
L.AddDefine(FDefines.Strings[DI]);
try
P := TParser.Create(L);
try
@ -316,10 +324,12 @@ begin
end;
end;
constructor TUnitLoader.Create(const ASearchPaths: TStringList);
constructor TUnitLoader.Create(const ASearchPaths: TStringList;
const ADefines: TStringList = nil);
begin
inherited Create();
FSearchPaths := ASearchPaths;
FDefines := ADefines;
FLoading := TStringList.Create();
FLoading.CaseSensitive := False;
FLoadedNames := TStringList.Create();

View file

@ -131,6 +131,11 @@ type
procedure TestRun_Const_Int64BitPattern;
procedure TestRun_Const_UInt64BitPattern;
procedure TestRun_Const_UntypedAboveInt64_IsUInt64;
{ Conditional compilation (issue #131): predefined BLAISE, DEFINE/UNDEF,
IFDEF/IFNDEF/ELSE/ENDIF, nesting. }
procedure TestRun_Ifdef_PredefinedBlaise;
procedure TestRun_Ifdef_DefineUndefIfndefNested;
end;
implementation
@ -1233,6 +1238,44 @@ begin
AssertRunsOnAll(Src, '200 -7 250' + LE, 0);
end;
procedure TE2EMiscTests.TestRun_Ifdef_PredefinedBlaise;
const
{ The headline cross-compiler use case: BLAISE is predefined. }
Src = '''
program P;
begin
{$IFDEF BLAISE}
WriteLn('blaise')
{$ELSE}
WriteLn('other')
{$ENDIF}
end.
''';
begin
if not ToolchainAvailable() then begin Ignore('toolchain unavailable'); Exit; end;
AssertRunsOnAll(Src, 'blaise' + LE, 0);
end;
procedure TE2EMiscTests.TestRun_Ifdef_DefineUndefIfndefNested;
const
{ DEFINE/UNDEF, IFNDEF, a predefined CPU/OS symbol, and nesting together. }
Src = '''
program P;
{$DEFINE FOO}
{$UNDEF FOO}
begin
{$IFDEF FOO}WriteLn('foo'){$ELSE}WriteLn('no-foo'){$ENDIF};
{$IFNDEF BAR}WriteLn('no-bar'){$ENDIF};
{$IFDEF LINUX}
{$IFDEF BLAISE}WriteLn('linux-blaise'){$ENDIF}
{$ENDIF}
end.
''';
begin
if not ToolchainAvailable() then begin Ignore('toolchain unavailable'); Exit; end;
AssertRunsOnAll(Src, 'no-foo' + LE + 'no-bar' + LE + 'linux-blaise' + LE, 0);
end;
procedure TE2EMiscTests.TestRun_Const_LargeHex_NotTruncated;
const
{ $080808080808 = 8830587504648, needs 64 bits — must not truncate to 32. }

View file

@ -99,6 +99,14 @@ type
procedure TestSeq_VarDecl;
procedure TestSeq_Assignment;
procedure TestSeq_ProcCall;
{ Conditional compilation (issue #131) }
procedure TestIfdef_PredefinedBlaise_KeepsBody;
procedure TestIfdef_Undefined_TakesElse;
procedure TestIfndef_Undefined_KeepsBody;
procedure TestDefine_ThenIfdef_KeepsBody;
procedure TestUndef_ThenIfdef_TakesElse;
procedure TestAddDefine_KeepsBody;
end;
TParseIntLiteralTests = class(TTestCase)
@ -565,6 +573,63 @@ begin
AssertEquals('EOF', Ord(tkEOF), Ord(t[4].Kind));
end;
{ Conditional compilation (issue #131). Each lexes a snippet with an
IFDEF/ELSE/ENDIF and asserts which identifier survives — proving which branch
the lexer kept. }
procedure TLexerTests.TestIfdef_PredefinedBlaise_KeepsBody;
var tok: TToken;
begin
{ BLAISE is always predefined, so the IFDEF body (yes) is kept. }
SetLexer('{$IFDEF BLAISE}yes{$ELSE}no{$ENDIF}');
tok := FLexer.Next();
AssertEquals('kept branch', 'yes', tok.Value);
end;
procedure TLexerTests.TestIfdef_Undefined_TakesElse;
var tok: TToken;
begin
SetLexer('{$IFDEF NOPE}yes{$ELSE}no{$ENDIF}');
tok := FLexer.Next();
AssertEquals('else branch', 'no', tok.Value);
end;
procedure TLexerTests.TestIfndef_Undefined_KeepsBody;
var tok: TToken;
begin
SetLexer('{$IFNDEF NOPE}yes{$ELSE}no{$ENDIF}');
tok := FLexer.Next();
AssertEquals('ifndef body', 'yes', tok.Value);
end;
procedure TLexerTests.TestDefine_ThenIfdef_KeepsBody;
var tok: TToken;
begin
SetLexer('{$DEFINE FOO}{$IFDEF FOO}yes{$ELSE}no{$ENDIF}');
tok := FLexer.Next();
AssertEquals('defined body', 'yes', tok.Value);
end;
procedure TLexerTests.TestUndef_ThenIfdef_TakesElse;
var tok: TToken;
begin
{ UNDEF removes a predefined symbol, so the ELSE branch is taken. }
SetLexer('{$UNDEF BLAISE}{$IFDEF BLAISE}yes{$ELSE}no{$ENDIF}');
tok := FLexer.Next();
AssertEquals('undefed body', 'no', tok.Value);
end;
procedure TLexerTests.TestAddDefine_KeepsBody;
var tok: TToken;
begin
{ AddDefine mirrors the -d command-line flag. SetLexer creates the lexer;
AddDefine then defines DEBUG so the IFDEF body is kept. }
SetLexer('{$IFDEF DEBUG}yes{$ELSE}no{$ENDIF}');
FLexer.AddDefine('DEBUG');
tok := FLexer.Next();
AssertEquals('cli-defined body', 'yes', tok.Value);
end;
{ Hex literals }
procedure TLexerTests.TestIntLit_Hex_Lowercase;

View file

@ -147,7 +147,23 @@ CARET = "^" ; (* pointer dereference / pointer type prefix *)
WHITESPACE = { " " | "\t" | "\r" | "\n" } ;
LINE_COMMENT = "//" { char } newline ;
BLOCK_COMMENT = "{" { char } "}" | "(*" { char } "*)" ;
DIRECTIVE = "{$" { char } "}" ; (* compiler directives — skipped *)
DIRECTIVE = "{$" { char } "}" ; (* compiler directive *)
(* Conditional-compilation directives, handled in the lexer. Symbols are
* case-insensitive. Predefined: BLAISE (always), and the target's CPU/OS
* symbols (CPUX86_64, CPUAMD64, LINUX, UNIX on Linux/x86-64). The -d/--define
* command-line flag and {$DEFINE} add symbols; {$UNDEF} removes them.
*
* {$DEFINE sym} define a symbol
* {$UNDEF sym} undefine a symbol
* {$IFDEF sym} ... {$ENDIF}
* {$IFDEF sym} ... {$ELSE} ... {$ENDIF}
* {$IFNDEF sym} ... [ {$ELSE} ... ] {$ENDIF}
*
* IFDEF keeps its body when the symbol is defined (IFNDEF when not); the
* optional ELSE branch is taken otherwise. IFDEF/IFNDEF blocks nest. There
* is deliberately no expression form ({$IF ...}) yet — only symbol presence.
* All other directives (e.g. {$H+}, {$mode ...}) are accepted and ignored. *)
(* ═══════════════════════════════════════════════════════════════════════════

View file

@ -5106,3 +5106,48 @@ values canonical, so equality and hashing are well-defined.
* *A fixed-point money type (Int64 scaled).* Already rejected for `TDecimal`
(see the previous section); `TMoney` inherits `TDecimal`'s compact `Int64`
fast path for the common case, so it needs no separate fixed-point design.
== Conditional Compilation — Symbol-Presence `{$IFDEF}`, Predefined `BLAISE`
=== Decision
Blaise supports symbol-presence conditional compilation in the lexer:
`{$DEFINE sym}`, `{$UNDEF sym}`, `{$IFDEF sym}`/`{$IFNDEF sym}` with an optional
`{$ELSE}` and a closing `{$ENDIF}`, and a `-d`/`--define` command-line flag.
Symbols are case-insensitive. The compiler always predefines `BLAISE` and the
target's CPU/OS symbols (`CPUX86_64`, `CPUAMD64`, `LINUX`, `UNIX` on
Linux/x86-64). There is deliberately no expression form (`{$IF expr}`) — only
whether a symbol is defined.
=== Rationale
The feature was held back on purpose (the project builds itself without it,
delegating build-time configuration to PasBuild), but it is invaluable for
*end-user* source that must compile under FPC, Delphi and Blaise from one file:
`{$IFDEF BLAISE} ... {$ELSE} ... {$ENDIF}`. That cross-compiler use case is the
whole point of predefining `BLAISE`, and the CPU/OS predefines mirror the FPC
family so existing portable source compiles unchanged. This is the same
"every mainstream native compiler has it" argument the language applies to
parenthesised calls — conditional compilation is a near-universal expectation
for native Pascal, even though it is absent from managed languages like Java.
Implementing it in the lexer (not the parser) keeps excluded source from ever
reaching the parser or semantic pass, matching FPC/Delphi and avoiding any
interaction with type checking.
=== Alternatives considered
* *No conditional compilation (status quo, rely on PasBuild).* Rejected for
end-user code: writing a whole project with custom build paths to toggle one
line is disproportionate, and it does not help a single source file shared
across compilers.
* *An expression form `{$IF declared(X) and (Y > 2)}`.* Deferred. It needs a
constant-expression evaluator in the lexer and a `declared()` intrinsic;
symbol-presence `{$IFDEF}` covers the cross-compiler and feature-flag cases
that motivated the request. Can be added later without breaking `{$IFDEF}`.
* *A version macro predefine (e.g. `BLAISE_0_12_0`).* Not added yet — there is
no concrete use for it until the language stabilises; `BLAISE` alone serves
the cross-compiler case. CPU/OS symbols are added because portable source
routinely keys on them.