Skip to content

perf-008: Static newline token in lexer #233

Description

@python-processing-unit

Problem

TOKEN_NEWLINE from \n and ; each does safe_strdup(\n) — heap alloc + copy for a constant 1-byte string that is the most numerous token type.

Location: src/lexer.c:293-301

Proposed Fix

Share a static const char g_newline_literal[] = \n across both creation sites. token_free must not free static-owned literals. Coordinate with perf-004 for consistent ownership model.

Status

Baseline still present — src/lexer.c:294,299 still use safe_strdup(\n). No shared literal.

Metadata

Metadata

Labels

interpreterRequires a code change in the interpreter.patchRequires a patch version change.performancePerformance improvement.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions