Divergence from Upstream¶

execsql2 is a maintained fork of execsql v1.130.1 by R. Dreas Nielsen. This page documents all user-visible changes since the fork was created: new features, changed behavior, security fixes, and removed functionality.

For a chronological view, see the Change Log.

Added Features¶

CLI Options¶

Export Formats¶

Metacommands¶

SCRIPT Enhancements¶

Bug Fixes¶

Conditional Tests¶

Configuration Options¶

New options in execsql.conf:

Tools¶

GUI¶

Authentication¶

Logging Enhancements¶

Developer / Packaging¶

Debugging¶

execsql2 adds an interactive debug REPL (no upstream equivalent), triggered either by the BREAKPOINT metacommand or the --debug CLI flag (step-through mode). The REPL prompts with execsql debug> and uses two-way dispatch (as of 2.19.0): input starting with . is a REPL command (.continue, .quit, .next, .vars, .vars VAR, .where, .stack, .set, .scripts, .cancel, .help); everything else is SQL, buffered across lines until terminated with ; and executed against the live connection (DML reports rows affected, DDL / transaction-control reports (statement executed), errors print inline without ending the session). BREAKPOINT is silently skipped when stdin is not a TTY so it does not hang CI. Full command list and behavior: Debugging guide.

Library API¶

execsql2 exposes execsql.run() for programmatic execution — no upstream equivalent.

from execsql import run

result = run(
    script="pipeline.sql",
    dsn="postgresql://user:pass@host/db",
    variables={"SCHEMA": "public"},
)
# result: ScriptResult(success, commands_run, elapsed, errors, variables)
result.raise_on_error()

Accepts dsn= or a pre-built connection= Database object. Each call runs in its own thread-local RuntimeContext, so concurrent calls from different threads are safe. halt_on_error=False collects errors instead of raising.

Changed Behavior¶

CLI Interface¶

The CLI framework changed from optparse to Typer with Rich-formatted help text. All original short flags (-a through -z) are preserved. The tool can be invoked as either execsql or execsql2.

Seven upstream long-form flags were renamed underscore → hyphen and the underscore forms are not accepted: --database-encoding, --script-encoding, --output-encoding, --import-encoding, --import-buffer, --user-logfile, --visible-prompts (upstream wrote these with underscores). Scripts and CI pipelines that invoke the long-form flags must update the spelling; the short letters (-e, -f, -g, -i, -l, -v, -z) are unchanged.

Default Database Type¶

The default database type (-t) changed from Access (a) to SQLite (l). Upstream defaulted to Access, which requires Windows and pyodbc. SQLite is cross-platform, ships with Python, and is the most common use case. Users targeting Access databases should pass -t a explicitly.

Configuration¶

• linux_config_file — now only active on Linux (sys.platform == "linux"). Upstream applied it to all POSIX systems, including macOS. A new macos_config_file option handles macOS specifically.

Internal State Management¶

All 33 mutable runtime globals in state.py have been consolidated into a RuntimeContext object stored in threading.local(). The module uses a transparent proxy so existing code is unaffected. Each thread gets its own isolated context, enabling concurrent from execsql import run calls and future PARALLEL blocks.

Substitution Variables¶

• $HOSTNAME — Network name of the machine running execsql (platform.node()). Useful for log messages and environment detection.
• $SYSTEM_CMD_PID — New system variable set to the PID of the background process when SHELL … CONTINUE is used.
• Cycle detection — substitute_vars() raises an error after 100 iterations to prevent infinite loops when variables reference each other cyclically. Upstream had no protection.
• O(1) substitution — Variable substitution uses a single combined regex and dict lookup instead of O(V) per-variable regex passes. Behavior is identical; performance is improved.
• Lazy $RANDOM/$UUID — These system variables are now computed on first access rather than generated unconditionally for every statement. Behavior is identical when referenced; scripts that never reference them skip the computation entirely.
• Static/dynamic system var split — System substitution variables are split into static (set once per script and refreshed on CONNECT/CHDIR) and dynamic (refreshed per statement). Eliminates redundant Path.resolve() syscalls and database pool lookups per statement.

CSV Import¶

• Fast-path CSV reader — Standard delimited imports (comma, tab, semicolon, pipe with doubled-quote escaping) now use Python's csv module. Non-standard formats (space-delimiter collapsing, escape characters) fall back to the original character-at-a-time parser.

Database Adapters¶

• Database is an ABC — open_db() and exec_cmd() are abstract methods. Subclasses that omit them raise TypeError at instantiation instead of at call time.
• Connection timeouts — PostgreSQL and SQLite adapters accept a connection timeout parameter (default 30 seconds).
• DuckDB temporal types — TIMESTAMPTZ, TIMESTAMP, DATE, TIME now map to native DuckDB types instead of TEXT.
• SQLite DT_Long mapping — DT_Long maps to "hugeint" in the SQLite type table. SQLite does not have a native HUGEINT type; the value receives TEXT affinity. In practice this is harmless because SQLite's type affinity system handles large integers transparently, but the mapping name differs from upstream.
• Database.auto_commits_ddl() not provided — 2.18.0 shipped this capability hook on the base Database class with True overrides on Oracle, MySQL, SQL Server, and MS Access (to signal that DDL on those drivers implicitly commits, so rollback() is a silent no-op across the DDL boundary). No call site ever consumed the hook, so 2.19.0 removed it. The asymmetry it described is still real — DDL inside a BEGIN BATCH … END BATCH block on Oracle / MySQL / SQL Server / MS Access cannot be rolled back — but execsql does not currently warn callers about it. (The companion hook Database.needs_explicit_commit_after_ddl(), used by the Firebird import path, remains.)

Execution Engine¶

The legacy flat command-list engine (_parse_script_lines / runscripts / CommandList.run_next()) has been replaced by an AST-based execution engine. Scripts are parsed into a tree of typed nodes, then the tree is walked for execution. This change is transparent to users — all metacommands, SQL, and control flow work identically.

• Native INCLUDE handling — The AST executor parses INCLUDE'd files with the AST parser and executes them through the tree-walking executor. Control flow structures (IF/LOOP/BATCH/SCRIPT) in included files are fully tree-driven, with correct deferred variable handling and source-span error reporting.
• Circular INCLUDE detection — The executor tracks the full include chain and detects circular references (e.g. A includes B includes A), reporting the full chain in the error message. Upstream has no such detection.
• BREAK outside LOOP is an error — BREAK outside a loop block now raises an error (exit 1) instead of being silently ignored. This catches script bugs that upstream would not report.
• Instance-scoped script registry — Named SCRIPT blocks are stored on the RuntimeContext instance instead of a module-level dict, preventing cross-execution contamination.
• Script source directory resolution — ScriptCmd resolves source_dir at construction time (when the command is parsed) rather than per-statement at execution time. This is functionally equivalent for all normal usage since the script file does not move between parse and execution.

Error Handling¶

• Exception hierarchy — All custom exceptions inherit from ExecSqlError, enabling except ExecSqlError to catch any execsql-originated error.
• Exception chaining — All raise statements inside except blocks preserve the original traceback via from.
• ASSERT error type — ASSERT failures now use a dedicated "assert" error type that produces **** Assertion failed. instead of **** Error in metacommand.. This distinguishes intentional script-level checks from actual metacommand errors. Upstream did not have ASSERT.

Security and Correctness Fixes¶

These are behavioral changes driven by security or correctness issues in the upstream code.

Injection Fixes¶

Template and Export Safety¶

Credential and Logging Safety¶

Bug Fixes¶

Removed Features¶