Adding a New Metacommand¶

This guide walks through every step required to add a new metacommand to execsql. The process involves four files in a fixed sequence: write the handler, register the regex, export the handler, and add tests.

Background: How Metacommands Work¶

The dispatch table¶

build_dispatch_table() in src/execsql/metacommands/dispatch.py populates a MetaCommandList with mcl.add() calls for every metacommand. __init__.py calls it at import time and assigns the result to the module-level DISPATCH_TABLE, which is then consumed at runtime via _state.metacommandlist.

MetaCommandList / MetaCommand¶

MetaCommandList (in src/execsql/script/engine.py) is an ordered list of MetaCommand objects with a keyword index for fast dispatch. Each MetaCommand holds one compiled regex and one handler. When MetaCommandList.eval() is called on a script line:

1. It extracts the leading keyword and narrows candidates via the keyword index, falling back to the full list if no keyword matches.
2. It calls the handler, passing all named regex groups plus "metacommandline" as keyword arguments.

Conditional branching (IF / ELSEIF / ELSE / ENDIF) is structural under the AST executor — the parser turns those metacommands into IfBlock AST nodes and the executor only invokes handlers for the branch it actually entered. Handlers therefore do not need to consult any IF state; the run_when_false and run_in_batch flags that used to gate dispatch under the flat command-list engine were removed in v2.18.1.

Handler naming conventions¶

Step-by-step: Adding a Metacommand¶

Step 1 — Write the handler function¶

Add the handler to whichever sibling module fits best:

If none of these fit, create a new module and follow the same structure.

Handler signature: all handlers accept only **kwargs. The keys are the named groups from the matching regex, plus "metacommandline" (the original unmodified command line).

# src/execsql/metacommands/data.py
from typing import Any
import execsql.state as _state


def x_my_command(**kwargs: Any) -> None:
    target = kwargs["target"]          # from (?P<target>...) in the regex
    value  = kwargs["value"]           # from (?P<value>...) in the regex
    # Optional groups are None when absent — check before using.
    option = kwargs.get("option")

    # Interact with global state as needed.
    _state.subvars.add_substitution(target, value)
    return None

Key points:

• Return None unless there is a specific reason to return a value (very rare).
• Optional regex groups will be None when the group did not participate in the match. Always guard with if option: or kwargs.get("option").
• Raise ErrInfo for expected failure conditions rather than using a bare raise or sys.exit.

from execsql.exceptions import ErrInfo

def x_my_command(**kwargs: Any) -> None:
    if not some_condition:
        raise ErrInfo("cmd", command_text=kwargs["metacommandline"],
                      other_msg="MY_COMMAND: something went wrong")

Step 2 — Register the regex in the dispatch table¶

Open src/execsql/metacommands/dispatch.py and add a mcl.add() call inside build_dispatch_table(). (__init__.py calls this function at import time and assigns the result to DISPATCH_TABLE; you don't edit __init__.py for new registrations.) Order matters: more specific patterns should appear before catch-all patterns. The list is traversed front-to-back and the first match wins.

mcl.add(
    r"^\s*MY_COMMAND\s+(?P<target>\w+)\s+(?P<value>.+)$",
    x_my_command,
    description="MY_COMMAND",
    category="action",
)

MetaCommandList.add() accepts these parameters:

All regexes are compiled with re.I (case-insensitive) automatically.

Passing multiple regex variants is common when a value may be quoted or unquoted:

mcl.add(
    (
        r'^\s*MY_COMMAND\s+(?P<target>\w+)\s+"(?P<value>[^"]+)"\s*$',
        r"^\s*MY_COMMAND\s+(?P<target>\w+)\s+(?P<value>\S+)\s*$",
    ),
    x_my_command,
    description="MY_COMMAND",
)

Using regex composition helpers (from src/execsql/utils/regex.py) avoids duplicating patterns for all quoting styles used by file-name arguments:

from execsql.utils.regex import ins_fn_rxs, ins_table_rxs

# ins_fn_rxs(prefix, suffix) generates variants for bare, quoted, and
# bracket-quoted filenames captured as (?P<filename>...).
mcl.add(
    ins_fn_rxs(r"^\s*MY_COMMAND\s+TO\s+", r"\s*$"),
    x_my_command,
)

Step 3 — Import the handler in dispatch.py¶

Add the function to the relevant import block at the top of src/execsql/metacommands/dispatch.py so the mcl.add() call in Step 2 can reference it:

from execsql.metacommands.data import (
    ...
    x_my_command,   # add here
)

Step 4 — Write tests¶

Integration tests (preferred)¶

Integration tests exercise the full CLI pipeline against a temporary SQLite database. Add a new test class to tests/test_metacommands.py (or the appropriate test_metacommands_*.py file):

class TestMyCommand:
    """MY_COMMAND metacommand."""

    def test_basic(self, script_runner):
        result, db = script_runner("""
            -- !x! my_command greeting hello
            create table result (val text);
            insert into result values ('!!greeting!!');
        """)
        assert result.exit_code == 0, result.output
        assert qdb(db, "SELECT val FROM result") == [("hello",)]

    def test_error_case(self, script_runner):
        result, db = script_runner("""
            -- !x! my_command
        """)
        # Bad syntax should not crash; exit code depends on HALT_ON_METACOMMAND_ERR
        assert "MY_COMMAND" in result.output or result.exit_code != 0

The script_runner fixture:

• Writes the script to a temp .sql file.
• Invokes execsql <script> <db> -t l -n via typer.testing.CliRunner.
• Waits for the FileWriter subprocess to flush all pending writes.
• Returns (result, db_path).

Use qdb(db_path, sql) to query the resulting SQLite database for assertions.

Unit tests¶

For handlers that are complex enough to warrant isolated testing, use direct function calls with mocked state. See tests/test_metacommands_connect.py for examples:

from unittest.mock import patch, MagicMock
from execsql.metacommands.data import x_my_command


def test_x_my_command_sets_subvar(minimal_conf, tmp_path):
    # Set up the minimal state that the handler touches.
    from tests.conftest import _setup_subvars
    sv = _setup_subvars()

    x_my_command(
        target="greeting",
        value="hello",
        option=None,
        metacommandline="MY_COMMAND greeting hello",
    )

    assert sv.get_substitution("greeting") == "hello"

Checklist¶

• Handler function added to the appropriate src/execsql/metacommands/*.py module
• Handler imported in src/execsql/metacommands/dispatch.py
• mcl.add(...) call added in build_dispatch_table() with description= and category=
• Run just install-vscode to regenerate the VS Code grammar
• Integration test added to tests/test_metacommands.py (or relevant file)
• pytest passes locally (including tests/test_registry.py keyword consistency checks)

Adding a Conditional Test Predicate (xf_*)¶

If your new functionality needs to be usable in IF/ELSEIF expressions (e.g., IF MY_TEST foo bar), add an xf_* function instead. These live in src/execsql/metacommands/conditions.py and are registered in a separate build_conditional_table() function in the same file.

def xf_my_test(**kwargs: Any) -> bool:
    """Return True if the condition is met."""
    return kwargs["value1"] == kwargs["value2"]

Then register it in build_conditional_table() (the same MetaCommandList-typed local is conventionally named mcl here too):

mcl.add(
    r"^\s*MY_TEST\s+(?P<value1>\S+)\s+(?P<value2>\S+)\s*$",
    xf_my_test,
    description="MY_TEST",
    category="condition",
)

The result is exposed as CONDITIONAL_TABLE and consumed at runtime via _state.conditionallist.