Skip to content

Databases

Base classes

Database defines the interface that every DBMS adapter must implement. Concrete adapters in sibling modules (sqlite.py, postgres.py, duckdb.py, etc.) subclass it, set a handful of required instance attributes in __init__, and override open_db() plus the schema-introspection methods. DatabasePool is the dict-like container (exposed as _state.dbs) that maps string aliases to open Database instances and tracks the active connection.

If you are adding support for a new database, start with the Adding Database Adapters guide.

__all__ = ['Database', 'DatabasePool'] module-attribute

Database(server_name, db_name, user_name=None, need_passwd=None, port=None, encoding=None)

Bases: ABC

Abstract base class for every DBMS adapter.

Concrete adapters in sibling modules (postgres.py, sqlite.py, duckdb.py, etc.) subclass Database and override the two abstract methods (:meth:open_db, :meth:exec_cmd) plus any introspection or data-handling method whose default ANSI information_schema implementation does not work for the target DBMS (schema_exists, table_exists, column_exists, table_columns, view_exists, role_exists, drop_table, populate_table).

The base class provides shared implementations of :meth:execute, :meth:select_data, :meth:select_rowsource, :meth:select_rowdict, :meth:commit, :meth:rollback, :meth:quote_identifier, and :meth:paramsubs that work for any DB-API 2.0 driver.

Adapter instances are owned by :class:DatabasePool (accessed at runtime via _state.dbs); metacommand handlers call _state.dbs.current() rather than constructing adapters directly.

Initialize common connection attributes for a database backend.

autocommit = True instance-attribute

conn = None instance-attribute

db_name = db_name instance-attribute

dt_cast property writable

Return the type-cast mapping, initialising it lazily on first access.

encode_commands = True instance-attribute

encoding = encoding instance-attribute

need_passwd = need_passwd instance-attribute

paramstr = '?' instance-attribute

password = None instance-attribute

port = port instance-attribute

server_name = server_name instance-attribute

type = None instance-attribute

user = user_name instance-attribute

__repr__()

Return a developer-friendly string representation of this connection.

autocommit_off()

Disable autocommit mode, grouping subsequent statements into a transaction.

autocommit_on()

Enable autocommit mode so each statement is committed immediately.

close()

Close the database connection, logging a warning if autocommit is off.

column_exists(table_name, column_name, schema_name=None)

Return True if column_name exists in table_name (optionally in schema_name).

commit()

Commit the current transaction if autocommit is enabled.

cursor()

Return a new cursor, opening the connection first if it has not been opened yet.

drop_table(tablename)

Drop tablename if it exists; tablename must already be schema-qualified and quoted.

exec_cmd(querycommand) abstractmethod

Execute a stored procedure or function by name.

execute(sql, paramlist=None)

Execute sql (optionally with paramlist), updating $LAST_ROWCOUNT.

Rolls back the current transaction and re-raises on any driver error.

import_entire_file(schema_name, table_name, column_name, file_name)

Insert the raw binary content of file_name as a single row into column_name of table_name.

import_tabular_file(schema_name, table_name, csv_file_obj, skipheader)

Import a CSV/tabular file into table_name; column names must be compatible.

name()

Return a human-readable description of this connection (DBMS + server/file).

needs_explicit_commit_after_ddl()

Return True if this adapter's driver does NOT auto-commit DDL.

Firebird is the notable case: a CREATE TABLE issued via the driver remains pending until commit, so callers that issue DDL followed by DML on a fresh table must commit in between. Most other adapters either auto-commit on DDL or run DDL inside the current transaction with no special handling required.

open_db() abstractmethod

Open the underlying database connection.

paramsubs(paramcount)

Return a comma-separated string of paramcount parameter placeholders.

populate_table(schema_name, table_name, rowsource, column_list, tablespec_src)

Bulk-insert rows from rowsource into table_name using the columns in column_list.

rowsource must be a generator yielding lists of values in column order. tablespec_src is a zero-argument callable that returns the table's type specification.

quote_identifier(identifier)

Return identifier wrapped in double-quotes with any embedded double-quotes escaped (standard SQL identifier quoting).

Override in subclasses for adapters whose native identifier quote differs (MySQL uses backticks, SQL Server uses square brackets).

quote_literal(value)

Return value as a SQL string literal, safely escaped.

Default ANSI/Postgres behaviour: wrap in single quotes, double embedded apostrophes, escape backslashes (so a value containing \' cannot terminate the literal on MySQL default mode or PostgreSQL E-strings), and reject embedded NUL bytes (most wire protocols truncate or reject them).

Override per-DBMS only when the wire protocol requires a different literal form.

quote_qualified_identifier(*parts)

Quote each non-empty part of a possibly-multi-segment identifier and join them with . (e.g. "schema"."table").

Skips None or empty parts so callers don't need to special-case schemaless databases (SQLite, Firebird).

role_exists(rolename)

Return True if rolename exists in this database.

The default implementation raises :class:~execsql.exceptions.DatabaseNotImplementedError; adapters for DBMSes that have a concept of roles (PostgreSQL, SQL Server, etc.) override this to query the appropriate catalog. Calling ROLE_EXISTS() from a script against a DBMS without role support will surface the raised error.

rollback()

Roll back the current transaction; swallows errors (best-effort).

schema_exists(schema_name)

Return True if schema_name exists in this database.

schema_qualified_table_name(schema_name, table_name)

Return the quoted, optionally schema-qualified form of table_name.

select_data(sql)

Execute sql and return (column_names, rows) with all rows fetched into memory.

select_rowdict(sql)

Execute sql and return (column_names, row_iterator) where each row is a dict.

select_rowsource(sql)

Execute sql and return (column_names, row_generator) for streaming large result sets.

table_columns(table_name, schema_name=None)

Return the ordered list of column names for table_name.

table_exists(table_name, schema_name=None)

Return True if table_name (optionally in schema_name) exists.

view_exists(view_name, schema_name=None)

Return True if view_name (optionally in schema_name) exists.

DatabasePool()

Maintains a set of database connection objects, each with a name (alias), and with the current and initial databases identified.

Initialize an empty connection pool with no active database.

current_db = None instance-attribute

do_rollback = True instance-attribute

initial_db = None instance-attribute

pool = {} instance-attribute

__repr__()

Return a string representation of the pool.

add(db_alias, db_obj)

Register db_obj under db_alias, setting it as initial/current if this is the first connection.

aliased_as(db_alias)

Return the Database registered under db_alias.

aliases()

Return a list of all currently registered database aliases.

closeall()

Roll back and close every connection in the pool, then reset the pool to empty.

current()

Return the currently active Database object.

current_alias()

Return the alias string for the currently active database.

disconnect(alias)

Close and remove the connection registered under alias from the pool.

initial()

Return the first Database that was added to the pool.

make_current(db_alias)

Set the active database to db_alias; raises ErrInfo if the alias is unknown.

Database factory

Convenience constructors used internally to create typed Database instances. The CLI calls the appropriate factory function based on the -t flag value. Each factory validates its arguments (e.g., checking that a file exists) before constructing and returning the adapter.

__all__ = ['db_Access', 'db_Dsn', 'db_DuckDB', 'db_Firebird', 'db_MySQL', 'db_Oracle', 'db_Postgres', 'db_SQLite', 'db_SqlServer'] module-attribute

db_Access(Access_fn, pw_needed=False, user=None, encoding=None)

Open an MS Access database file (.mdb or .accdb) via DAO/ODBC.

db_Dsn(dsn_name, user=None, pw_needed=True, encoding=None, password=None)

Open a connection to any ODBC data source registered under dsn_name.

db_DuckDB(duckdb_fn, new_db=False, encoding=None)

Open a DuckDB in-process analytics database file via the duckdb package.

db_Firebird(server_name, database_name, user=None, pw_needed=True, port=None, encoding=None, password=None)

Open a Firebird database connection via the firebird-driver package.

db_MySQL(server_name, database_name, user=None, pw_needed=True, port=None, encoding=None, password=None)

Open a MySQL or MariaDB connection via pymysql.

db_Oracle(server_name, database_name, user=None, pw_needed=True, port=None, encoding=None, password=None)

Open an Oracle database connection via cx_Oracle (python-oracledb).

db_Postgres(server_name, database_name, user=None, pw_needed=True, port=None, encoding=None, new_db=False, password=None)

Open a new PostgreSQL connection via psycopg2.

db_SQLite(sqlite_fn, new_db=False, encoding=None)

Open a SQLite database file via the standard-library sqlite3 module.

db_SqlServer(server_name, database_name, user=None, pw_needed=True, port=None, encoding=None, password=None)

Open a Microsoft SQL Server connection via pyodbc.

Adapters

DEFAULT_CONNECT_TIMEOUT = 30 module-attribute

__all__ = ['PostgresDatabase'] module-attribute

PostgresDatabase(server_name, db_name, user_name, need_passwd=False, port=5432, new_db=False, encoding='UTF8', password=None, connect_timeout=DEFAULT_CONNECT_TIMEOUT)

Bases: Database

PostgreSQL adapter using psycopg2, with schema support, server-side COPY, and keyring auth.

autocommit = True instance-attribute

conn = None instance-attribute

connect_timeout = connect_timeout instance-attribute

db_name = db_name instance-attribute

encode_commands = False instance-attribute

encoding = encoding or 'UTF8' instance-attribute

need_passwd = need_passwd instance-attribute

new_db = new_db instance-attribute

paramstr = '%s' instance-attribute

password = None instance-attribute

port = port if port else 5432 instance-attribute

server_name = server_name instance-attribute

type = dbt_postgres instance-attribute

user = user_name instance-attribute

__repr__()

exec_cmd(querycommand)

Execute a stored function by name.

import_entire_file(schema_name, table_name, column_name, file_name)

Import an entire binary file into a single column of a table.

import_tabular_file(schema_name, table_name, csv_file_obj, skipheader)

Import a delimited file into a PostgreSQL table, using COPY when possible.

open_db()

Open a connection to the PostgreSQL database.

role_exists(rolename)

Return True if the named role exists in the PostgreSQL cluster.

table_exists(table_name, schema_name=None)

Return True if the named table exists in the PostgreSQL database.

vacuum(argstring)

Run VACUUM with the given arguments in autocommit mode.

Note: argstring is interpolated directly into SQL because VACUUM is DDL and does not accept parameterized arguments. This is safe in execsql's trust model (the script author already has full SQL access), but should not be exposed to untrusted input.

view_exists(view_name, schema_name=None)

Return True if the named view exists in the PostgreSQL database.

DEFAULT_CONNECT_TIMEOUT = 30 module-attribute

__all__ = ['SQLiteDatabase'] module-attribute

SQLiteDatabase(SQLite_fn, timeout=DEFAULT_CONNECT_TIMEOUT)

Bases: Database

SQLite adapter using the Python standard-library sqlite3 module.

autocommit = True instance-attribute

conn = None instance-attribute

db_name = SQLite_fn instance-attribute

encode_commands = False instance-attribute

encoding = 'UTF-8' instance-attribute

need_passwd = False instance-attribute

paramstr = '?' instance-attribute

server_name = None instance-attribute

timeout = timeout instance-attribute

type = dbt_sqlite instance-attribute

user = None instance-attribute

__repr__()

column_exists(table_name, column_name, schema_name=None)

Return True if the named column exists in the given SQLite table.

drop_table(tablename)

Drop the named table from the SQLite database if it exists.

exec_cmd(querycommand)

Execute a query command as a view selection, since SQLite lacks stored procedures.

import_entire_file(schema_name, table_name, column_name, file_name)

Import an entire binary file into a single column of a table.

open_db()

Open a connection to the SQLite database file.

populate_table(schema_name, table_name, rowsource, column_list, tablespec_src)

Populate a SQLite table from a row source generator.

schema_exists(schema_name)

Return False; SQLite does not support schemas.

table_columns(table_name, schema_name=None)

Return a list of column names for the given SQLite table.

table_exists(table_name, schema_name=None)

Return True if the named table exists in the SQLite database.

view_exists(view_name)

Return True if the named view exists in the SQLite database.

__all__ = ['DuckDBDatabase'] module-attribute

DuckDBDatabase(DuckDB_fn)

Bases: Database

DuckDB in-process analytics adapter using the duckdb package.

autocommit = True instance-attribute

catalog_name = Path(DuckDB_fn).stem instance-attribute

conn = None instance-attribute

db_name = DuckDB_fn instance-attribute

encode_commands = False instance-attribute

encoding = 'UTF-8' instance-attribute

need_passwd = False instance-attribute

paramstr = '?' instance-attribute

server_name = None instance-attribute

type = dbt_duckdb instance-attribute

user = None instance-attribute

__repr__()

exec_cmd(querycommand)

Execute a query command as a view selection, since DuckDB lacks stored procedures.

open_db()

Open a connection to the DuckDB database file.

schema_exists(schema_name)

Return True if the named schema exists in the current DuckDB catalog.

view_exists(view_name)

Return True if the named view exists in the DuckDB database.

__all__ = ['SqlServerDatabase'] module-attribute

SqlServerDatabase(server_name, db_name, user_name, need_passwd=False, port=1433, encoding='latin1', password=None)

Bases: Database

Microsoft SQL Server adapter using pyodbc, trying drivers from newest to oldest.

autocommit = True instance-attribute

conn = None instance-attribute

db_name = db_name instance-attribute

encode_commands = True instance-attribute

encoding = encoding or 'latin1' instance-attribute

need_passwd = need_passwd instance-attribute

paramstr = '?' instance-attribute

password = None instance-attribute

port = port if port else 1433 instance-attribute

server_name = server_name instance-attribute

type = dbt_sqlserver instance-attribute

user = user_name instance-attribute

__repr__()

drop_table(tablename)

Drop the named table from the SQL Server database.

exec_cmd(querycommand)

Execute a stored procedure by name.

import_entire_file(schema_name, table_name, column_name, file_name)

Import an entire binary file into a single column of a table.

open_db()

Open a connection to the SQL Server database via pyodbc.

quote_identifier(identifier)

SQL Server native identifier quoting uses square brackets.

Override of :meth:Database.quote_identifier for B07a/F021: the base ANSI "…" form only works while QUOTED_IDENTIFIER is ON (the adapter sets this at open_db, but user SQL that toggles the session setting would silently break execsql-generated DDL). ] inside an identifier is escaped to ]].

role_exists(rolename)

Return True if the named role or principal exists in the SQL Server database.

schema_exists(schema_name)

Return True if the named schema exists in the SQL Server database.

__all__ = ['MySQLDatabase'] module-attribute

MySQLDatabase(server_name, db_name, user_name, need_passwd=False, port=3306, encoding='latin1', password=None)

Bases: Database

MySQL and MariaDB adapter using the pymysql package.

autocommit = True instance-attribute

conn = None instance-attribute

db_name = str(db_name) if db_name is not None else None instance-attribute

encode_commands = True instance-attribute

encoding = encoding or 'latin1' instance-attribute

need_passwd = need_passwd instance-attribute

paramstr = '%s' instance-attribute

password = None instance-attribute

port = port if port else 3306 instance-attribute

server_name = str(server_name) if server_name is not None else None instance-attribute

type = dbt_mysql instance-attribute

user = str(user_name) if user_name is not None else None instance-attribute

__repr__()

column_exists(table_name, column_name, schema_name=None)

exec_cmd(querycommand)

Execute a stored procedure by name.

import_tabular_file(schema_name, table_name, csv_file_obj, skipheader)

Import a delimited file into a MySQL table.

open_db()

Open a connection to the MySQL or MariaDB server.

quote_identifier(identifier)

MySQL / MariaDB native identifier quoting uses backticks.

Override of :meth:Database.quote_identifier for B07a/F021: the base ANSI "…" form only works after the connection sets sql_mode=ANSI / ANSI_QUOTES (the adapter does this at open_db, but user SQL that resets the mode would silently break execsql-generated DDL).

role_exists(rolename)

Return True if the named role or user exists in the MySQL server.

schema_exists(schema_name)

Return False; MySQL does not support schemas in the execsql sense.

table_exists(table_name, schema_name=None)

view_exists(view_name, schema_name=None)

__all__ = ['OracleDatabase'] module-attribute

OracleDatabase(server_name, db_name, user_name, need_passwd=False, port=1521, encoding='UTF8', password=None)

Bases: Database

Oracle adapter using the cx_Oracle (python-oracledb) driver.

autocommit = True instance-attribute

conn = None instance-attribute

db_name = db_name instance-attribute

encode_commands = False instance-attribute

encoding = encoding or 'UTF8' instance-attribute

need_passwd = need_passwd instance-attribute

paramstr = ':1' instance-attribute

password = None instance-attribute

port = port if port else 1521 instance-attribute

server_name = server_name instance-attribute

type = dbt_oracle instance-attribute

user = user_name instance-attribute

__repr__()

column_exists(table_name, column_name, schema_name=None)

Return True if the named column exists in the given Oracle table.

drop_table(tablename)

Drop the named table with cascade constraints.

exec_cmd(querycommand)

Execute a stored function by name.

execute(sql, paramlist=None)

Execute a SQL command, stripping any trailing semicolon for Oracle.

open_db()

Open a connection to the Oracle database.

paramsubs(paramcount)

Return Oracle-style positional parameter placeholders (:1, :2, ...).

role_exists(rolename)

Return True if the named role or user exists in the Oracle database.

dba_roles is restricted to DBA accounts and would raise ORA-00942: table or view does not exist for non-DBA users; we try it first and fall back to session_roles (always readable for the current session) if the catalog isn't accessible.

schema_exists(schema_name)

Raise DatabaseNotImplementedError; schema_exists is not supported for Oracle.

select_data(sql)

Return column names and all rows from a SELECT statement.

select_rowdict(sql)

Return column names and an iterable that yields rows as dictionaries.

select_rowsource(sql)

Return column names and an iterable that yields rows one at a time.

table_columns(table_name, schema_name=None)

Return a list of column names for the given Oracle table.

table_exists(table_name, schema_name=None)

Return True if the named table exists in the Oracle database.

view_exists(view_name, schema_name=None)

Return True if the named view exists in the Oracle database.

__all__ = ['FirebirdDatabase'] module-attribute

FirebirdDatabase(server_name, db_name, user_name, need_passwd=False, port=3050, encoding='latin1', password=None)

Bases: Database

Firebird adapter using the firebird-driver (fdb) package.

autocommit = True instance-attribute

conn = None instance-attribute

db_name = str(db_name) instance-attribute

encode_commands = True instance-attribute

encoding = encoding or 'latin1' instance-attribute

need_passwd = need_passwd instance-attribute

paramstr = '?' instance-attribute

password = None instance-attribute

port = port if port else 3050 instance-attribute

server_name = str(server_name) instance-attribute

type = dbt_firebird instance-attribute

user = str(user_name) instance-attribute

__repr__()

column_exists(table_name, column_name, schema_name=None)

Return True if the named column exists in the given Firebird table.

drop_table(tablename)

Drop the named table from the Firebird database.

exec_cmd(querycommand)

Execute a stored procedure by name.

needs_explicit_commit_after_ddl()

Firebird leaves DDL pending until commit — callers issuing CREATE TABLE then INSERT on a fresh table must commit in between.

open_db()

Open a connection to the Firebird database.

role_exists(rolename)

Return True if the named role or user exists in the Firebird database.

schema_exists(schema_name)

Return False; Firebird does not support schemas.

table_columns(table_name, schema_name=None)

Return a list of column names for the given Firebird table.

table_exists(table_name, schema_name=None)

Return True if the named table exists in the Firebird database.

view_exists(view_name, schema_name=None)

Return True if the named view exists in the Firebird database.

__all__ = ['AccessDatabase'] module-attribute

AccessDatabase(Access_fn, need_passwd=False, user_name=None, encoding=None, password=None)

Bases: Database

MS Access adapter connecting to .mdb/.accdb files via DAO (win32com) with pyodbc fallback.

autocommit = True instance-attribute

conn = None instance-attribute

connection_strings = (('DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=%s;ExtendedAnsiSQL=1;', True), ('DRIVER={Microsoft Access Driver (*.mdb)};DBQ=%s;', False), ('Provider=Microsoft.ACE.OLEDB.15.0; Data Source=%s;', True), ('Provider=Microsoft.ACE.OLEDB.12.0; Data Source=%s;', True)) class-attribute instance-attribute

dao_conn = None instance-attribute

db_name = Access_fn instance-attribute

dt_cast = dict(self.dt_cast) instance-attribute

encode_commands = True instance-attribute

encoding = encoding or 'windows-1252' instance-attribute

jet4 = len(Access_fn) > 6 and Access_fn.lower()[(-6):] == '.accdb' instance-attribute

last_dao_time = 0.0 instance-attribute

need_passwd = need_passwd instance-attribute

paramstr = '?' instance-attribute

password = None instance-attribute

server_name = None instance-attribute

temp_query_names = [] instance-attribute

temp_rx = re.compile('^\\s*create(?:\\s+or\\s+replace)?(\\s+temp(?:orary)?)?\\s+(?:(view|query))\\s+(\\w+) as\\s+', re.I) class-attribute instance-attribute

type = dbt_access instance-attribute

user = user_name instance-attribute

__repr__()

as_datetime(val)

Convert a value to a datetime object suitable for Access.

close()

Close both the DAO and ODBC connections.

column_exists(table_name, column_name, schema_name=None)

Return True if the named column exists in the given Access table.

dao_flush_check()

Wait if needed for Jet's read buffer to flush after a DAO command.

drop_table(tablename)

Drop the named table from the Access database.

exec_cmd(querycommand)

Execute a stored query command via DAO.

exec_dao(querystring)

Execute a query using the DAO connection.

execute(sqlcmd, paramlist=None)

Execute a SQL command, handling encoding, DAO flush, and temporary queries.

import_entire_file(schema_name, table_name, column_name, file_name)

Import an entire binary file into a single column of a table.

int_or_bool(val)

Convert a value to an integer, recognizing Access boolean values.

open_dao()

Open a DAO connection to the Access database.

open_db()

Open an ODBC connection to the Access database.

schema_exists(schema_name)

Return False; Access does not support schemas.

select_data(sql)

Return column names and all rows from a SELECT statement.

select_rowdict(sql)

Return column names and an iterable that yields rows as dictionaries.

select_rowsource(sql)

Return column names and an iterable that yields rows one at a time.

table_columns(table_name, schema_name=None)

Return a list of column names for the given Access table.

table_exists(table_name, schema_name=None)

Return True if the named table exists in the Access database.

Uses ODBC's catalog function (cursor.tables()) instead of querying MSysObjects directly. Access 2016+ refuses ad-hoc queries on the system catalog with the error Record(s) cannot be read; no read permission on 'MSysObjects'; the ODBC catalog path is permission-clean and matches the table view that CREATE TABLE modifies, unlike DAO TableDefs which caches independently.

view_exists(view_name, schema_name=None)

Return True if the named view or query exists in the Access database.

Uses DAO QueryDefs (Access's saved queries play the role of views) rather than querying MSysObjects directly — see :meth:table_exists for the permission rationale.

__all__ = ['DsnDatabase'] module-attribute

DsnDatabase(dsn_name, user_name, need_passwd=False, encoding=None, password=None)

Bases: Database

Generic ODBC adapter that connects to any data source registered as an ODBC DSN via pyodbc.

autocommit = True instance-attribute

conn = None instance-attribute

db_name = dsn_name instance-attribute

encode_commands = True instance-attribute

encoding = encoding instance-attribute

need_passwd = need_passwd instance-attribute

paramstr = '?' instance-attribute

password = None instance-attribute

port = None instance-attribute

server_name = None instance-attribute

type = dbt_dsn instance-attribute

user = user_name instance-attribute

__repr__()

exec_cmd(querycommand)

Execute a stored procedure by name.

import_entire_file(schema_name, table_name, column_name, file_name)

Import an entire binary file into a single column of a table.

open_db()

Open an ODBC connection using the configured DSN.