Integration

Info

This guide showcases how to make your package support omop-config configure <package> and use oa-configurator for engine creation and logging.

---

Overview

Each package that integrates with oa-configurator:

1. Subclasses PackageConfigBase with its typed config fields
2. Registers the class via an entry point in pyproject.toml
3. Calls MyPackageConfig.get_config() to read its config
4. Resolves its database entry with Resolver.from_active_config().resolve_database(name) and builds its SQLAlchemy engines from it
5. Calls configure_logging(verbosity=verbose, extra_namespaces=["<package>"]) at startup

---

Steps for integration

1: Add the dependency

In your pyproject.toml:

[project.dependencies]
"oa-configurator>=0.2.0"  # version may vary

---

2. Define your config class

In src/<package>/config.py:

from typing import Annotated, ClassVar
from pydantic import Field
from oa_configurator import CDMDatabaseConfig, PackageConfigBase, RefTo


class MyPackageConfig(PackageConfigBase):
    tool_name: ClassVar[str] = "my_package"   # maps to [tools.my_package] in TOML

    # A field naming an entry in [databases.*] of kind "cdm". `omop-config
    # configure` offers to reuse an existing one or create it on the spot.
    cdm_db: Annotated[str, RefTo(CDMDatabaseConfig)] = "cdm_db"

    # Plain typed fields, backed by the [tools.my_package] TOML section
    backend: str = Field(default="default", description="Backend to use.")
    data_path: str | None = Field(default=None, description="Path to local data files.")

get_config() is inherited from PackageConfigBase: call MyPackageConfig.get_config() to load from the active stack config. It delegates to Resolver.resolve_package_config(), which reads the [tools.my_package] section and validates every RefTo-marked field against it. If the section is missing, fields fall back to their defaults.

What package users get from the CLI

omop-config configure my_package checks the complete proposed section with MyPackageConfig before it saves anything. This includes ordinary field constraints, cross-field validators, and missing or wrong-kind RefTo targets. CLI strings are converted to the model's real Python types before being written to TOML, and an invalid proposal leaves the existing file unchanged.

A default RefTo value is still a real reference. For example, if cdm_db defaults to the name "cdm_db", that database must exist before a non-interactive configure command can succeed. In scripted setup, create the target first or pass nested --set values that create the target chain in the same command. In the interactive flow, the user can create or select the missing target when prompted.

Add configuration to your own application

If your package has its own web form, desktop screen, or API, use plan_configure() rather than duplicating the CLI's field and RefTo logic:

from oa_configurator import plan_configure

candidate = plan_configure(MyPackageConfig, current_config, proposed_values)

The result is a new complete stack; current_config is unchanged. Submitted nested dictionaries can create or update the entries behind RefTo fields, while omitted fields continue to use their stored values. Use PackageConfigInvalidError.errors() to place schema messages beside the relevant inputs, and present ConfigurationError as a reference problem. Once the user has reviewed the result, persist it with a separate save_stack_config(candidate) call. The library functions raise exceptions without printing CLI text or raising typer.Exit.

---

3. Register the entry point

[project.entry-points."omop.config"]
my_package = "my_package.config:MyPackageConfig"

After installing your package, omop-config configure my_package will find and prompt through your fields.

---

4. Engine creation

resolve_database(name) returns a different type depending on the entry's kind, and each type has its own engine call:

Entry kind	Resolves to	Engine call	Returns
generic (GenericDatabaseConfig)	ResolvedDatabase	create_engine()	one engine
cdm (CDMDatabaseConfig)	ResolvedCDMDatabase	create_engines()	(primary, vocab)

A generic database has one connection and one schema:

from oa_configurator import Resolver

engine = Resolver.from_active_config().resolve_database("my_store").create_engine()

A CDM database can keep its vocabulary on a database of its own, so it always yields two engines:

from oa_configurator import Resolver

config = MyPackageConfig.get_config()
primary, vocab = Resolver.from_active_config().resolve_database(config.cdm_db).create_engines()

vocab is the same object as primary when the vocabulary shares the primary database. Both engines apply the schema_translate_map, so OMOP ORM models route to the right schemas without changes. create_engine() and Resolver.resolve_engine() raise TypeError on a CDM entry.

Every engine create_engines() builds refuses a statement that references a table hosted on the other database (CrossDatabaseStatementError), so sending vocabulary work to the primary engine fails instead of reading a stale copy. Raw text() SQL carries no table metadata and is not checked.

---

5. Logging

At your package's CLI entry point or startup:

from oa_configurator import configure_logging

# verbosity comes from the -v/-vv CLI flag count
configure_logging(verbosity=verbose, extra_namespaces=["my_package"])

Pass load_stack_config() as the first argument instead of verbosity= to use the [logging] block from the config file:

configure_logging(load_stack_config(), verbosity=verbose, extra_namespaces=["my_package"])

---

Testing

Tests fall into two tiers with different requirements.

Unit tests: StackConfig.for_session()

Unit and mock-based tests must never touch ~/.config/omop/config.toml. Use StackConfig.for_session() with monkeypatch to inject a fully in-memory config:

from oa_configurator import StackConfig, Resolver

def test_something(monkeypatch):
    cfg = StackConfig.for_session(
        connections={"db": {"dialect": "sqlite", "database_name": ":memory:"}},
        databases={"cdm": {"kind": "cdm", "connection": "db"}},
        tools={"my_package": {"backend": "test_backend"}},
    )
    monkeypatch.setattr("my_package.module.load_stack_config", lambda: cfg)
    # ... test against the in-memory config

This covers the vast majority of tests. No file I/O, no environment-specific setup needed.

for_session() + monkeypatch is NOT a fallback

for_session() + monkeypatch is for isolated unit tests where only config values matter, not config source. It MUST NEVER be used to paper over missing configuration in CI or local dev. If a code path needs get_config()/load_stack_config() to succeed at all, you need to provide a real configuration for the test case.

Integration tests: a dedicated test database

For tests that exercise a real database (e.g. PostgreSQL-specific SQL, bulk loading, trigger management), use a dedicated named database in the user's config, never a copy of the production database under a different guise.

The convention is test_<package>_db_<dialect> (e.g. test_cdm_db_pg for a Postgres-backed field, test_cdm_db_sqlite for a SQLite one), one field per dialect your package's tests actually exercise. What marks a field as a test field is RefTo(CDMDatabaseConfig, is_test=True) (or RefTo(GenericDatabaseConfig, is_test=True), whichever kind the package's real database is), not its Python name, which carries no meaning to oa-configurator itself. Keeping the value distinct from the production database is still a mandatory safety guard: the test suite must never accidentally connect to a production database. Declare one field per dialect rather than reusing a single field for both: a field that's genuinely dialect-agnostic (works whether it happens to resolve to Postgres or SQLite) is rare in practice, and a field pinned to a specific dialect via dialect= (below) will raise if it ever resolves to the wrong one, since a mismatch is treated as a real misconfiguration, never silently substituted.

In conftest.py, resolve the test database via the isolated_test_database() context manager, which skips cleanly whether config.toml is entirely missing or simply doesn't have this database configured yet, provisions it if needed, and yields an already-isolated .connection/.session pair. - Postgres wraps the whole thing in one transaction that's rolled back on exit (nothing your test does ever commits, so concurrent test runs against the same shared server can't collide); - SQLite gets a fresh, disposable database per call:

@pytest.fixture
def pg_db():
    from oa_configurator.testing import isolated_test_database
    from my_package.config import MyPackageConfig

    with isolated_test_database(MyPackageConfig, "test_cdm_db_pg") as db:
        yield db  # db.connection (Core) / db.session (ORM)

isolated_test_database(cls, field_name) always takes the field name explicitly, with no auto-discovery, so it resolves without requiring the rest of your package's config (e.g. a required cdm_db field) to also be configured. That's deliberate on both counts: a CI runner that only provisions a test database shouldn't need a "production" database configured just to find it, and a class with more than one is_test=True field (one per dialect, per the convention above) would otherwise have no way to say which one a caller means.

Pass dialect= to pin the resolved connection to a specific dialect regardless of what the field happens to resolve to. A mismatch always raises, and an unconfigured field falls back to that dialect's own zero-config provisioning if it supports one (SQLite always does; Postgres doesn't, since it needs a real server):

@pytest.fixture
def empty_engine():
    with isolated_test_database(
        MyPackageConfig, "test_cdm_db_sqlite", dialect="sqlite",
    ) as db:
        yield db.connection.engine

For the rare case where the code under test constructs its own engine/connection and needs to see real, committed state (rather than the rolled-back transaction above), use isolated_test_schema(engine) instead (see oa_configurator.testing's module docstring).

Migrating from resolve_test_database (removed in 2.0)

resolve_test_database/ensure_test_db_exists/create_fresh_test_db/drop_test_db/ensure_test_user_exists/require_pg_extension and the requires_database pytest marker are removed as of 2.0. Replace any use of them with isolated_test_database(), which provisions automatically and needs no manual teardown.

Provisioning the test database

The test database must be provisioned for real before pytest runs. There is no fallback that papers over a missing one, by design (see the callout above). A field targeting SQLite needs no provisioning at all: leave it unconfigured in config.toml, and isolated_test_database(..., dialect="sqlite") builds a disposable instance with no server, no credentials, nothing to set up.

A package field marked is_test=True (e.g. test_cdm_db_pg: Annotated[str | None, RefTo(CDMDatabaseConfig, is_test=True)] = None) gets special handling in omop-config configure <package>'s interactive flow: it asks whether to configure a test database, and if you accept, recurses through creating both the connection and the database, marking the connection test_only = true automatically and refusing to reuse (or collide with) a non-test connection's host/database combination. resolve_package_config() (used by get_config() and this interactive flow alike) separately enforces, every time your config loads, that an is_test=True field always resolves to a test_only=true connection and an is_test=False field never does. Pointing either one at the wrong kind of connection raises ConfigurationError immediately, not just when a test happens to run.

Non-interactively, --test-only is an ordinary flag on connections add (accepting true/false/yes/no/1/0):

omop-config connections add test_cdm \
  --dialect postgresql+psycopg --host localhost --port 5432 \
  --user test --password test --database-name test_db --test-only true

omop-config databases add cdm test_cdm_db_pg --connection test_cdm --cdm-schema <my-schema>

omop-config configure <package> --test-cdm-db-pg test_cdm_db_pg

The --test-cdm-db-pg flag above is the field's own auto-generated flag (test_cdm_db_pg -> --test-cdm-db-pg); it points the field at an already-created database by name, same as any other RefTo field passed non-interactively. Or do it in the single configure call directly with --set (see Docker Compose below):

omop-config configure <package> \
  --set test_cdm_db_pg.kind=cdm \
  --set test_cdm_db_pg.connection.dialect=postgresql+psycopg \
  --set test_cdm_db_pg.connection.host=localhost \
  --set test_cdm_db_pg.connection.database_name=test_db \
  --set test_cdm_db_pg.connection.test_only=true \
  --set test_cdm_db_pg.cdm_schema=<my-schema>

Local developmentCI

Run omop-config configure <package> and answer Y when asked to configure a test database field.

Either provision the connection and database ahead of time (as part of image build or a setup step) and pass the package's test-field flag, or do it all in one --set-based configure call, as shown above.

Safety

The test database must point to a dedicated, empty database. If your test session drops and recreates schemas, add a runtime guard that compares the resolved URL of test_cdm_db_pg against all other configured databases and calls pytest.fail() on any match.

Testing across dialects

A fixture that needs to exercise the same test body against every supported dialect should use DIALECT_PARAMS rather than a hand-rolled params= list:

from oa_configurator.testing import DIALECT_PARAMS, isolated_test_database

# request.param is SQLAlchemy's own dialect name ("postgresql", "sqlite",
# ...), not the field-naming convention's abbreviation ("pg"), so this maps
# one to the other explicitly rather than interpolating request.param
# straight into the field name.
_FIELD_BY_DIALECT = {"postgresql": "test_cdm_db_pg", "sqlite": "test_cdm_db_sqlite"}


@pytest.fixture(params=DIALECT_PARAMS)
def db(request):
    with isolated_test_database(
        MyPackageConfig, _FIELD_BY_DIALECT[request.param], dialect=request.param
    ) as db:
        yield db

DIALECT_PARAMS is generated from the same registry that backs isolated_test_database()'s dialect dispatch, so it always matches whatever dialects your installed oa-configurator actually supports, with no list to keep in sync by hand. Each param always carries its own dialect name (sqlite, postgresql, ...), plus a second, generic db_dialect mark, but only if that dialect is actually capable of the corruption described below; a dialect that structurally can't cause it (SQLite today) skips that second mark, since excluding it from the default run would buy nothing.

Add this to your package's pyproject.toml so tests capable of that corruption are excluded from the default run and only execute when explicitly selected:

[tool.pytest.ini_options]
addopts = "-m 'not db_dialect'"

Then pytest -m postgresql (or -m sqlite) runs just that dialect's tests, explicitly, regardless of whether it's excluded by default. This is the shape a CI pipeline should use for its Postgres step, typically as a second job or step alongside the default run.

Reason for exclusion

The exclusion exists because SQLAlchemy's create_all() permanently mutates shared ForeignKeyConstraint state the first time it defers a circular-dependency foreign key on a dialect whose supports_alter is true, which corrupts a later create_all() call against the same shared metadata on any other dialect in the same process. The fix is making sure that never happens within one pytest invocation, not chasing the mutation itself, and only excluding dialects that can actually cause it in the first place.

If a test already gets its Postgres connection through a pg_db-named fixture (rather than DIALECT_PARAMS), it's marked postgresql + db_dialect automatically, with no changes needed, as long as the fixture keeps the established pg_db name.

Adding a new dialect

oa_configurator.testing dispatches to one TestDatabaseStrategy subclass per dialect via a plain dict, keyed off SQLAlchemy's own dialect name string. Supporting a new dialect in tests means writing one subclass and registering it there. Once registered, that dialect's pytest marker, its slot in DIALECT_PARAMS, and whether it's excluded from -m 'not db_dialect' all follow automatically, based on that dialect's own real supports_alter value, with no further changes needed anywhere.

---

Multiple environments

Adding a second database of the same kind

A package's own field just names a database by default (e.g. cdm_db: Annotated[str, RefTo(CDMDatabaseConfig)] = "cdm_db"). To point at a second one, for example a production CDM alongside a local development one, create the extra database under its own name and pass the field's own flag:

omop-config databases add cdm cdm_db_prod --connection cdm_prod --cdm-schema omop
omop-config configure omop_alchemy --cdm-db cdm_db_prod

This creates cdm_db_prod without touching the existing cdm_db, and points omop_alchemy at it.

Choosing between databases of the same kind

There is no config-level "default database" toggle. When more than one database of the same kind exists, the caller names the one it wants explicitly. For two CDM databases:

from oa_configurator import Resolver
from omop_alchemy.config import OmopAlchemyConfig

config = OmopAlchemyConfig.get_config()
resolver = Resolver.from_active_config()
prod_primary, prod_vocab = resolver.resolve_database("cdm_db_prod").create_engines()
dev_primary, dev_vocab = resolver.resolve_database(config.cdm_db).create_engines()  # whatever cdm_db currently resolves to

For switching between whole environments (dev vs. prod) rather than picking one database among several, use distinctly-named connections and databases per environment, and point each deployment's omop-config configure flags at the right ones. There is no profile/overlay mechanism; naming is the only axis.

Cross-package database references

A package that consumes a database owned by another package declares its own field with the same RefTo(CDMDatabaseConfig) type and the same default name (e.g. cdm_db: Annotated[str, RefTo(CDMDatabaseConfig)] = "cdm_db"). There is no typed cross-package pointer: two packages share an entry simply because both fields resolve to the same name. The shared database must be configured once, under that name, for both packages' references to resolve.

---

Docker Compose

How it works

~/.config/omop/config.toml lives on the host (or in a container's home directory) and is the single source of truth. Docker Compose is only needed to provide database credentials at container startup. The app itself always reads from the TOML file, never from environment variables at runtime.

The workflow:

1. A gitignored .env file holds secrets that Docker Compose substitutes into its YAML.
2. The container's startup command calls omop-config connections add/databases add with --flags, then omop-config configure <package> to point the package at what was just created, writing those values into config.toml once at startup.
3. After that, the app reads config.toml normally without any environment variables involved.

The .env file is a Docker Compose concern only. It is never loaded by the Python app.

Example

.env (gitignored):

POSTGRES_USER=omop
POSTGRES_PASSWORD=secret
POSTGRES_DB=omop_cdm

docker-compose.yml:

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}

  app:
    build: .
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    command: >
      bash -c "
        omop-config connections add cdm
          --dialect postgresql+psycopg --host db --port 5432
          --user $$POSTGRES_USER --password $$POSTGRES_PASSWORD
          --database-name $$POSTGRES_DB &&
        omop-config databases add cdm cdm_db --connection cdm --cdm-schema omop &&
        omop-config configure my_package --cdm-db cdm_db &&
        exec my_app_entrypoint
      "

$$VAR escaping

Use $$VAR (double dollar) inside a command: string so Docker Compose passes the literal variable name to the shell rather than substituting it at YAML-parse time.

If your stack has more than one package pointing at the same database (e.g., my_package and omop_alchemy), the connections add/databases add steps only need to run once; add one configure call per package, chained with &&:

command: >
  bash -c "
    omop-config connections add cdm
      --dialect postgresql+psycopg --host db --port 5432
      --user $$POSTGRES_USER --password $$POSTGRES_PASSWORD
      --database-name $$POSTGRES_DB &&
    omop-config databases add cdm cdm_db --connection cdm --cdm-schema omop &&
    omop-config configure omop_alchemy --cdm-db cdm_db &&
    omop-config configure my_package --cdm-db cdm_db &&
    exec my_app_entrypoint
  "

Each configure call is scoped to its own package's flags; connections add/databases add are shared setup steps run once regardless of how many packages point at the result.

One-shot alternative: --set

For a single package pointing at a database nobody else needs to share, configure can create the connection and database in the same call via repeated --set field.subfield=value flags, instead of the three separate commands above:

command: >
  bash -c "
    omop-config configure my_package
      --set cdm_db.kind=cdm
      --set cdm_db.connection.dialect=postgresql+psycopg
      --set cdm_db.connection.host=db
      --set cdm_db.connection.port=5432
      --set cdm_db.connection.user=$$POSTGRES_USER
      --set cdm_db.connection.password=$$POSTGRES_PASSWORD
      --set cdm_db.connection.database_name=$$POSTGRES_DB
      --set cdm_db.cdm_schema=omop &&
    exec my_app_entrypoint
  "

cdm_db here is the name of the package's own field (Annotated[str, RefTo(CDMDatabaseConfig)]), and connection is DatabaseConfig's own field (shared by both kinds) naming a [connections.*] entry. The dotted path can go as deep as the reference chain does. The connection this creates is named after the database (cdm_db, from the field's own default), or pass --set cdm_db.connection.name=<explicit-name> to choose one. Prefer the three-command form above when more than one package needs to point at the same database, since --set creates a fresh one per call.

Security note

The config.toml written by the container will contain the database password in plaintext. This is acceptable for local development containers. Restrict the file permissions:

chmod 600 ~/.config/omop/config.toml

omop-config will warn at load time if the file has looser permissions.

---

TOML snippet for your README

Add a Configuration section to your package README:

## Configuration

Requires `omop-config` to be run once. See the
[`oa-configurator` quickstart](link) for initial setup.

Add the following to `~/.config/omop/config.toml`:

\`\`\`toml
[tools.my_package]
backend   = "default"
data_path = "/path/to/data"
\`\`\`

Then run:
\`\`\`bash
omop-config configure my_package
\`\`\`