Skip to content

core

Foundational services with no clinical-domain assumptions. A concept resolver behaves the same whether it is mapping tumour morphology or procedures; a patient timeline is the same object whatever populates it. Domain-specific concept sets, thresholds, and grading rules belong in analytics, not here.

Concept resolution

Turns a declarative description of which concepts belong in a lookup into a runtime resolver that maps free text and source codes to OMOP concept IDs.

from omop_alchemy.toolkit.core.concepts import make_concept_resolver

resolver = make_concept_resolver(
    session,
    name="condition lookup",
    domain_id="Condition",
)
concept_id = resolver.lookup("Adenocarcinoma of lung")

Resolve free-text terms and source codes to OMOP concept IDs.

Source data rarely arrives with concept IDs attached. This module turns a declarative description of which concepts are eligible into a runtime resolver that maps incoming text to those concepts, applying the same normalisation on both sides so that matching is predictable.

Three pieces make up the workflow:

LookupSpec Declares which concepts belong in a lookup — by vocabulary, domain, concept class, or explicit ancestry — and which text fields are indexed.

LookupIndex The materialised table of normalised text keys to concept IDs that a spec produces against the vocabulary tables.

ConceptResolver Wraps an index and resolves terms at runtime, applying the same normalisation used to build the keys.

make_concept_resolver bundles all three for the common case::

from omop_alchemy.toolkit.core.concepts import make_concept_resolver

resolver = make_concept_resolver(
    session,
    name="condition lookup",
    domain_id="Condition",
)
concept_id = resolver.lookup("Adenocarcinoma of lung")

Normalisation is composable. compose_normalizers chains individual rules — normalize_default for whitespace and casing, strip_uicc and make_stage for staging text, site_to_NOS for site generalisation — so that a resolver's matching behaviour is stated explicitly rather than implied.

Building an index queries the vocabulary tables, so resolvers are worth reusing. ConceptResolverRegistry constructs each resolver on first access and caches it for the registry's lifetime.

Governed concept sets are the other half of this module. Where a resolver maps text to concepts, a concept group answers whether a concept ID belongs to a governed clinical set::

from omop_alchemy.toolkit.core.concepts import (
    ConceptGroupSpec,
    resolve_concept_group,
)

RT = ConceptGroupSpec(name="radiotherapy", unit=...)   # governed anchors

group = resolve_concept_group(session, RT)
procedure.procedure_concept_id in group        # Python, O(1)
group.expression(Procedure_Occurrence.procedure_concept_id)   # SQL

Both access paths derive from one spec, so they cannot disagree. Expansions are cached per vocabulary, not per engine, so recreating an engine against the same database does not re-run the closure queries — see :mod:.identity for how an engine declares which vocabulary it reads, and note that a caller building its own engines must register that itself.

Nothing here touches a database at import time: specs are declarative and registries build on first request.

CacheStats dataclass

CacheStats(
    entries: int = 0,
    cached_bytes: int = 0,
    evictions: int = 0,
    rebuilds_after_evict: int = 0,
)

Observability for a bounded registry.

rebuilds_after_evict is the load-bearing number. Evicting an entry that is never asked for again is exactly what a bound is for; evicting one that is then rebuilt is thrashing, and it is otherwise invisible because it presents as ordinary slowness rather than as a cache problem. While it stays at zero the bound is correct.

ConceptGroupRegistry

ConceptGroupRegistry(
    engine: Engine,
    *,
    max_bytes: int = DEFAULT_MAX_CACHE_BYTES,
)

Bases: _LazyBoundedRegistry[ResolvedConceptGroup]

Lazy registry for governed concept groups, scoped to one vocabulary.

Obtain one through :func:concept_group_registry rather than constructing it directly, so registries are shared per vocabulary identity instead of per engine.

register_spec

register_spec(spec: ConceptGroupSpec) -> None

Register spec under its governed name, if not already present.

ConceptGroupSpec dataclass

ConceptGroupSpec(
    name: str,
    unit: Any,
    include_descendants: bool = True,
    require_standard: bool = False,
    include_classification: bool = True,
)

A governed omop-semantics semantic unit plus how to expand it.

Declarative and side-effect free — constructing one performs no I/O and touches no database.

Parameters:

Name Type Description Default
name str

Stable identifier for this group, used as the cache key. Use the governed semantic-unit name so the key derives from the governed identity rather than a locally invented label.

required
unit Any

The omop-semantics RuntimeSemanticUnit supplying anchors. Read lazily, so a spec can be declared at module scope without loading the semantics runtime. omop-semantics 0.6.0 put mixed-role composition on the semantic unit rather than on RuntimeGroup, which is why this takes a unit: parent_ids expand through descendants while exact_ids are matched directly.

required
include_descendants bool

Expand parent_ids through concept_ancestor. When False only the anchors themselves are members.

True
require_standard bool

Restrict expansion to concepts carrying a standardness flag. Defaults to False, matching the historical oncology behaviour of reading concept_ancestor without a standard filter. Note this is the opposite default to OMOPConceptSource.descendants; the difference is deliberate and declared here rather than left implicit.

False
include_classification bool

Widens require_standard to admit classification ('C') concepts. Defaults to True so a governed group can be anchored on a classification node (ATC, for example) without silently losing it. Has no effect unless require_standard is set.

True

exact_ids

exact_ids() -> tuple[int, ...]

Governed members matched directly, without descendant expansion.

excluded_parent_ids

excluded_parent_ids() -> tuple[int, ...]

Governed anchors whose descendants are subtracted.

expression_for

expression_for(
    column: SQLColumnExpression[Any],
) -> sa.ColumnElement[bool]

SQL membership for this group, as a subquery over concept_ancestor.

Available on the spec because the SQL form needs no session: the traversal is performed by the database when the query runs. That is what lets a hybrid_property expose the same governed set at class level, where no session exists.

Deliberately a subquery rather than a literal IN list built from a resolved group: a closure of tens of thousands of IDs degrades query plans and can exceed driver parameter limits.

parent_ids

parent_ids() -> tuple[int, ...]

Governed descendant-expanding anchors.

ConceptResolver

ConceptResolver(
    index: LookupIndex,
    *,
    normalizer: Normaliser | None = None,
    corrections: list[Callable[[str], str]] | None = None,
)

Runtime resolver for mapping free-text terms to OMOP concept IDs.

A ConceptResolver wraps a pre-built LookupIndex and applies runtime normalisation and optional correction passes to resolve arbitrary input strings to concept IDs. It is intentionally lightweight and stateless: all semantic scope and vocabulary constraints are encoded upstream in the LookupSpec and LookupIndex.

Resolution proceeds in ordered stages: 1. Apply the primary normaliser to the input term and attempt a direct lookup. 2. If no hit is found, apply each correction function in turn, re-normalise, and retry the lookup. 3. If no match is found, return the configured unknown concept ID.

This design allows simple, explicit handling of common data quality issues (e.g. formatting differences, legacy codes, mild normalisation errors) without introducing fuzzy matching, probabilistic scoring, or hidden inference logic.

Parameters:

Name Type Description Default
index LookupIndex

Pre-built LookupIndex providing the normalised key → concept_id mapping.

required
normalizer Normaliser | None

Optional normalisation function applied to input terms at lookup time. Defaults to normalize_default. This should be compatible with the normaliser used when constructing the LookupIndex.

None
corrections list[Callable[[str], str]] | None

Optional ordered list of correction functions applied to the raw input term prior to normalisation and lookup. Each correction is tried in sequence until a match is found.

None
Notes
  • ConceptResolver performs no database access and no dynamic expansion of vocabularies; it operates over the materialised LookupIndex.
  • Resolution is deterministic and transparent: there is no fuzzy matching, ranking, or probabilistic inference.
  • Correction functions are applied conservatively and in-order; later corrections do not override earlier successful matches.
  • lookup_exact bypasses correction passes and performs a single normalised lookup, which is useful for validation and debugging.

Examples:

>>> resolver = ConceptResolver(index)
>>> resolver.lookup("Stage III")
123456
>>> resolver.lookup("stage-3")
123456

all_concepts cached property

all_concepts: set[int]

Every concept ID reachable through this resolver's index.

Cached: the index is fixed at construction, and callers legitimately union several resolvers' sets, which previously rebuilt each one per access. Returned by reference, so treat it as read-only.

estimated_bytes

estimated_bytes() -> int

Approximate retained size, for cache accounting.

A name/code/synonym index costs several times more per concept than a bare ID set, which is why the cache bound is measured in bytes rather than entry counts. Measured at ~350 bytes per concept for OMOP-length names and codes with one synonym each.

ConceptResolverRegistry

ConceptResolverRegistry(
    engine: Engine,
    *,
    max_bytes: int = DEFAULT_MAX_CACHE_BYTES,
)

Bases: _LazyBoundedRegistry[ConceptResolver]

Lazy registry for ConceptResolvers.

Resolvers are constructed on first access and cached for the lifetime of this registry instance. The registry is scoped to a SQLAlchemy Engine, ensuring vocab lookups are built once per database.

LookupIndex dataclass

LookupIndex(
    name: str, unknown: int | None, mapping: dict[str, int]
)

Materialised lookup table from normalised text keys to OMOP concept IDs.

A LookupIndex is the runtime artifact produced by a LookupSpec and a ConceptSource. It represents a flat, precomputed mapping from one or more normalised string representations (e.g. concept names, codes, synonyms) to OMOP concept IDs.

Attributes:

Name Type Description
name str

Human-readable identifier for the lookup.

unknown int | None

Concept ID to return when a lookup fails, or None if failures should propagate as null.

mapping dict[str, int]

Dictionary mapping normalised string keys to OMOP concept IDs. Keys are expected to already be normalised at build time.

Notes

The mapping may contain multiple textual representations pointing to the same concept ID (e.g. name + code + synonym).

LookupSpec dataclass

LookupSpec(
    name: str,
    unknown: int | None = 0,
    domain_id: str | None = None,
    concept_class_id: list[str] | None = None,
    vocabulary_id: list[str] | None = None,
    require_standard: bool = True,
    include_classification: bool = True,
    require_active: bool = False,
    code_filter: str | None = None,
    parents: list[int] | None = None,
    include_non_standard_descendants: bool = False,
    include_synonyms: bool = False,
    normalizer: Normaliser = normalize_default,
    include: tuple[str, ...] = (
        "concept_name",
        "concept_code",
    ),
)

Declarative specification for constructing a vocabulary lookup index.

A LookupSpec defines what concepts should be included in a lookup and which textual representations should be indexed for resolution.

The spec is consumed by a OMOPConceptSource, which materialises a LookupIndex by querying an OMOP vocabulary source and extracting the requested fields.

This separation allows lookup semantics (domain, vocabulary, hierarchy, standardness, synonyms, normalisation) to be expressed explicitly and versioned independently of runtime resolution logic.

Attributes:

Name Type Description
name str

Stable identifier for this lookup specification.

unknown int | None

Concept ID to return for unmatched terms. Set to None to preserve nulls, or to a sentinel concept ID to force closed-world behaviour.

domain_id str | None

Optional OMOP domain filter

concept_class_id list[str] | None

Optional list of OMOP concept_class_id values to restrict the lookup

vocabulary_id list[str] | None

Optional list of OMOP vocabulary_id values to restrict the lookup

require_standard bool

If True, restricts the lookup to concepts carrying a standardness flag. Named to match ConceptFilter.require_standard, which it delegates to.

include_classification bool

Widens require_standard to admit classification ('C') concepts. Defaults to True: a lookup exists to recognise vocabulary terms, and a classification concept is a legitimate thing to recognise even though it is not a valid mapping target. Set False for selection-shaped lookups.

require_active bool

If True, excludes concepts with an invalid_reason. Defaults to False so recognition stays permissive — a deprecated concept that matches the text can still be resolved forward through "Maps to" / "Concept replaced by", whereas filtering it out here discards the term entirely.

code_filter str | None

Optional substring filter applied to concept_code (ILIKE-based). Useful for coarse scoping (e.g. AJCC-only codes).

parents list[int] | None

Optional list of ancestor concept IDs from which to expand the lookup via the Concept_Ancestor table.

include_non_standard_descendants bool

If True, includes non-standard concepts when expanding from parents. Has no effect if parents is None.

include_synonyms bool

If True, include Concept_Synonym entries in the lookup keys.

normalizer Normaliser

Function applied to all indexed strings at build time. This should match (or be compatible with) the normalisation used at resolution time by ConceptResolver.

include tuple[str, ...]

Tuple of ConceptRow attribute names to index as keys (e.g. ("concept_name", "concept_code")). This controls which textual fields become resolvable inputs.

Notes
  • LookupSpec encodes semantic intent; LookupIndex encodes runtime state.
  • Specs are designed to be stable, inspectable configuration objects that can be versioned and reviewed as part of phenotype or ETL definitions.
  • Normalisation and correction policies are intentionally split between build-time (this spec) and runtime (ConceptResolver) to make lookup behaviour explicit and testable.

OMOPConceptSource

Concrete ConceptSource backed by OMOP CDM vocabulary tables.

It is a thin, explicit adapter between SQLAlchemy + OMOP CDM and higher-level vocabulary indexing logic.

Used exclusively to builds a query based on provided parameters (adds
filter for each non-None parameter, and joins to Concept_Ancestor if parents are specified).

fetch_concepts staticmethod

fetch_concepts(
    session: Session,
    *,
    domain_id: str | None = None,
    concept_class_id: Iterable[str] | None = None,
    vocabulary_id: Iterable[str] | None = None,
    require_standard: bool = True,
    include_classification: bool = True,
    require_active: bool = False,
    code_filter: str | None = None,
    parents: Iterable[int] | None = None,
    include_non_standard_descendants: bool = False,
) -> list[ConceptRow]

Fetch concepts matching the provided constraints.

This method supports two primary modes: 1. Flat filtering by domain / class / vocabulary 2. Hierarchical expansion from parent concept(s)

Domain, vocabulary, standardness and validity are delegated to :class:~omop_alchemy.cdm.query.ConceptFilter so this layer shares one implementation of the OMOP flag rules with the rest of the package. In particular the flag comparisons are normalised, so a blank or whitespace-padded standard_concept is classified the same way here as it is by Concept.is_standard.

Recognition is permissive by default, because this layer resolves text to a concept, not a concept to a mapping target. Both defaults follow from that:

  • include_classification is True — classification concepts are legitimate things to recognise, even though they are not valid mapping targets.
  • require_active is False — a deprecated concept that matches the text is a useful hit, because the caller can follow Maps to / Concept replaced by to a valid successor. Filtering it out at recognition time discards the term entirely, with nothing to resolve forward from.

Pass include_classification=False / require_active=True when the index feeds concept selection rather than recognition, where the strict OMOP mapping-target rules apply.

fetch_synonyms staticmethod

fetch_synonyms(
    session: Session,
    *,
    concept_ids: Iterable[int] | None = None,
) -> list[tuple[int, str]]

Return (concept_id, synonym) pairs for concept synonyms.

The join to concept scopes the result to synonyms whose concept actually exists, and concept_ids narrows it further to a caller-supplied set. Both are applied in SQL: without them this streams every synonym row in the vocabulary back to Python to be discarded, which on a full Athena load is millions of rows for a lookup covering a handful of domains.

ResolvedConceptGroup dataclass

ResolvedConceptGroup(
    spec: ConceptGroupSpec, ids: frozenset[int]
)

A governed group expanded against one vocabulary.

Membership is (descendants(parents) - descendants(excluded parents)) | exact_ids. Exact members are explicit inclusions and are never subtracted by the exclusion step, matching omop-semantics' composition contract.

estimated_bytes

estimated_bytes() -> int

Approximate retained size, for cache accounting.

Measured at ~80 bytes per ID for a frozenset of OMOP-magnitude concept IDs: a distinct PyLong each, plus hash-table overhead at a 0.6 load factor.

expression

expression(
    column: SQLColumnExpression[Any],
) -> sa.ColumnElement[bool]

SQL membership for this group — delegates to the spec.

Kept here so a caller holding a resolved group can reach either rendering without going back to the spec, but the definition lives in one place.

build_concept_group

build_concept_group(
    session: Session, spec: ConceptGroupSpec
) -> ResolvedConceptGroup

Expand a governed group against the vocabulary behind session.

Performs up to two queries — the include closure and, when the group declares exclusions, the exclude closure. Callers should normally go through the cache in registry rather than calling this directly.

clear_concept_group_cache

clear_concept_group_cache() -> None

Drop every cached group expansion, across all vocabularies.

Per-vocabulary keying means this is rarely needed — moving database gives a different identity and therefore a different registry. It remains an escape hatch for a dataset reloaded in place under an unchanged identity.

clear_vocabulary_identity

clear_vocabulary_identity(engine: Engine) -> None

Forget engine's registered identity, if it had one.

concept_group_cache_stats

concept_group_cache_stats() -> dict[
    str | int, dict[str, int]
]

Per-scope cache statistics, for monitoring the bound.

Watch rebuilds_after_evict: while it is zero the shared bound is holding, and if it climbs the per-scope byte totals are the evidence for raising it or splitting the budget by payload kind.

concept_group_registry

concept_group_registry(
    session: Session,
) -> ConceptGroupRegistry

Return the group registry for the vocabulary behind session.

Registries are keyed on vocabulary identity where one has been registered (see :mod:.identity), so recreating an engine against the same dataset reuses expansions. Otherwise they are keyed weakly on the engine, which is still built-once-per-engine but is not shared across engines.

In-memory SQLite intentionally lands in the second case: two such engines are separate databases despite identical configuration, so cross-engine sharing would serve one database's concept sets for another.

make_concept_resolver

make_concept_resolver(
    session: Session,
    *,
    name: str,
    unknown: int | None = 0,
    domain_id: str | None = None,
    concept_class_id: list[str] | None = None,
    vocabulary_id: list[str] | None = None,
    require_standard: bool = True,
    include_classification: bool = True,
    require_active: bool = False,
    code_filter: str | None = None,
    parents: list[int] | None = None,
    include_non_standard_descendants: bool = False,
    include_synonyms: bool = False,
    include: tuple[str, ...] = (
        "concept_name",
        "concept_code",
    ),
    build_normalizer: Normaliser = normalize_default,
    runtime_normalizer: Normaliser | None = None,
    corrections: list[Callable[[str], str]] | None = None,
) -> ConceptResolver

Convenience factory for constructing a ConceptResolver from declarative inputs.

This function bundles the common workflow of: - defining a LookupSpec - materialising a LookupIndex from OMOP - constructing a ConceptResolver for runtime use

Parameters:

Name Type Description Default
session Session

Active SQLAlchemy session connected to the OMOP CDM database.

required
name str

Stable identifier for this lookup specification, used in logging and debugging.

required
unknown int | None

Concept ID to return for unmatched terms. Set to None to preserve nulls, or to a sentinel concept ID to force closed-world behaviour.

0
domain_id str | None

Optional OMOP domain filter for the concepts to include in the lookup.

None
concept_class_id list[str] | None

Optional list of OMOP concept_class_id values to restrict the lookup.

None
vocabulary_id list[str] | None

Optional list of OMOP vocabulary_id values to restrict the lookup.

None
require_standard bool

If True, restricts the lookup to concepts carrying a standardness flag. Named to match ConceptFilter.require_standard, which it delegates to.

True
include_classification bool

Widens require_standard to admit classification ('C') concepts. Defaults to True: a lookup exists to recognise vocabulary terms, and a classification concept is a legitimate thing to recognise even though it is not a valid mapping target. Set False for selection-shaped lookups.

True
require_active bool

If True, excludes concepts with an invalid_reason. Defaults to False so recognition stays permissive — a deprecated concept that matches the text can still be resolved forward through "Maps to" / "Concept replaced by", whereas filtering it out here discards the term entirely.

False
code_filter str | None

Optional substring filter applied to concept_code (ILIKE-based). Useful for coarse scoping (e.g. AJCC-only codes).

None
parents list[int] | None

Optional list of ancestor concept IDs from which to expand the lookup via the Concept_Ancestor table.

None
include_non_standard_descendants bool

If True, includes non-standard concepts when expanding from parents. Has no effect if parents is None.

False
include_synonyms bool

If True, include Concept_Synonym entries in the lookup keys.

False
include tuple[str, ...]

Tuple of ConceptRow attribute names to index as keys (e.g. ("concept_name", "concept_code")). This controls which textual fields become resolvable inputs.

('concept_name', 'concept_code')
build_normalizer Normaliser

Normalisation function applied to all indexed strings at build time. This should
match (or be compatible with) the normaliser used at resolution time by ConceptResolver.

normalize_default
runtime_normalizer Normaliser | None

Optional normalisation function applied to input terms at lookup time. Defaults to normalize_default. This should be compatible with the normaliser used when constructing the LookupIndex.

None
corrections list[Callable[[str], str]] | None

Optional ordered list of correction functions applied to the raw input term prior to normalisation and lookup

None

register_vocabulary_identity

register_vocabulary_identity(
    engine: Engine, identity: str
) -> None

Declare which vocabulary dataset engine reads.

Concept-set caches keyed on this identity are shared by every engine registered under it, so recreating an engine reuses the expansion.

Pass the engine your factory returns. ResolvedCDMDatabase.create_engine ends with execution_options(schema_translate_map=...), which yields a derived OptionEngine; that is the object sessions bind to, and the one lookups will see.

Do not register an identity for an ephemeral database — notably in-memory SQLite, where two engines built from identical configuration are genuinely separate databases. Those correctly fall back to per-engine caching.

resolve_concept_group

resolve_concept_group(
    session: Session, spec: ConceptGroupSpec
) -> ResolvedConceptGroup

Expand spec against session's vocabulary, cached.

The cache key is spec.name within the vocabulary's registry, so it derives from the governed semantic-unit name rather than a locally invented label.

Patient timelines

Projects a person's clinical rows — conditions, measurements, drug exposures — into a single time-ordered event stream. This has its own dedicated page, since it predates the rest of the toolkit reorg: see Patient Timelines.

Unit conversion

Converts measurement values to canonical units. Kilograms, pounds, centimetres, and inches mean the same thing in every clinical domain, so the conversion rules live here rather than with any one domain's measurement logic — see analytics.body_metrics for where those domain-specific measurements are resolved and normalised using these rules.

Convert measurement values to canonical units.

Measurement rows carry whatever unit the source system recorded. Before values can be compared, trended, or fed into a calculation they need to agree on a unit, and that conversion must be explicit about what it will and will not accept.

BodyUnitConversionRules converts anthropometric measurements to kilograms and centimetres, driven by the unit concept on each row rather than by guesswork. default_body_unit_conversion_rules supplies the standard rule set; INCH_TO_CM and LB_US_TO_KG are the underlying factors where a caller needs them directly.

A reading whose unit concept is unrecognised is not converted and not silently passed through — callers are told the value could not be normalised, so an unconvertible unit never reaches a calculation disguised as a valid one.

BodySizeUnitConcepts dataclass

BodySizeUnitConcepts(
    kg: int, lb: int, cm: int, inch: int, m2: int
)

The unit concepts anthropometric conversion recognises.

Kilograms, pounds, centimetres, inches, and square metres mean the same thing in every clinical domain, so the mapping from concept ID to unit lives here rather than with any one domain's measurement logic.

BodyUnitConversionRules dataclass

BodyUnitConversionRules(units: BodySizeUnitConcepts)

Unit normalisation for anthropometric measurements.

Conversion is driven by the unit concept recorded on each row. A value whose unit concept is not in units converts to None rather than being passed through, so a reading in an unrecognised unit cannot reach a calculation disguised as a valid one.

Deployments that record body size against non-standard unit concepts can supply their own units instead of the governed defaults.