CLI Reference
omop-emb provides a CLI for concept ingestion, similarity search, index management, and diagnostics. Configuration is read from ~/.config/omop/config.toml via oa-configurator; there is no .env file.
Commands are organised into three subcommand groups:
| Group | Purpose |
|---|---|
embeddings |
Ingestion, search, index creation |
maintenance |
Model management, FAISS export/import |
diagnostics |
Health checks |
Run omop-emb <group> --help to list commands within a group.
Verbosity flag placement
The --verbose / -v flag is a global option and must appear before the subcommand name, not after it:
omop-emb -v embeddings add-embeddings # ✓ correct
omop-emb embeddings add-embeddings -v # ✗ flag is ignored
Use -v for INFO level and -vv for DEBUG level.
Prerequisites
- Backend installed:
pip install omop-emb(sqlite-vec) orpip install "omop-emb[pgvector]". - Configured via oa-configurator:
omop-config configure omop_emb(see Getting Started: Configuration). Resolvescdm_db,embedding_model_name, andvector_store_name. - OMOP CDM (
cdm_db): required only for concept ingestion (add-embeddings,add-embeddings-with-index), forprune-embeddings, and for enrichingsearchresults with concept names. Not required forlist-models,rebuild-index,delete-model, or diagnostics.
embeddings group
add-embeddings
Bulk-generate and store embeddings for OMOP concepts that do not yet have embeddings. Models are registered with a FLAT index; use maintenance rebuild-index afterwards to build an HNSW index.
omop-emb embeddings add-embeddings [OPTIONS]
Embedding API Options
| Option | Short | Default | Description |
|---|---|---|---|
--model-name |
-m |
value configured via omop-config | Name of a [models.*] entry (see omop-config models add/list). |
--batch-size |
-b |
100 |
Concepts per API batch. |
Concept Filters
| Option | Short | Default | Description |
|---|---|---|---|
--standard-only |
False |
Embed only standard concepts (standard_concept = 'S'). |
|
--include-invalid |
False |
Also embed invalid concepts (invalid_reason of D or U). They are skipped by default. |
|
--vocabulary |
None |
Restrict to specific OMOP vocabularies (repeatable). | |
--domain |
None |
Restrict to specific OMOP domains (repeatable). | |
--num-embeddings |
-n |
None |
Cap on total concepts processed (useful for testing). |
Index Options
| Option | Default | Description |
|---|---|---|
--metric-type |
cosine |
Metric of a new model's table. Locked in for flat on sqlite-vec; ignored for flat on pgvector and for a model that is already registered. See Metrics. |
| Option | Short | Description |
|---|---|---|
--verbose |
-v |
Increase log verbosity (pass twice for DEBUG). |
add-embeddings-with-index
Ingest embeddings and immediately build an index in one step. Equivalent to running add-embeddings followed by create-index.
omop-emb embeddings add-embeddings-with-index [OPTIONS]
Accepts all options from add-embeddings (its --metric-type also applies to the index), plus:
Index Options
| Option | Default | Description |
|---|---|---|
--index-type |
flat |
Index to build after ingestion (flat or hnsw). |
--index-hnsw-num-neighbors |
None |
HNSW graph connectivity (M). |
--index-hnsw-ef-search |
None |
HNSW query recall parameter. |
--index-ef-construction |
None |
HNSW build quality parameter. |
create-index
Build or rebuild the index for a model that already has embeddings stored.
omop-emb embeddings create-index [OPTIONS]
Embedding API Options
| Option | Short | Default | Description |
|---|---|---|---|
--model-name |
-m |
value configured via omop-config | Name of a [models.*] entry to build the index for. |
Index Options
| Option | Default | Description |
|---|---|---|
--index-type |
flat |
flat or hnsw. |
--metric-type |
cosine |
Distance metric. Locked in for hnsw and for flat on sqlite-vec; ignored for flat on pgvector. See Metrics. |
--index-hnsw-num-neighbors |
None |
HNSW M parameter. |
--index-hnsw-ef-search |
None |
HNSW query recall parameter. |
--index-ef-construction |
None |
HNSW build quality parameter. |
| Option | Short | Description |
|---|---|---|
--verbose |
-v |
Increase log verbosity. |
search
Query stored embeddings for nearest OMOP concepts. Outputs tab-separated rows: query_id, query_text, rank, concept_id, similarity, concept_name.
If cdm_db is configured, results are enriched with concept names from the CDM. Without it, the concept_name column is left empty (a RuntimeError from CDM resolution is caught and logged, not fatal).
omop-emb embeddings search --query "hypertension" [OPTIONS]
Embedding API Options
| Option | Short | Default | Description |
|---|---|---|---|
--model-name |
-m |
value configured via omop-config | Name of a [models.*] entry. |
--batch-size |
-b |
100 |
Batch size for embedding generation. |
Search Options
| Option | Default | Description |
|---|---|---|
--query |
None |
Query text (repeatable). At least one of --query or --queries-file is required. |
--queries-file |
None |
Path to a .txt file with one query per line. |
--metric-type |
cosine |
Distance metric for search. |
--k |
10 |
Number of nearest concepts to return per query. |
--faiss-cache-dir |
vector store's configured faiss_cache_dir |
Use a FAISS sidecar index instead of the primary backend. Requires omop-emb[faiss-cpu]. |
Concept Filters
| Option | Default | Description |
|---|---|---|
--standard-only |
False |
Return only standard OMOP concepts. |
--vocabulary |
None |
Filter results to specific vocabularies (repeatable). |
--domain |
None |
Filter results to specific domains (repeatable). |
| Option | Short | Description |
|---|---|---|
--verbose |
-v |
Increase log verbosity. |
maintenance group
list-models
List all registered embedding models in the configured backend.
omop-emb maintenance list-models [OPTIONS]
| Option | Short | Default | Description |
|---|---|---|---|
--model |
-m |
None |
Filter by model name. |
--provider-type |
None |
Filter by provider. | |
--verbose |
-v |
Increase log verbosity. |
rebuild-index
Build or rebuild the storage index for an already-registered model. Use this to switch between FLAT and HNSW without re-ingesting. The canonical model name is passed directly via --model; supply --provider-type to canonicalize a raw name if needed.
omop-emb maintenance rebuild-index --model <CANONICAL_NAME> [OPTIONS]
| Option | Short | Default | Description |
|---|---|---|---|
--model |
-m |
required | Canonical model name. |
--provider-type |
None |
Provider used to canonicalize the model name when needed. |
Index Options
| Option | Default | Description |
|---|---|---|
--index-type |
flat |
flat or hnsw. |
--metric-type |
cosine |
Distance metric. Locked in for hnsw and for flat on sqlite-vec, where another metric recreates the table from the stored vectors. Ignored for flat on pgvector. See Metrics. |
--index-hnsw-num-neighbors |
None |
HNSW M parameter. |
--index-hnsw-ef-search |
None |
HNSW query recall parameter. |
--index-ef-construction |
None |
HNSW build quality parameter. |
--vacuum / --no-vacuum |
--vacuum |
Maintenance after the rebuild. pgvector runs VACUUM (ANALYZE) on the table to refresh planner statistics. sqlite-vec runs VACUUM on the database file after a metric change, which needs free disk space for a second copy of the data. Neither returns the space deleted rows left behind, see prune-embeddings --compact. Use --no-vacuum to skip it. |
| Option | Short | Description |
|---|---|---|
--verbose |
-v |
Increase log verbosity. |
delete-model
Permanently delete a registered model and all its stored embeddings. This operation is irreversible.
omop-emb maintenance delete-model --model <NAME> [OPTIONS]
| Option | Short | Default | Description |
|---|---|---|---|
--model |
-m |
required | Canonical model name. |
--provider-type |
None |
Provider used to canonicalize the model name when needed. | |
--yes |
-y |
False |
Skip confirmation prompt. |
--verbose |
-v |
Increase log verbosity. |
prune-embeddings
Remove stored embeddings whose concept is outside the given scope or no longer exists in the CDM. By default the scope is all valid concepts, so invalid concepts embedded earlier are removed. Without --apply the command only reports what would be deleted. Applying a prune also marks the model as updated, so FAISS caches built before it report as stale.
omop-emb maintenance prune-embeddings --model <NAME> [OPTIONS]
| Option | Short | Default | Description |
|---|---|---|---|
--model |
-m |
required | Canonical model name. |
--provider-type |
None |
Provider used to canonicalize the model name when needed. | |
--apply |
False |
Delete the rows. Without it the command is a dry run. | |
--compact |
False |
After deleting, return the disk space held by deleted and rewritten rows. Needs --apply. |
|
--verbose |
-v |
Increase log verbosity. |
Scope
| Option | Default | Description |
|---|---|---|
--include-invalid |
False |
Keep invalid concepts instead of pruning them. |
--standard-only |
False |
Keep only standard concepts. |
--vocabulary |
None |
Keep only these vocabularies (repeatable). |
--domain |
None |
Keep only these domains (repeatable). |
Reclaiming space
Deleting rows does not shrink the store on disk. --compact rewrites the model's storage to return the space, and also runs when nothing is left to prune, so it can follow earlier prunes or re-embedding (sqlite-vec upserts delete and insert, which leaves the same gaps).
- sqlite-vec recreates the table from its stored vectors when that frees chunks, then runs
VACUUMon the database file. It needs free disk space for a second copy, and does nothing when no chunk can be freed. - pgvector runs
VACUUM FULLon the model's table, which rewrites it and rebuilds its indexes and holds an exclusive lock on it while it runs. PlainVACUUMonly makes dead space reusable.
Without --compact, rebuild-index still shrinks the HNSW index of a pgvector model but not its table.
export
Stream all embeddings for one model from the primary backend into a single, self-describing HDF5 bundle (raw vectors, never normalized, plus concept metadata). This is the lossless source of truth for backup/restore and for migrating raw embeddings to another backend (see import). Peak memory stays close to one batch's worth regardless of table size.
Stored vectors carry no metric. A model locked to one (HNSW, or sqlite-vec) carries it inside its index_config, and a backend that does not lock FLAT models (pgvector) drops it on import. Bundles are written as schema version 2. Version 1 bundles, which also stored a metric_type, still import. Export opens the store read-only. The output filename is derived from the model's storage identifier, not a literal path you choose.
omop-emb maintenance export --model <NAME> --output-dir <DIR> [OPTIONS]
| Option | Short | Default | Description |
|---|---|---|---|
--model |
-m |
required | Canonical model name. |
--output-dir |
-o |
required | Directory to write the HDF5 bundle into. The filename is derived from the model's storage identifier. |
--provider-type |
None |
Provider used to canonicalize the model name when needed. | |
--batch-size |
-b |
100000 |
Rows streamed from the backend (and written to the bundle) per batch. |
| Option | Short | Description |
|---|---|---|
--verbose |
-v |
Increase log verbosity. |
build-faiss-cache
Build (or rebuild) a local FAISS search-acceleration cache directly from the backend. FAISS indices are a derived, disposable artifact, never the round-trip source of truth. This function streams every embedding for the model straight out of the backend in bounded-memory batches; it has no dependency on export/import or any bundle file. Requires pip install "omop-emb[faiss-cpu]".
omop-emb maintenance build-faiss-cache --model <NAME> --faiss-cache-dir <DIR> [OPTIONS]
| Option | Short | Default | Description |
|---|---|---|---|
--model |
-m |
required | Canonical model name to build the FAISS cache for. |
--faiss-cache-dir |
required | Root directory for FAISS index files. | |
--provider-type |
None |
Provider used to canonicalize the model name when needed. | |
--batch-size |
-b |
100000 |
Rows streamed from the backend per batch. |
Index Options
| Option | Default | Description |
|---|---|---|
--metric-type |
cosine |
Distance metric for the FAISS index (cosine or l2). |
--index-type |
flat |
FAISS index type: flat (exact) or hnsw (approximate). |
--hnsw-m |
32 |
HNSW number of neighbours. Only used when --index-type=hnsw. |
| Option | Short | Description |
|---|---|---|
--verbose |
-v |
Increase log verbosity. |
check-faiss-cache
Check whether the FAISS index on disk is fresh relative to the primary backend. Exits with code 0 if fresh, 1 if stale or missing.
omop-emb maintenance check-faiss-cache --model <NAME> --cache-dir <DIR> [OPTIONS]
| Option | Short | Default | Description |
|---|---|---|---|
--model |
-m |
required | Canonical model name. |
--cache-dir |
required | Root cache directory. | |
--provider-type |
None |
Provider used to canonicalize the model name when needed. |
Index Options
| Option | Default | Description |
|---|---|---|
--metric-type |
cosine |
Metric of the index to check. |
--index-type |
flat |
Index type to check (flat or hnsw). |
| Option | Short | Description |
|---|---|---|
--verbose |
-v |
Increase log verbosity. |
import
Import embeddings from an export bundle into the primary backend. Reads raw vectors straight from the bundle's embeddings dataset so magnitudes are preserved exactly regardless of metric. Registers the model from the bundle's own metadata if it isn't already registered. A brand-new registration is backdated to the bundle's own exported_at (the source data's snapshot time) rather than "now" (see "Reusing a FAISS cache across machines" below for why this matters).
omop-emb maintenance import --bundle-file <FILE.h5> [OPTIONS]
| Option | Short | Default | Description |
|---|---|---|---|
--bundle-file |
required | Path to the HDF5 bundle produced by export. |
|
--force |
False |
Overwrite existing embeddings without prompting. | |
--batch-size |
-b |
10000 |
Vectors upserted per backend call. |
--rebuild-index |
False |
Build the index recorded in the bundle's own index_config right after import. |
| Option | Short | Description |
|---|---|---|
--verbose |
-v |
Increase log verbosity. |
Reusing a FAISS cache across machines
A FAISS cache built on one machine can be shipped alongside its export bundle and reused directly on another, without rebuilding it. The build order matters: run export then build-faiss-cache from the same backend, so the cache's exported_at is later than the bundle's. Ship both the .h5 bundle and the FAISS cache directory together. On the target machine:
omop-emb maintenance import --bundle-file model.h5
Point faiss_cache_dir (or --cache-dir) at the copied cache directory: check-faiss-cache now reports it as fresh, since the freshly-registered model is backdated to the bundle's exported_at, which the shipped cache's own exported_at is newer than.
Caveat: passing --rebuild-index (or running rebuild-index at any point afterward) bumps the registry's updated_at back to "now" and invalidates any cache shipped this way. Run build-faiss-cache again in that case. This only matters when the bundle's index_config is HNSW (registration always starts as FLAT); for FLAT bundles, skip --rebuild-index and the shipped cache stays valid.
diagnostics group
health-check
Verify backend connectivity and list registered models with embedding counts.
omop-emb diagnostics health-check [--verbose]
| Option | Short | Description |
|---|---|---|
--verbose |
-v |
Increase log verbosity. |