API Reference

This reference is automatically generated from the source code.

omop_llm

backend

ModelBackend: the one calling contract every consumer uses.

A thin wrapper around a single, already-constructed any-llm provider instance (an entry of :data:omop_llm.providers.registry.PROVIDER_REGISTRY). Chat completion, embeddings, and structured extraction are all methods on one object, gated by :class:~omop_llm.capabilities.Capabilities, rather than split across separate classes per modality.

Every method has a synchronous form and an async_-prefixed asynchronous form (complete/async_complete, embed_texts/async_embed_texts, and so on). This was a deliberate choice, not an oversight.

Consumers only ever see this class, never a raw any-llm provider instance. If any-llm needed replacing, only this module's method bodies, and the providers/ subclasses, would need to change.

ModelBackend dataclass

One resolved, ready-to-call model.

Built by :func:build_model_backend. Wraps a single constructed any-llm provider instance and binds model/configuration to it, so callers do not repeat them on every call.

Parameters:

Name	Type	Description	Default
_client	AnyLLM	The constructed any-llm provider instance backing this backend.	required
model	str	The canonical model name or identifier passed to the underlying provider.	required
capabilities	Capabilities	What this resolved backend can actually do.	required
configuration	dict	Default keyword arguments merged into every call, overridden by any argument the caller passes explicitly. Pure provider passthrough only, and doesn't include any of the other fields below.	dict()
embedding_dim	int	Configured embedding dimension override, read by :meth:dimensions.	None
document_prefix	str	Prefix prepended to document/passage text before embedding.	None
query_prefix	str	Prefix prepended to query text before embedding.	None
_api_base	str	The base URL this backend was constructed with, if any. Threaded through to provider-specific fast paths such as :meth:~omop_llm.providers.supported.OllamaProvider.embedding_dimension_hint.	None

provider property

The provider key this backend was resolved to, e.g. "ollama".

Read directly off _client's own any-llm PROVIDER_NAME class attribute rather than stored separately at construction time, so there is exactly one place this string is ever defined (see :data:omop_llm.providers.registry.PROVIDER_REGISTRY, whose keys are derived from the same attribute).

async_complete(messages, *, tools=None, response_format=None, max_tokens=None, temperature=None, reasoning_effort=None, **kwargs) async

Run one chat completion.

Parameters:

Name	Type	Description	Default
messages	list of dict	Chat history in OpenAI message format.	required
tools	list of dict	Raw OpenAI-style tool schema list. any-llm normalizes tool-call parsing per provider, so callers doing multi-turn agentic tool use pass the same schema regardless of which provider is resolved. Requires self.capabilities.tool_use.	None
response_format	dict or type	A raw JSON-schema dict, or a Pydantic model class. any-llm translates a Pydantic class into each provider's own native structured-output mechanism. See :meth:extract/:meth:async_extract for a convenience method that validates and unwraps the result.	None
max_tokens	int	Maximum number of tokens to generate.	None
temperature	float	Sampling temperature.	None
reasoning_effort	ReasoningEffort	Requested extended-thinking effort, any-llm's own normalized parameter across providers. Only meaningful when self.capabilities.extended_thinking is True; a provider without reasoning support ignores it.	None
**kwargs	Any	Additional provider-specific arguments, passed through unchanged. stream is rejected: this method always returns a ChatCompletion, never a chunk iterator, and streaming is not designed or supported here yet.	{}

Returns:

Type	Description
ChatCompletion	The completion response.

async_dimensions() async

Discover this model's embedding dimensionality. Three tiers: 1. a configured override (self.embedding_dim), 2. a provider-specific fast path (e.g. Ollama's POST /api/show), and 3. a live probe (embed one short string and measure the vector).

Returns:

Type	Description
int	The embedding vector length.

Raises:

Type	Description
UnsupportedCapabilityError	If self.capabilities.embeddings is False.

async_embed_texts(texts, *, role=None, batch_size=None) async

Embed a batch of texts asynchronously.

Parameters:

Name	Type	Description	Default
texts	list of str	Texts to embed.	required
role	EmbeddingRole	Whether texts are being indexed (DOCUMENT) or used to search (QUERY). When given, prepends whichever of self.document_prefix/self.query_prefix matches, needed for asymmetric embedding models (e.g. nomic-embed-text, E5, BGE). Omit for symmetric models, or when texts are already prefixed.	None
batch_size	int	If given, texts is chunked into sub-batches of at most this size, each sent as its own call, rather than one call with the entire list. Useful for bulk callers embedding more texts than a single provider request should carry. Default is None (one call for the whole list).	None

Returns:

Type	Description
list of list of float	One embedding vector per input text, in the same order.

Raises:

Type	Description
UnsupportedCapabilityError	If self.capabilities.embeddings is False.
ValueError	If batch_size is not a positive integer.

async_extract(messages, response_model, *, max_retries=0, **kwargs) async

Extract one validated response_model instance from a chat call.

A thin convenience method built on :meth:async_complete with response_format=response_model. Checks that a parsed instance actually came back, and unwraps it.

Notes

max_retries is native (not instructor-based), so it works for all providers, unlike :func:omop_llm.structured.extract_with_retry.

Parameters:

Name	Type	Description	Default
messages	list of dict	Chat history in OpenAI message format.	required
response_model	type of BaseModel	The Pydantic model to constrain and validate the response against.	required
max_retries	int	Number of additional attempts after a validation failure. Default is 0, i.e. fail immediately.	0
**kwargs	Any	Additional arguments forwarded to :meth:async_complete.	{}

Returns:

Type	Description
BaseModel	A validated instance of response_model.

Raises:

Type	Description
UnsupportedCapabilityError	If self.capabilities.structured_output is False.
NoParsedOutputError	If the provider returned no parsed instance (refusal or empty content), after exhausting max_retries.
LengthFinishReasonError	If the response was truncated before completing.
ContentFilterFinishReasonError	If a content filter blocked the response.
ValidationError	If the model's output does not match response_model's schema, after exhausting max_retries.

async_is_available(**kwargs) async

Check whether this backend can actually be reached, asynchronously.

Probes list_models against the resolved provider. Swallows any error and reports False rather than raising, since the point of a health check is to answer "can I use this," not to propagate the specific failure.

Parameters:

Name	Type	Description	Default
**kwargs	Any	Forwarded to the underlying alist_models call, e.g. timeout=2.0.	{}

Returns:

Type	Description
bool	Whether listing models against this backend succeeded.

complete(messages, *, tools=None, response_format=None, max_tokens=None, temperature=None, reasoning_effort=None, **kwargs)

Run one chat completion synchronously. See :meth:async_complete for parameters.

dimensions()

Discover this model's embedding dimensionality synchronously. Three tiers: 1. a configured override (self.embedding_dim), 2. a provider-specific fast path (e.g. Ollama's POST /api/show), and 3. a live probe (embed one short string and measure the vector).

Returns:

Type	Description
int	The embedding vector length.

Raises:

Type	Description
UnsupportedCapabilityError	If self.capabilities.embeddings is False.

embed_texts(texts, *, role=None, batch_size=None)

Embed a batch of texts.

Parameters:

Name	Type	Description	Default
texts	list of str	Texts to embed.	required
role	EmbeddingRole	Whether texts are being indexed (DOCUMENT) or used to search (QUERY). When given, prepends whichever of self.document_prefix/self.query_prefix matches, needed for asymmetric embedding models (e.g. nomic-embed-text, E5, BGE). Omit for symmetric models, or when texts are already prefixed.	None
batch_size	int	If given, texts is chunked into sub-batches of at most this size, each sent as its own call, rather than one call with the entire list. Useful for bulk callers embedding more texts than a single provider request should carry. Default is None (one call for the whole list).	None

Returns:

Type	Description
list of list of float	One embedding vector per input text, in the same order.

Raises:

Type	Description
UnsupportedCapabilityError	If self.capabilities.embeddings is False.
ValueError	If batch_size is not a positive integer.

extract(messages, response_model, *, max_retries=0, **kwargs)

Extract one validated response_model instance from a chat call synchronously.

A thin convenience method built on :meth:complete with response_format=response_model. Checks that a parsed instance actually came back, and unwraps it.

Notes

max_retries is native (not instructor-based), so it works for all providers, unlike :func:omop_llm.structured.extract_with_retry.

Parameters:

Name	Type	Description	Default
messages	list of dict	Chat history in OpenAI message format.	required
response_model	type of BaseModel	The Pydantic model to constrain and validate the response against.	required
max_retries	int	Number of additional attempts after a validation failure. Default is 0, i.e. fail immediately.	0
**kwargs	Any	Additional arguments forwarded to :meth:complete.	{}

Returns:

Type	Description
BaseModel	A validated instance of response_model.

Raises:

Type	Description
UnsupportedCapabilityError	If self.capabilities.structured_output is False.
NoParsedOutputError	If the provider returned no parsed instance (refusal or empty content), after exhausting max_retries.
LengthFinishReasonError	If the response was truncated before completing.
ContentFilterFinishReasonError	If a content filter blocked the response.
ValidationError	If the model's output does not match response_model's schema, after exhausting max_retries.

is_available(**kwargs)

Check whether this backend can actually be reached, synchronously.

Probes list_models against the resolved provider. Swallows any error and reports False rather than raising, since the point of a health check is to answer "can I use this," not to propagate the specific failure.

Parameters:

Name	Type	Description	Default
**kwargs	Any	Forwarded to the underlying list_models call, e.g. timeout=2.0.	{}

Returns:

Type	Description
bool	Whether listing models against this backend succeeded.

build_model_backend(provider, model, *, model_capabilities, base_url=None, api_key=None, configuration=None, embedding_dim=None, document_prefix=None, query_prefix=None)

Resolve a provider and model into a ready-to-call backend.

Plain keyword arguments in, a :class:ModelBackend out, mirroring the shape oa-configurator's own database resolution already uses (Resolver(stack).resolve_resource(name).create_engine(**kwargs) returns a plain sqlalchemy.Engine, no intermediate config object). See :func:build_model_backend_from_resolved for the oa-configurator integration built on top of this function.

Canonicalizes model for the resolved provider (see :func:omop_llm.providers.registry.canonical_model_name), so a :class:ModelBackend's model attribute is always canonical.

Parameters:

Name	Type	Description	Default
provider	str	A key in :data:omop_llm.providers.registry.PROVIDER_REGISTRY.	required
model	str	Raw model name or identifier; canonicalized before use.	required
model_capabilities	Capabilities	What this specific model is declared to support. Required, not optional: neither any-llm nor omop-llm can introspect this per model, so the caller has to say. Pass Capabilities() to declare none of them, explicitly rather than by omission.	required
base_url	str	The base URL for this specific deployment of the provider.	None
api_key	str	The API key for this specific deployment, if one is required.	None
configuration	dict	Default keyword arguments merged into every call this backend makes (e.g. max_tokens, temperature). Pure provider passthrough -- use embedding_dim/document_prefix/query_prefix below for those, never this dict.	None
embedding_dim	int	Configured embedding dimension override.	None
document_prefix	str	Prefix prepended to document/passage text before embedding.	None
query_prefix	str	Prefix prepended to query text before embedding.	None

Returns:

Type	Description
ModelBackend	A backend ready to call, for example, :meth:ModelBackend.complete or :meth:ModelBackend.async_complete.

Raises:

Type	Description
ValueError	If model cannot be made canonical for the resolved provider (e.g. an Ollama name with no explicit tag), or if embedding_dim is given but the effective capabilities don't include embeddings.

build_model_backend_from_resolved(resolved)

Build a backend from an oa-configurator ResolvedModel.

The oa-configurator integration point: oa-configurator itself knows nothing about omop-llm (its ResolvedModel is plain data, the same way ResolvedResource is), so this glue lives here instead, mirroring omop_alchemy.config.create_cdm_engine(resolved: ResolvedResource) -> sa.Engine: a consumer of oa-configurator takes its plain resolved output and does its own construction from it.

A typical caller (e.g. a package's own config module) does::

from oa_configurator import Resolver, load_stack_config
from omop_llm import build_model_backend_from_resolved

stack = load_stack_config()
resolved = Resolver(stack).resolve_model(config.embedding_model)
backend = build_model_backend_from_resolved(resolved)

A thin translator: resolved.embedding_dim/document_prefix/query_prefix are forwarded as-is, and resolved.embeddings/tool_use/structured_output/extended_thinking are collected into one :class:~omop_llm.capabilities.Capabilities. resolved.configuration is passed through untouched -- nothing gets folded into it.

Parameters:

Name	Type	Description	Default
resolved	ResolvedModel	A model resolved via oa_configurator.Resolver.resolve_model().	required

Returns:

Type	Description
ModelBackend	A backend ready to call, for example, :meth:ModelBackend.complete or :meth:ModelBackend.async_complete.

Raises:

Type	Description
ValueError	If resolved.model cannot be made canonical for the resolved provider (e.g. an Ollama name with no explicit tag).

capabilities

What something -- a provider, a model, or a resolved backend -- can do.

Capabilities dataclass

A capability declaration: what something can do. Used for providers, models, and resolved backends (the combination of the two).

Opt-in: every field defaults to False. Neither any-llm nor omop_llm can introspect a model's real capabilities, so nothing is assumed.

Parameters:

Name	Type	Description	Default
streaming	bool	Whether streaming completions are supported.	False
embeddings	bool	Whether the embeddings endpoint is supported.	False
extended_thinking	bool	Whether reasoning/extended-thinking output is supported.	False
tool_use	bool	Whether tool/function calling is supported.	False
structured_output	bool	Whether structured (schema-constrained) output is supported.	False

__and__(other)

Element-wise AND: a capability is only available if both sides have it.

embeddings

Embedding role prefixing: EmbeddingRole and a best-effort sanity check.

Asymmetric embedding models (nomic-embed-text, the E5 family, BGE, and others) are trained with distinct prefixes for the text being indexed versus the text used to search it. Sending text without the correct prefix produces a valid-looking embedding that just retrieves badly, with no error to notice.

EmbeddingRole

Bases: StrEnum

Role of text being embedded, for models with asymmetric prefixes.

apply_embedding_prefix(texts, role, *, document_prefix, query_prefix)

Prepend role's prefix to each of texts, if one is given.

warn_if_prefixes_look_wrong(*, model, document_prefix, query_prefix)

Log a warning for a missing or unrecognized prefix.

Called once, at :func:~omop_llm.backend.build_model_backend time, for any backend that declares embeddings support. Never raises: a prefix outside :data:KNOWN_EMBEDDING_PREFIXES is not necessarily wrong, this is a heads-up, not validation.

errors

Exceptions raised by omop_llm.

NoParsedOutputError

Bases: OmopLlmError

Raised when a structured-output call produced no parsed instance to unwrap.

OmopLlmError

Bases: RuntimeError

Base class for all omop_llm errors.

UnsupportedCapabilityError

Bases: OmopLlmError

Raised when a requested capability is not available on the resolved backend.

UnsupportedProviderError

Bases: OmopLlmError

Raised when a provider key is not in omop_llm's supported registry.

providers

canonical_model_name(provider_key, name)

Canonicalize a model name for one registered provider.

Useful for deciding what to persist as a model's stable identity (e.g. in a database) independently of building a full :class:~omop_llm.backend.ModelBackend. :func:~omop_llm.backend.build_model_backend also calls this internally, so a backend's model attribute is always canonical without callers needing to remember to do it themselves.

Strips surrounding whitespace and rejects an empty result before dispatching to the provider's own canonical_model_name.

Parameters:

Name	Type	Description	Default
provider_key	str	A key expected to be in :data:PROVIDER_REGISTRY.	required
name	str	Raw model name to canonicalize.	required

Returns:

Type	Description
str	The canonical model name for this provider.

Raises:

Type	Description
UnsupportedProviderError	If provider_key is not registered.
ValueError	If name is empty or whitespace-only, or cannot otherwise be made canonical for this provider (e.g. an Ollama name with no explicit tag).

provider_capabilities_for(provider_key)

Build the provider-wide capability ceiling for one registered provider.

This is the provider's transport-level ceiling, not a specific model's effective capabilities.

streaming, embeddings, and extended_thinking come straight from any-llm's own get_provider_metadata(). tool_use and structured_output come from the class attributes each provider subclass declares itself, since any-llm tracks neither.

Parameters:

Name	Type	Description	Default
provider_key	str	A key expected to be in :data:PROVIDER_REGISTRY.	required

Returns:

Type	Description
Capabilities	The capability declaration for this provider.

Raises:

Type	Description
UnsupportedProviderError	If provider_key is not registered.

provider_class_for(provider_key)

Look up a registered provider class.

Parameters:

Name	Type	Description	Default
provider_key	str	A key expected to be in :data:PROVIDER_REGISTRY.	required

Returns:

Type	Description
type of AnyLLM	The provider class registered for provider_key.

Raises:

Type	Description
UnsupportedProviderError	If provider_key is not registered.

supported_providers()

List the provider keys omop_llm will construct a backend for.

Returns:

Type	Description
tuple of str	The registered provider keys, sorted alphabetically.

base

Shared base for omop_llm's own provider subclasses.

ProviderMixin

Bases: ABC

Marks a class as one of omop_llm's own provider subclasses.

Declares the two capabilities any-llm does not track itself (TOOL_USE, STRUCTURED_OUTPUT) as class attributes, alongside any-llm's own SUPPORTS_* flags on the sibling base class. Also declares the two provider-specific hooks a resolved backend needs: :meth:canonical_model_name and :meth:embedding_dimension_hint.

Every provider omop_llm supports gets a real subclass built on this mixin. Any-llm's own base class, AnyLLM, is already an abc.ABC with real abstract methods, so this mixin composes with it safely. canonical_model_name is a required override, not a default passthrough, so adding a new provider forces a deliberate decision about its naming rules rather than silently inheriting "no transformation needed."

Attributes:

Name	Type	Description
TOOL_USE	bool	Whether this provider supports tool/function calling.
STRUCTURED_OUTPUT	bool	Whether this provider supports structured (schema-constrained) output.

async_embedding_dimension_hint(model, *, api_base) async

Look up this model's embedding dimension via a provider-specific fast path.

Default: no fast path available. Override where a provider exposes model metadata directly (e.g. Ollama's POST /api/show). Used as the middle tier of :meth:omop_llm.backend.ModelBackend.async_dimensions, between a configured override and a live embedding probe.

Parameters:

Name	Type	Description	Default
model	str	The canonical model name.	required
api_base	str	The resolved base URL this backend was constructed with. May be None if it was not explicitly configured; providers that need it to build a fast-path request should return None in that case rather than guessing a default.	required

Returns:

Type	Description
int or None	The embedding dimension, or None if this provider has no fast path for it.

canonical_model_name(name) abstractmethod classmethod

Return the canonical form of a model name for this provider.

The canonical form is the identifier used as a stable key wherever a consumer persists model identity, and the model value :func:~omop_llm.backend.build_model_backend resolves to. Implementations must be idempotent: calling this on an already-canonical name returns the same string unchanged.

Parameters:

Name	Type	Description	Default
name	str	Raw model name as supplied by the caller, e.g. "llama3" or "text-embedding-3-small".	required

Returns:

Type	Description
str	The canonical model name for this provider.

Raises:

Type	Description
ValueError	If the name cannot be made canonical (e.g. an Ollama name with no explicit tag).

embedding_dimension_hint(model, *, api_base)

Synchronous counterpart to :meth:async_embedding_dimension_hint.

Parameters:

Name	Type	Description	Default
model	str	The canonical model name.	required
api_base	str	The resolved base URL this backend was constructed with. See :meth:async_embedding_dimension_hint.	required

Returns:

Type	Description
int or None	The embedding dimension, or None if this provider has no fast path for it.

registry

The closed set of providers omop_llm exposes.

any-llm itself supports around fifty providers; omop_llm intentionally supports six, matched to what this stack actually runs: local (ollama, llama-server via llamacpp, vllm) and cloud (openai, anthropic, gemini). See :mod:omop_llm.providers.supported for the six classes themselves. A provider not defined there is structurally unreachable through omop_llm's public API, regardless of what any-llm itself supports.

PROVIDER_REGISTRY is built by discovering :class:~omop_llm.providers.base.ProviderMixin's own subclasses, not by a second, separately-maintained list of classes: the set of supported providers is defined exactly once, in :mod:omop_llm.providers.supported, and this module can't drift out of sync with it because it has nothing of its own to drift. One caveat that comes with discovery over a class registry: any other direct subclass of ProviderMixin loaded into the process (e.g. a test fixture) would also appear here. Nothing in this package does that; if a test ever needs a fake provider, it should not subclass the mixin directly.

canonical_model_name(provider_key, name)

Canonicalize a model name for one registered provider.

Useful for deciding what to persist as a model's stable identity (e.g. in a database) independently of building a full :class:~omop_llm.backend.ModelBackend. :func:~omop_llm.backend.build_model_backend also calls this internally, so a backend's model attribute is always canonical without callers needing to remember to do it themselves.

Strips surrounding whitespace and rejects an empty result before dispatching to the provider's own canonical_model_name.

Parameters:

Name	Type	Description	Default
provider_key	str	A key expected to be in :data:PROVIDER_REGISTRY.	required
name	str	Raw model name to canonicalize.	required

Returns:

Type	Description
str	The canonical model name for this provider.

Raises:

Type	Description
UnsupportedProviderError	If provider_key is not registered.
ValueError	If name is empty or whitespace-only, or cannot otherwise be made canonical for this provider (e.g. an Ollama name with no explicit tag).

provider_capabilities_for(provider_key)

Build the provider-wide capability ceiling for one registered provider.

This is the provider's transport-level ceiling, not a specific model's effective capabilities.

streaming, embeddings, and extended_thinking come straight from any-llm's own get_provider_metadata(). tool_use and structured_output come from the class attributes each provider subclass declares itself, since any-llm tracks neither.

Parameters:

Name	Type	Description	Default
provider_key	str	A key expected to be in :data:PROVIDER_REGISTRY.	required

Returns:

Type	Description
Capabilities	The capability declaration for this provider.

Raises:

Type	Description
UnsupportedProviderError	If provider_key is not registered.

provider_class_for(provider_key)

Look up a registered provider class.

Parameters:

Name	Type	Description	Default
provider_key	str	A key expected to be in :data:PROVIDER_REGISTRY.	required

Returns:

Type	Description
type of AnyLLM	The provider class registered for provider_key.

Raises:

Type	Description
UnsupportedProviderError	If provider_key is not registered.

supported_providers()

List the provider keys omop_llm will construct a backend for.

Returns:

Type	Description
tuple of str	The registered provider keys, sorted alphabetically.

supported

The providers omop_llm supports, as explicit classes.

any-llm itself supports around fifty providers (see its own reference: https://docs.mozilla.ai/any-llm/providers/). omop_llm currently supports the following models: - local (ollama, llama-server via llamacpp, vllm), and - cloud (openai, anthropic, gemini).

Each class here subclasses both :class:~omop_llm.providers.base.ProviderMixin (our contract: TOOL_USE/STRUCTURED_OUTPUT, and the required canonical_model_name override) and any-llm's own provider class for that provider.

AnthropicProvider

Bases: ProviderMixin, AnthropicProvider

Anthropic (Claude).

base_url defaults to anthropic SDK default when not given. api_key is required (explicit, or the ANTHROPIC_API_KEY env var).

Notes

any-llm's get_provider_metadata() reports embedding=False for Anthropic (it has no embeddings API), so :meth:omop_llm.backend.ModelBackend.embed_texts refuses this provider.

canonical_model_name(name) classmethod

No transformation: Anthropic model names have no mutable-tag concern.

GeminiProvider

Bases: ProviderMixin, GeminiProvider

Gemini, e.g. gemini-2.5-pro.

base_url defaults to gemini SDK default when not given. api_key is required (explicit, or the GEMINI_API_KEY/GOOGLE_API_KEY env var).

canonical_model_name(name) classmethod

No transformation: Gemini model names have no mutable-tag concern.

LlamacppProvider

Bases: ProviderMixin, LlamacppProvider

llama.cpp's llama-server. Covers local dev and a CUDA/TRE fallback profile.

base_url defaults to http://127.0.0.1:8080/v1 if not given. api_key is not required.

canonical_model_name(name) classmethod

No transformation: llama-server model names have no mutable-tag concern.

OllamaProvider

Bases: ProviderMixin, OllamaProvider

Wrapped Ollama provider, for local dev and TRE fallback. Extends any-llm's own OllamaProvider with canonical model naming and embedding-dimension lookup via Ollama's native POST /api/show. Supports structured output natively using response_format.

base_url defaults to http://localhost:11434 if not given. api_key is not required.

async_embedding_dimension_hint(model, *, api_base) async

See :meth:omop_llm.providers.base.ProviderMixin.async_embedding_dimension_hint.

canonical_model_name(name) classmethod

Require an explicit, immutable Ollama model tag.

Rejects both untagged names and the mutable :latest tag: :latest can silently repoint after an ollama pull, breaking consistency between stored embeddings and new query embeddings.

Parameters:

Name	Type	Description	Default
name	str	Model name with an explicit tag, e.g. "llama3:8b" or "nomic-embed-text:v1.5".	required

Returns:

Type	Description
str	The input name, validated.

Raises:

Type	Description
ValueError	If the name has no tag, or if the tag is :latest.

embedding_dimension_hint(model, *, api_base)

See :meth:omop_llm.providers.base.ProviderMixin.embedding_dimension_hint.

OpenaiProvider

Bases: ProviderMixin, OpenaiProvider

OpenAI, e.g. gpt-4o.

Defaults to https://api.openai.com/v1 (any-llm's own explicit default) when base_url is not given, the real OpenAI API, same as leaving base_url unset in the openai SDK directly. Requires api_key (explicit, or the OPENAI_API_KEY environment variable); raises if neither is set.

canonical_model_name(name) classmethod

No transformation: OpenAI model names have no mutable-tag concern.

VllmProvider

Bases: ProviderMixin, VllmProvider

vLLM, the preferred TRE/NVIDIA backend.

base_url defaults to http://localhost:8000/v1 if not given. api_key is optional since self-hosted vLLM commonly runs without auth.

canonical_model_name(name) classmethod

No transformation: vLLM model names have no mutable-tag concern.

structured

Optional fallback for structured extraction: instructor's validate-and-retry loop.

The primary structured-extraction path lives on :meth:omop_llm.backend.ModelBackend.extract/:meth:~omop_llm.backend.ModelBackend.async_extract, built directly on any-llm's own response_format=<PydanticModel> passthrough. This module is a separate, explicitly scoped alternative for callers that specifically want resilience against a model returning almost-valid JSON, kept out of backend.py so importing omop_llm never requires the optional instructor dependency.

It is not wired in as a silent alternative for every provider. This was checked directly against instructor's own source (instructor.v2.auto_client._PROVIDER_BUILDERS):

• ollama is not safe to route through it: instructor's own Ollama builder constructs a plain openai.AsyncOpenAI(base_url=".../v1") client, the OpenAI-compat shim, not native /api/chat, and picks TOOLS-vs-JSON mode from a hardcoded model-name-substring list
• llamacpp/vllm have no dedicated builder in instructor at all. They are reachable only by routing through instructor's openai builder with an explicit base_url override, which is what :func:extract_with_retry/:func:async_extract_with_retry do.
• anthropic/gemini are not offered here either: this module only vouches for providers whose any-llm integration is already OpenAI-compat-native, so there is no native-transport distinction to lose.

async_extract_with_retry(provider, model, messages, response_model, *, base_url=None, api_key=None, max_retries=2, **kwargs) async

Extract via instructor's validate-and-retry loop.

Always builds instructor's openai client, a generic OpenAI-API-compatible constructor, to pass through all providers _INSTRUCTOR_SAFE_PROVIDERS allows.

Requires the instructor optional extra (pip install 'omop-llm[instructor]').

Parameters:

Name	Type	Description	Default
provider	str	One of {"openai", "llamacpp", "vllm"} (see module docstring). llamacpp/vllm are routed through instructor's openai builder with an explicit base_url, which is therefore required for those two, to avoid silently falling back to instructor's real-OpenAI default endpoint.	required
model	str	The model name or identifier.	required
messages	list of dict	Chat history in OpenAI message format.	required
response_model	type of BaseModel	The Pydantic model to constrain and validate the response against.	required
base_url	str	The provider's base URL. Required when provider is not "openai".	None
api_key	str	The API key for this provider, if one is required.	None
max_retries	int	Number of validate-and-retry attempts. Default is 2.	2
**kwargs	Any	Additional arguments forwarded to instructor's chat.completions.create.	{}

Returns:

Type	Description
BaseModel	A validated instance of response_model.

Raises:

Type	Description
UnsupportedCapabilityError	If provider is not in {"openai", "llamacpp", "vllm"}, or if the instructor optional extra is not installed.
ValueError	If provider is not "openai" and base_url is not given.

extract_with_retry(provider, model, messages, response_model, *, base_url=None, api_key=None, max_retries=2, **kwargs)

Extract via instructor's validate-and-retry loop, synchronously.

Always builds instructor's openai client, a generic OpenAI-API-compatible constructor, to pass through all providers _INSTRUCTOR_SAFE_PROVIDERS allows. See :func:async_extract_with_retry for parameters.