LLM Configuration
Configure LLM provider API keys, model selection, Azure OpenAI, and custom endpoints for Emend.
Emend uses LiteLLM for multi-provider LLM support. You must configure an API key for at least one provider.
API Keys
Method 1: Environment Variables (Recommended)
Set provider-specific environment variables in your .env file. LiteLLM picks them up automatically.
# .env — set one or more provider keysOPENAI_API_KEY=sk-...ANTHROPIC_API_KEY=sk-ant-...GEMINI_API_KEY=...DEEPSEEK_API_KEY=...OpenAI is a common starting point for text generation. Embeddings default to Emend's local model unless you override embedding_model_name.
Method 2: Programmatic Configuration
Set API keys via the Config object. Programmatic keys take precedence over environment variables.
from emend.models.config_schema import APIKeyConfig, OpenAIConfig config = client.get_config()config.api_key_config = APIKeyConfig( openai=OpenAIConfig(api_key="sk-your-key-here"))client.set_config(config)Supported Providers
| Prop | Type | Description |
|---|---|---|
OpenAI | OpenAIConfig | OPENAI_API_KEY |
Anthropic | AnthropicConfig | ANTHROPIC_API_KEY |
Google Gemini | GeminiConfig | GEMINI_API_KEY |
DeepSeek | DeepSeekConfig | DEEPSEEK_API_KEY |
OpenRouter | OpenRouterConfig | OPENROUTER_API_KEY |
MiniMax | MiniMaxConfig | MINIMAX_API_KEY |
DashScope (Qwen) | DashScopeConfig | DASHSCOPE_API_KEY |
Zhipu AI | ZAIConfig | ZAI_API_KEY |
Moonshot | MoonshotConfig | MOONSHOT_API_KEY |
xAI | XAIConfig | XAI_API_KEY |
zai/glm-5.2, Emend uses https://api.z.ai/api/coding/paas/v4 as the built-in API base. Configure only the ZAI_API_KEY; the coding endpoint is a runtime default, not a setting. An explicit per-call API base or a configured custom endpoint still takes precedence.Custom OpenAI-Compatible Endpoints
Use CustomEndpointConfig to connect to any OpenAI-compatible API. Custom endpoints take priority over other providers for text generation (but not embeddings).
from emend.models.config_schema import APIKeyConfig, CustomEndpointConfig config.api_key_config = APIKeyConfig( custom_endpoint=CustomEndpointConfig( model="my-model", api_key="your-key", api_base="http://localhost:8000/v1" ))client.set_config(config)Azure OpenAI
Use AzureOpenAIConfig nested inside OpenAIConfig to connect to Azure OpenAI:
from emend.models.config_schema import ( APIKeyConfig, OpenAIConfig, AzureOpenAIConfig) config.api_key_config = APIKeyConfig( openai=OpenAIConfig( azure_config=AzureOpenAIConfig( api_key="your-azure-key", endpoint="https://your-resource.openai.azure.com/", api_version="2024-02-15-preview", deployment_name="gpt-4o" ) ))client.set_config(config)Model Selection
Emend uses different models for different tasks. Sensible defaults are provided, but you can override them via LLMConfig. Only set fields you want to override — None fields keep the defaults.
| Prop | Type | Description |
|---|---|---|
should_run_model_name | str | Default: minimax/MiniMax-M2.5. Fast check to decide if extraction should run on a given interaction. |
generation_model_name | str | Default: minimax/MiniMax-M2.5. Profile extraction, playbook generation, and evaluation. |
embedding_model_name | str | Default: OSS: local/minilm-l6-v2; Enterprise: custom. Vector embeddings for semantic search. Enterprise custom uses the operator-configured embedding service without exposing its concrete model. |
pre_retrieval_model_name | str | Default: minimax/MiniMax-M2.5. Model for pre-retrieval query reformulation. |
Embedding input formatting plus the default retrieval and playbook-clustering thresholds are selected by the exact embedding model:
| Embedding model | Input formatting | Retrieval threshold | Clustering similarity |
|---|---|---|---|
| local/minilm-l6-v2 | No prefix | 0.30 | 0.30 |
| local/nomic-embed-text-v1.5 | search_document: for stored text; search_query: for queries | 0.70 | 0.85 |
| local/nomic-embed-v1.5 (compatibility alias) | search_document: for stored text; search_query: for queries | 0.70 | 0.85 |
| Enterprise custom service | Resolved from the service policy | Model-specific | Model-specific |
| Other explicit models | No prefix | 0.45 | 0.30 |
0.0. In hybrid search, the threshold filters only the vector arm; full-text matches remain eligible. An explicit playbook clustering_similarity also wins. Playbook embeddings use only their normalized trigger; they do not fall back to playbook content. This normalization applies when an embedding is computed and does not automatically backfill existing playbook vectors.from emend.models.config_schema import LLMConfig config = client.get_config()config.llm_config = LLMConfig( generation_model_name="openai/gpt-4o", embedding_model_name="text-embedding-3-small",)client.set_config(config)Model names use LiteLLM's "provider/model-name" format (e.g., "openai/gpt-4o", "anthropic/claude-3-5-sonnet", "deepseek/deepseek-chat").
Z.ai models use Emend's prompt-backed structured-output path, and the live-verified zai/glm-5.2 model supports extraction-agent tool loops. Fallback lists may freely mix providers and structured-output strategies — Emend walks the primary and fallback models itself, one at a time, and rebuilds the request (structured-output strategy, api_base, per-model timeout) for each one. That means a primary using native JSON Schema can fall back to a prompt-backed provider like Z.ai, and vice versa; no transport compatibility restriction applies to structured-output or text fallback lists.
text-embedding-3-small and gemini/gemini-embedding-001 bypass the embedding service when the corresponding provider credentials are configured and the model produces 512-dimensional vectors — regeneration validates the vector width and fails the job if a model returns a different dimension.The input-formatting policy is also part of the embedding space. If an existing non-Nomic database was populated by an Emend version that added Nomic-style prefixes to every model, rebuild the local SQLite database or run enterprise embedding regeneration after upgrading.