Aegis Orchestrator
Deployment

Configuring LLM Providers

The BYOLLM model alias system — configuring OpenAI, Anthropic, and Ollama providers and mapping them to agent-facing aliases.

Configuring LLM Providers

AEGIS uses a BYOLLM (Bring Your Own LLM) system. Agent manifests reference model aliases (e.g., default, fast, smart), not provider names. Node configuration maps these aliases to concrete providers and models. This decouples agent code from infrastructure choices — you can swap providers or models without changing any agent manifest.


How the Alias System Works

Agent manifest:  spec.runtime.model = "fast"
         ↓
bootstrap.py:    POST /v1/dispatch-gateway  { "model": "fast" }
         ↓
Node config:     spec.llm_providers[].models[].alias = "fast"
                   → provider: ollama, model: qwen2.5-coder:7b
         ↓
LLM API call to http://ollama:11434

Credential resolution happens in the orchestrator. The agent process never receives API keys.

Standard Aliases

AEGIS defines seven conventional aliases. You can map any of these (or create your own) in your node config:

AliasIntentTypical Mapping
defaultGeneral-purpose, used when no model is specifiedGPT-4o, Claude Sonnet, phi3
fastLow latency, high throughputGPT-4o-mini, Claude Haiku, qwen2.5-coder:7b
smartMaximum capability for complex tasksGPT-4o, Claude Sonnet, deepseek-coder-v2
cheapMinimize API costGPT-4o-mini, Ollama local models
localAir-gapped / zero API costAny Ollama model
judgeThe built-in judge agents that score an iteration's outputA reasoning model
tool-judgeThe inner-loop judge of tool calls, tool-call-policy-judgeThe model judge names

Configuring Providers

All provider configuration lives under spec.llm_providers in aegis-config.yaml.

Ollama (Local / Air-Gapped)

Ollama runs a local inference server. No API key is required. This is ideal for air-gapped environments or cost-sensitive workloads.

spec:
  llm_providers:
    - name: "local"
      type: "ollama"
      endpoint: "http://ollama:11434"
      models:
        - alias: "default"
          model: "phi3:mini"
          capabilities: ["code", "reasoning"]
          context_window: 4096
        - alias: "fast"
          model: "qwen2.5-coder:7b"
          capabilities: ["code"]
          context_window: 4096
        - alias: "local"
          model: "phi3:mini"
          capabilities: ["code", "reasoning"]
          context_window: 4096

Ensure the Ollama server is running and the model is pulled before starting the AEGIS daemon:

ollama pull phi3:mini
ollama pull qwen2.5-coder:7b
ollama serve

OpenAI

spec:
  llm_providers:
    - name: "openai"
      type: "openai"
      endpoint: "https://api.openai.com/v1"
      api_key: "env:OPENAI_API_KEY"
      models:
        - alias: "default"
          model: "gpt-4o"
          capabilities: ["code", "reasoning"]
          context_window: 128000
        - alias: "fast"
          model: "gpt-4o-mini"
          capabilities: ["code"]
          context_window: 128000
        - alias: "smart"
          model: "gpt-4o"
          capabilities: ["code", "reasoning"]
          context_window: 128000
        - alias: "cheap"
          model: "gpt-4o-mini"
          capabilities: ["code"]
          context_window: 128000

A provider has no base_url or api_version field: endpoint is the base URL every request to the provider is sent to, and it is required for every provider type.

Anthropic

spec:
  llm_providers:
    - name: "anthropic"
      type: "anthropic"
      endpoint: "https://api.anthropic.com/v1"
      api_key: "env:ANTHROPIC_API_KEY"
      models:
        - alias: "default"
          model: "claude-sonnet-4-5"
          capabilities: ["code", "reasoning"]
          context_window: 200000
        - alias: "fast"
          model: "claude-3-5-haiku-20241022"
          capabilities: ["code"]
          context_window: 200000
        - alias: "smart"
          model: "claude-sonnet-4-5"
          capabilities: ["code", "reasoning"]
          context_window: 200000

OpenAI-Compatible Providers

Any provider that exposes an OpenAI-compatible API (e.g., vLLM, LM Studio, Together AI) can be used with type: "openai" and its own endpoint. type: "openai-compatible" uses the same adapter and counts as a local provider for the selection strategy:

    - name: "together"
      type: "openai"
      endpoint: "https://api.together.xyz/v1"
      api_key: "env:TOGETHER_API_KEY"
      models:
        - alias: "cheap"
          model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"
          capabilities: ["code"]
          context_window: 8192

An assistant message that carries only tool calls is sent with content: "", so servers that require content on every message accept the second request of a tool cycle. A tool name the model returns is mapped back to the exact name AEGIS sent, so tools such as fs.create_dir and fs.multi_edit round-trip unchanged.

Request Headers

A provider of type openai or openai-compatible can carry headers, a map of header name to value. The orchestrator sends each header on every request to that provider, chat and health check alike, beside Authorization. Use it for a value the provider's API needs on each call, such as the AI Gateway a Cloudflare Workers AI request is routed through:

spec:
  llm_providers:
    - name: "workers-ai"
      type: "openai-compatible"
      endpoint: "https://api.cloudflare.com/client/v4/accounts/<account-id>/ai/v1"
      api_key: "env:CLOUDFLARE_AI_API_TOKEN"
      headers:
        cf-aig-gateway-id: "<gateway-name>"
      enabled: true
      models:
        - alias: "default"
          model: "@cf/zai-org/glm-5.3-flash"
          capabilities: ["chat", "code", "reasoning"]
          context_window: 1048576

A header value is configuration, not a secret: keep keys in api_key. The node configuration is refused at validation when a provider configures a header named authorization in any letter case, because the key has its own field. A header name or value that is not valid HTTP refuses that provider's adapter with an error naming the provider and the header, never the value. The anthropic, gemini and ollama adapters do not send headers.


Selection Strategy

The spec.llm_selection block controls how the orchestrator resolves aliases when multiple providers define the same alias:

spec:
  llm_selection:
    strategy: "prefer-local"     # Prefer local providers, fall back to cloud
    default_provider: "local"    # Must name a provider in llm_providers; does not choose one
    max_retries: 3               # Retry count on transient failures
    retry_delay_ms: 1000         # Delay between retries
StrategyWhen two enabled providers map the same alias, the alias goes to
prefer-local (default)A local provider (ollama, openai-compatible) over a cloud one (openai, anthropic, gemini)
prefer-cloudA cloud provider over a local one
cost-optimizedThe model with the lower cost_per_1k_tokens
latency-optimizedAs prefer-local

Otherwise the provider listed first keeps the alias. Each alias is resolved to one provider and model when the node starts.

A model answer with no text and no tool call is not returned to the agent as its output: it counts as a failed attempt (empty generation: model '<model>' on alias '<alias>' returned no text and no tool call) and is retried within max_retries, then tried once on fallback_provider when one is set and the alias's model entry names no fallback_alias. When every attempt is spent, the error reaches the agent and fails the iteration.

A provider's time limit. When an openai or openai-compatible provider answers HTTP 408 with an error of code 3046, the answer Cloudflare Workers AI gives for a generation it ended at its own time limit, the identical request is not sent to that model again and the alias's remaining retries are skipped. The request goes once to the alias's fallback_alias when its model entry names one (Fallback Alias), otherwise once to fallback_provider when one is set. With neither, the call fails at once with provider timeout: alias '<alias>': model '<model>' was ended by the provider at its time limit after <n> s (HTTP 408, code 3046); the identical request is not sent again: …. When the fallback's own attempt fails, the error adds ; the fallback model '<model>' was tried once and failed: <error>. Every other 408 is retried within max_retries as before.

A provider's rate limit or unavailability. An answer of rate limited (429) is the provider's rate limit, and any 5xx other than its time limit (above) is its unavailability: an openai, openai-compatible, gemini, anthropic or ollama provider reads a 500, 502, 503 or 504 as unavailable, with the HTTP status and the provider's body in the error, among them Cloudflare Workers AI's 500 with code 4009. A model call an agent makes inside a try retries either answer past max_retries for as long as the next attempt can start before the try's time left, less the margin kept after a call for a command's result; outside a try, before the call's llm_timeout_seconds. Each attempt after such a wait runs in a fresh llm_timeout_seconds window of its own. The wait doubles from retry_delay_ms (at least 100 ms) up to 16 s, the last wait is shortened to leave one attempt, and a longer Retry-After from the provider is honoured; a Retry-After reaching past the try's end ends the call. When the provider's limit still ends the try, the agent's request is answered HTTP 429 with upstream_rate_limit: The model on alias '<alias>' was over its rate limit for <t> of retries; this try ended there and is not counted., or HTTP 503 with upstream_unavailable: The model on alias '<alias>' was unavailable for <t> of retries; this try ended there and is not counted., where <t> reads <s> s or <m> min <s> s. That try is not counted against the agent's tries; the wait before the next try and the execution's own bound stay.

Fallback Alias

A model entry can name another alias of the node as its fallback_alias:

      models:
        - alias: "smart"
          model: "<larger-model>"
          capabilities: ["chat", "code", "reasoning"]
          context_window: 131072
          fallback_alias: "fast"
        - alias: "fast"
          model: "<smaller-model>"
          capabilities: ["chat", "code"]
          context_window: 131072
          max_output_tokens: 16384
          temperature: 0.2

After a provider's time limit on the alias's own model, and only then, the request is sent once to the named alias's model, with the named alias's max_output_tokens and temperature where its entry sets them and the caller's otherwise. That attempt is not retried; it is bounded by llm_attempt_timeout_secs and, on a chat call, falls inside llm_overall_timeout_secs. Its answer is returned; an answer with no text and no tool call counts as its failure, and a failure is returned naming both models. Every other failure on the alias behaves as it does without the key. An alias that names a fallback_alias never reaches fallback_provider, after its time limit or after any other failure. The orchestrator logs Trying fallback alias: alias='<alias>', fallback_alias='<fallback>', model='<model>' when it sends one.

A fallback is one level. The node refuses to start when a fallback_alias:

  • names an alias no enabled provider maps, an empty name included: model alias '<alias>' names fallback_alias '<fallback>', which is not mapped by any enabled provider;
  • names the entry's own alias: model alias '<alias>' names itself as its fallback_alias;
  • names an alias whose own entry names a fallback_alias: model alias '<alias>' names fallback_alias '<fallback>', which itself names a fallback_alias: a fallback is one level only.

Using Aliases in Agent Manifests

Set spec.runtime.model in your agent manifest to select a model alias:

apiVersion: 100monkeys.ai/v1
kind: Agent
metadata:
  name: "my-agent"
  version: "1.0.0"
spec:
  runtime:
    language: "python"
    version: "3.11"
    model: "smart"        # ← Model alias
  task:
    instruction: "..."

If model is omitted, it defaults to "default".

Aliases the Built-In Agents Name

The built-in agent templates name four aliases, so a node that deploys them maps each one:

AliasBuilt-in agents
smartthe bash, python and javascript executors, the code validator, the output formatter, the agent creator, the workflow creator's validator and planner, hello-world, and the Forge's seven agents (requirements-analyst, architect-agent, tester-agent, coder-agent, code-reviewer-agent, critic-agent, security-auditor-agent)
judgeagent-generator-judge, code-quality-judge, workflow-generator-judge, goal-judge
tool-judgetool-call-policy-judge, the inner-loop tool judge that hello-world, the agent creator and the workflow creator's validator name in tool_validation
defaultskill-validator

An alias the node does not map fails every call made on it with Model alias '<alias>' not found. The configuration aegis init writes maps default, smart, judge and tool-judge on the provider you choose, and smart, judge and tool-judge on each additional provider it adds; tool-judge is always on the same model as judge.

Any caller holding a JWT or an aegis_* API key can read which model an alias resolves to with GET /v1/llm/aliases/{alias}.


Image Generation Models

An alias whose capabilities carry image_generation is an image generation model. When the provider registry is built, it is bound to the provider's image generation endpoint, never to a language model; only the openai and openai-compatible types have one, and an alias of another type is not built, and the log's warning ends "provider type '<type>' has no image generation adapter". The tool aegis.image.generate names one by its alias, and takes the first such alias in the configuration when it names none; a node with none answers it "this node has no image generation model".

models:
  - alias: image
    model: gpt-image-1
    capabilities: [image_generation]
    context_window: 0

On Cloudflare Workers AI. An openai-compatible image generation alias whose endpoint is a Workers AI /ai/v1 base (https://api.cloudflare.com/client/v4/accounts/<account-id>/ai/v1) runs its model at <account>/ai/run/<model> and reads the image Workers AI answers in result.image. The request is a JSON body {prompt, width?, height?} for every model but @cf/black-forest-labs/flux-2-dev, which takes a multipart form; @cf/black-forest-labs/flux-1-schnell takes no size, so a size asked of it is accepted and not sent, and the image comes at the model's own size. The provider's headers are sent but for the AI Gateway's cf-aig- headers. A capacity refusal (HTTP 429) answers that the model is unavailable, as any other provider failure does.

spec:
  llm_providers:
    - name: "workers-ai"
      type: "openai-compatible"
      endpoint: "https://api.cloudflare.com/client/v4/accounts/<account-id>/ai/v1"
      api_key: "env:CLOUDFLARE_AI_API_TOKEN"
      enabled: true
      models:
        - alias: "image"
          model: "@cf/black-forest-labs/flux-1-schnell"
          capabilities: ["image_generation"]
          context_window: 0

An image generation alias is never an agent's model. An agent whose spec.runtime.model names one is refused by aegis.agent.create, aegis.agent.update, POST /v1/agents and PATCH /v1/agents/{id} with "'<alias>' generates images; an agent's model must be a language model".

Hot-Swapping Models

To change which model backs a given alias:

  1. Update aegis-config.yaml:

    spec:
      llm_providers:
        - name: "openai"
          type: "openai"
          endpoint: "https://api.openai.com/v1"
          api_key: "env:OPENAI_API_KEY"
          models:
            - alias: "default"
              model: "gpt-4o"          # Change this to swap models
              capabilities: ["code", "reasoning"]
              context_window: 128000
  2. Restart the AEGIS daemon:

    aegis daemon stop
    aegis daemon start --config aegis-config.yaml

Running executions continue using the provider that was active when they started. New executions use the updated alias mapping.


Provider Configuration Reference

Provider Fields

FieldTypeRequiredDescription
namestringYesInternal identifier for the provider.
typeopenai | anthropic | gemini | ollama | openai-compatibleYesProvider adapter to use. openai and openai-compatible share the OpenAI adapter; openai-compatible counts as local for the selection strategy.
api_keystringExcept OllamaAPI key value or env:VAR_NAME reference. Never hardcode keys.
endpointstringYesThe API base URL requests are sent to. An empty string selects the type's default URL, such as https://api.openai.com/v1 for openai and http://localhost:11434 for ollama.
headersmap of string to stringNoHeaders sent on every request beside Authorization, by the openai and openai-compatible adapters. A header named authorization is refused. See Request Headers.

Model Entry Fields

FieldTypeRequiredDescription
aliasstringYesAgent-facing name (e.g., default, fast, smart).
modelstringYesModel identifier as accepted by the provider's API.
capabilitiesstring[]YesTags like code, reasoning, vision. image_generation makes the alias an image generation model.
context_windowintegerYesProvider's context window size in tokens.
max_output_tokensintegerNoMaximum output tokens to request from the provider. Default 8192. Important for thinking-mode models (e.g., Gemini 2.5 Pro) that use output budget for internal reasoning.
temperaturenumberNoDefault LLM sampling temperature for this alias. 0.0 = deterministic, 1.0 = creative. Default 0.4. Agents can override per-execution via spec.runtime.temperature in their manifest.
fallback_aliasstringNoAnother alias of this node, tried once after this alias's model is ended at the provider's time limit, with that alias's own max_output_tokens and temperature. See Fallback Alias.

Credential Security

API keys configured as env:VAR_NAME are read from the daemon's environment at startup. They are stored in memory and never written to disk or logs. When OpenBao secrets management is configured (see Secrets Management), credentials can also be sourced from OpenBao with the secret:path/to/secret syntax.

Agents never receive API keys. All LLM API calls are made by the orchestrator host process using the resolved credential.

On this page