Runtime configuration

Runtime sections control gateway history, agent routes, content moderation, automatic provider selection, model lists, and response caches.

Use the request type reference

Use runtime request types for responses, messages, and chat_completions.

The reference also includes structured outputs, prompt caches, session_id, plugin controls, embeddings, transcription, speech, silent engine, and optional shadow routes.

History

The public schema uses history.capture: for history capture.

history:
  capture:
    enabled: true
    mode: raw
    include_blocked: false

Field	Type	Necessary	Default	Description
capture.enabled	boolean	no	false in a specified block	This field enables history writes.
capture.mode	string	no	metadata_only	raw, redacted, metadata_only, or disabled
capture.include_blocked	boolean	no	false	This field includes blocked requests in History.
capture.fail_open	boolean	no	true	This field changes history failure logs between warning and error. Requests stay fire-and-forget in each mode.

When the history section is missing, a connected control plane runtime automatically enables metadata-only history.

When a specified history.capture block has no enabled field, the block uses the default value of false.

Metadata-only mode

This mode gives audit evidence without the retention of complete prompts or responses.

history:
  capture:
    enabled: true
    mode: metadata_only
    include_blocked: true

Disabling history

history:
  capture:
    enabled: false

Agents

The agents.runtime: section sets the default agent identity for the gateway.

agents:
  runtime:
    default_agent_id: agent-123

Field	Type	Necessary	Default	Description
runtime.default_agent_id	string	no	-	The gateway adds this default agent ID to events and traces.

Moderation

The moderation: section configures the external moderation backend for /v1/moderations.

moderation:
  provider: openai
  secret_key_ref:
    env: VERDICTAN_OPENAI_API_KEY
  categories:
    - violence
    - hate
    - self-harm
  threshold: 0.7

Field	Type	Necessary	Default	Description
provider	string	yes	-	openai or azure
secret_key_ref	object	no (schema)	-	This environment credential is necessary for OpenAI and Azure calls.
endpoint	string	no (schema)	-	This field is necessary for Azure. It is an optional OpenAI endpoint override.
categories	string[]	no	all	This field limits moderation to selected categories.
threshold	number	no	0.5	This is the confidence limit for a flagged result.

Azure moderation

moderation:
  provider: azure
  secret_key_ref:
    env: VERDICTAN_AZURE_CONTENT_SAFETY_KEY
  endpoint: https://my-resource.cognitiveservices.azure.com
  categories:
    - violence
    - hate
  threshold: 0.5

:::warning Fail-open external path The public configuration has no fail_closed field for this top-level moderation handler.

If credentials or the Azure endpoint are missing, the handler returns an unflagged allow-style result with HTTP 200.

Network failures, unsuccessful responses, and malformed upstream responses cause the same result. The credential and Azure endpoint are necessary configuration items.

Do not use this endpoint as a fail-closed safety boundary. :::

Auto provider

The auto: section gives a virtual provider that selects configured targets by cost and latency.

auto:
  enabled: true
  name: auto
  routing:
    cost_weight: 0.6
    latency_weight: 0.4
    max_price_per_1m_tokens: 10.0

Field	Type	Necessary	Default	Description
enabled	boolean	no	true	This field turns the virtual auto provider on or off.
name	string	no	auto	Clients use this model name to select automatic routes.
routing.cost_weight	number	no	0.5	This is the price weight in the route score.
routing.latency_weight	number	no	0.5	This is the latency weight in the route score.
routing.max_price_per_1m_tokens	number	no	-	This field removes targets whose configured price is above the limit.

Set enabled explicitly. The runtime default is true, while the active schema material shows a different default.

If the price limit removes each target, routing returns no_eligible_provider. It does not restore the excluded targets.

The accepted routing fields are cost_weight, latency_weight, and max_price_per_1m_tokens. Automatic selection pins the request to one target. It does not schedule an alternate-provider fallback after a provider failure.

Models endpoint

The models: section controls the public /v1/models list endpoint.

models:
  disabled: false
  include_disabled: false

Field	Type	Necessary	Default	Description
disabled	boolean	no	false	When this field is true, /v1/models returns 404.
include_disabled	boolean	no	false	This field includes disabled nested models in the published list.

Provider credentials for the runtime

Connected gateways use a Verdictan runtime token for control plane access. They use secret_key_ref for provider credentials.

Keep provider secrets out of YAML. Reference them through secret_key_ref.

pack:
  name: config-runtime-providers-14
  version: 1.0.0
  enabled: true
providers:
  targets:
    - id: openai-chatgpt-shared
      provider: openai
      model: your-openai-model
      base_url: https://api.openai.com
      secret_key_ref:
        store: OPENAI_API_KEY
policies:
  chain:
    - audit-logger
policy:
  audit-logger: {}

Use secret_key_ref.env for local shells. Use secret_key_ref.store for a credential stored in Verdictan.

Cache

The cache: section enables specified or semantic response caches. Active cache lookup and replay support only non-streaming Chat Completions and Responses requests.

These requests must be read-only. Streaming traffic and other request types do not use this cache path.

Environment variables set the cache backend and global TTL.

Exact cache

cache:
  mode: exact
  default_on: true

The exact cache hashes the complete request body. Equal requests return the cached response.

Semantic cache

cache:
  mode: semantic
  similarity_threshold: 0.85
  embedding_provider: openai-prod
  default_on: true

The semantic cache embeds the request and finds prior requests above the configured similarity limit.

This operation occurs only when embedding_provider resolves and embeds the request.

If the provider is missing or unresolved, the cache skips semantic lookup and storage. An embedding call failure has the same result.

The ordinary specified-key cache path stays available.

Field	Type	Necessary	Default	Description
mode	string	yes	exact	exact or semantic
similarity_threshold	number	no	0.85	This is the minimum cosine similarity for a semantic cache hit.
embedding_provider	string	no (schema), yes for semantic behavior	-	The cache uses this provider target ID to calculate embeddings.
default_on	boolean	no	true	This field enables the cache by default for matching requests.

Cache backend

Select the backend with VERDICTAN_LLM_CACHE_BACKEND. The active auto setting selects the node-local filesystem backend.

Backend	Necessary environment	Optional environment
memory	—	—
filesystem or auto	—	—
redis or valkey	VERDICTAN_LLM_CACHE_REDIS_URL	VERDICTAN_LLM_CACHE_REDIS_KEY_PREFIX
s3	VERDICTAN_LLM_CACHE_S3_BUCKET, VERDICTAN_LLM_CACHE_S3_REGION, VERDICTAN_LLM_CACHE_S3_ACCESS_KEY_ID, VERDICTAN_LLM_CACHE_S3_SECRET_ACCESS_KEY	VERDICTAN_LLM_CACHE_S3_ENDPOINT, VERDICTAN_LLM_CACHE_S3_PREFIX, VERDICTAN_LLM_CACHE_S3_FORCE_PATH_STYLE
gcs	VERDICTAN_LLM_CACHE_GCS_BUCKET, VERDICTAN_LLM_CACHE_GCS_ACCESS_KEY_ID, VERDICTAN_LLM_CACHE_GCS_SECRET_ACCESS_KEY	VERDICTAN_LLM_CACHE_GCS_REGION, VERDICTAN_LLM_CACHE_GCS_ENDPOINT, VERDICTAN_LLM_CACHE_GCS_PREFIX, VERDICTAN_LLM_CACHE_GCS_FORCE_PATH_STYLE
qdrant	VERDICTAN_LLM_CACHE_QDRANT_URL	VERDICTAN_LLM_CACHE_QDRANT_COLLECTION, VERDICTAN_LLM_CACHE_QDRANT_API_KEY

Semantic lookup and storage support memory, filesystem, redis, valkey, and qdrant.

The s3 and gcs backends support only specified caches. Do not use them with cache.mode: semantic.

The GCS backend uses interoperability access and secret keys. Redis, Valkey, S3, GCS, and Qdrant must use the CLI distributed feature.

If this feature or a necessary value is missing, cache startup fails.

VERDICTAN_LLM_CACHE_TTL_SECS=<positive-seconds> overrides the global cache TTL for each backend.

export VERDICTAN_LLM_CACHE_BACKEND=redis
export VERDICTAN_LLM_CACHE_TTL_SECS=86400
export VERDICTAN_LLM_CACHE_REDIS_URL=redis://localhost:6379/1

Complete runtime example

Replace the provider model placeholders with IDs that are active for those accounts.

pack:
  name: full-runtime
  version: 1.0.0
  enabled: true
providers:
  targets:
    - id: openai-prod
      provider: openai
      model: your-openai-model
      secret_key_ref:
        env: VERDICTAN_OPENAI_API_KEY
    - id: azure-prod
      provider: azure
      provider_type: azure-openai
      format: openai
      model: your-azure-openai-model
      base_url: https://replace-with-resource-name.openai.azure.com
      secret_key_ref:
        env: VERDICTAN_AZURE_OPENAI_API_KEY
      azure_api_version: your-supported-api-version
      azure_deployment: your-azure-deployment
history:
  capture:
    enabled: true
    mode: raw
    include_blocked: true
agents:
  runtime:
    default_agent_id: your-agent-id
moderation:
  provider: openai
  secret_key_ref:
    env: VERDICTAN_OPENAI_API_KEY
  categories:
    - violence
    - hate
    - self-harm
  threshold: 0.7
auto:
  enabled: true
  routing:
    cost_weight: 0.6
    latency_weight: 0.4
    max_price_per_1m_tokens: 10.0
models:
  disabled: false
  include_disabled: false
cache:
  mode: semantic
  similarity_threshold: 0.85
  embedding_provider: openai-prod
  default_on: true
policies:
  chain:
    - prompt-injection
    - pii-detector
    - quality-scorer
    - audit-logger
policy:
  prompt-injection:
    response:
      action: block
  pii-detector:
    action: redact
  quality-scorer:
    assertions:
      - type: word-count
        config:
          min: 20
    thresholds:
      min_aggregate: 0.7
  audit-logger: {}

In the active gateway, audit-logger is an allow-only marker. It also does not control retention.

Next steps

• Provider configuration
• Configuration tests
• Declarative configuration reference
• Quality scorer