Skip to main content

Provider control features

These settings control model groups, payload logs, context limits, and zero-completion handling.

Before you start the gateway, replace each your-...-model placeholder with an active model ID for that provider account.

Model groups​

providers:
  model_groups:
    - name: fast
      aliases:
        - default-fast
      description: Low-latency chat targets
      targets:
        - openai-mini
        - openai-standard
      fallback_group: standard
    - name: standard
      targets:
        - openai-prod
        - openai-shadow
FieldTypeNotes
namestringThis field is necessary.
targetsstring[]This field is necessary and must reference known provider target IDs.
aliasesstring[]This optional field contains other names.
descriptionstringThis optional field contains a readable description.
fallback_groupstringThis optional field continues the active target chain. It must reference a known group and must not make a cycle.

When a request model resolves to a group, the runtime puts the group targets first. It then adds each configured fallback_group.

The usual provider attempt rules control movement through the chain.

Logging controls​

providers:
  logging:
    redact_message_bodies: true

The gateway applies redact_message_bodies to payload logs. The default value is false.

Context compression​

providers:
  targets:
    - id: openai-primary
      provider: openai
      model: your-openai-model
      max_context_tokens: 128000
      secret_key_ref:
        env: VERDICTAN_OPENAI_API_KEY

The active request path uses middle-out compression when a selected target sets max_context_tokens and the request exceeds that value.

Zero-completion insurance​

providers:
  zero_completion_insurance:
    enabled: true

For buffered Chat Completions, enabled: true moves a detected zero completion to the next eligible target. Do not use this control when replay to a different target is not approved.

Complete example​

pack:
  name: advanced-provider-controls
  version: 1.0.0
  enabled: true
providers:
  targets:
  - id: openai-prod
    provider: openai
    provider_type: openai
    format: openai
    model: your-openai-model
    max_context_tokens: 128000
    secret_key_ref:
      env: VERDICTAN_OPENAI_API_KEY
  - id: openai-secondary
    provider: openai
    provider_type: openai
    format: openai
    model: your-secondary-openai-model
    secret_key_ref:
      env: VERDICTAN_OPENAI_SECONDARY_API_KEY
  model_groups:
  - name: standard
    targets:
    - openai-prod
    - openai-secondary
  logging:
    redact_message_bodies: true
  zero_completion_insurance:
    enabled: true
policies:
  chain:
  - audit-logger
policy:
  audit-logger: {}
  • Validate all model group target IDs. An unknown target causes a parse failure.
  • Set target max_context_tokens when context compression must run.
  • Enable zero-completion handling only when a replay to the next target is approved.

Next steps​