Skip to main content

verdictan gateway run

Start the Verdictan gateway.

Usage​

verdictan gateway run [OPTIONS]

Key options​

FlagDescription
--listen <host:port>Listen address. Default: 0.0.0.0:41002
--agent <selector>With --policy-config, an agent name to create or reuse. Without local config, use a configured agent name or ID.
--policy-config <path>One or more declarative config files. You must also use --agent.
--config-name <name>Optional configuration name when syncing a local pack
--fail-mode <allow|block>Behavior when each upstream attempt ends in a transport error. Default: block
--upstream <url>Debug-only upstream override
VERDICTAN_UPSTREAM_API_KEYDebug-only upstream credential override supplied through the environment
--max-concurrency <n>Runtime concurrency ceiling
VERDICTAN_API_TOKENRuntime token for control-plane sync, telemetry, and inspection

Important rule​

When you pass --policy-config, you must supply these two items:

  1. --agent <name>
  2. a runtime API token in VERDICTAN_API_TOKEN

This combination lets the CLI sync the YAML to a control-plane agent identity before the gateway starts.

Example: connected runtime from local YAML​

export VERDICTAN_API_URL="https://api.verdictan.com"
export VERDICTAN_API_TOKEN="vdt_your_gateway_runtime_token"
export VERDICTAN_OPENAI_API_KEY="provider-secret"

verdictan gateway run \
  --agent docs-demo \
  --listen 127.0.0.1:41002 \
  --policy-config policy-config.yaml

Advanced fail-open debugging​

verdictan gateway run \
  --agent docs-demo \
  --listen 127.0.0.1:41002 \
  --policy-config policy-config.yaml \
  --fail-mode allow
warning

Use --fail-mode allow only when a synthetic success response is an approved product requirement. It can make an upstream failure appear to succeed.

Fail-open does not send an unreachable request to a different unconfigured service. When each upstream attempt ends in a transport error, it returns an HTTP 200 synthetic response with cause proxy.degraded_allow. The default block mode returns 503. It returns 504 for a timeout.

The CLI default listens on all interfaces. Local examples bind to 127.0.0.1. If intentional inbound traffic, firewall, TLS, and application-token controls are available, you can use an all-interface address.

Multi-provider example​

policy-config.yaml
pack:
  name: multi-provider
  version: 0.1.0
  enabled: true

providers:
  targets:
    - id: openai-primary
      provider: openai
      model: your-openai-model
      base_url: https://api.openai.com
      secret_key_ref:
        env: VERDICTAN_OPENAI_API_KEY
    - id: azure-fallback
      provider: azure
      provider_type: azure-openai
      format: openai
      model: your-azure-openai-model
      base_url: https://replace-with-resource-name.openai.azure.com
      secret_key_ref:
        env: VERDICTAN_AZURE_OPENAI_API_KEY
      azure_api_version: your-supported-api-version
      azure_deployment: your-azure-deployment

  routing:
    strategy: ordered

policies:
  chain:
    - prompt-injection
    - pii-detector

Replace the two model placeholders with IDs supported by those provider accounts.

With ordered routing, a transport error can select the next eligible target. The same behavior applies to HTTP 408, 429, 500, 502, 503, and 504. The gateway returns other upstream errors without a subsequent target attempt. To use only the first eligible target, configure one target. Alternatively, set providers.routing.allow_fallbacks: false.

Fallback can repeat a billable or side-effecting request. Use an approved provider idempotency control when duplicate dispatch can cause a problem. For a streaming request, fallback can occur only before the gateway sends chunks to the client.

See Providers Configuration for routing and circuit-breaker fields.

Run the gateway in a container​

Use the published image when you do not install the host binary. Mount the policy file read-only. Pass credentials through the environment:

docker pull verdictan/verdictan:latest

export VERDICTAN_API_TOKEN="vdt_your_gateway_runtime_token"
export VERDICTAN_OPENAI_API_KEY="provider-secret"

docker run --rm -p 127.0.0.1:41002:41002 \
  -e VERDICTAN_API_URL="https://api.verdictan.com" \
  -e VERDICTAN_API_TOKEN \
  -e VERDICTAN_OPENAI_API_KEY \
  -v "$PWD/policy-config.yaml:/config/policy-config.yaml:ro" \
  verdictan/verdictan:latest gateway run \
    --listen 0.0.0.0:41002 \
    --agent docs-demo \
    --policy-config /config/policy-config.yaml

The container listens on all container interfaces. The example publishes it only on the host loopback address. Do not publish the port on an external interface without the necessary firewall, TLS, and application-token controls.

Send traffic through the gateway​

Use a client API token that is different from the runtime token:

export VERDICTAN_REQUEST_TOKEN="vdt_your_application_token"

curl http://localhost:41002/v1/chat/completions \
  -H "Authorization: Bearer ${VERDICTAN_REQUEST_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-openai-model",
    "messages": [
      {"role": "user", "content": "Explain quantum computing"}
    ]
  }'

Pin a provider target​

curl http://localhost:41002/v1/chat/completions \
  -H "Authorization: Bearer ${VERDICTAN_REQUEST_TOKEN}" \
  -H "Content-Type: application/json" \
  -H "X-Verdictan-Provider: openai-primary" \
  -H "X-Verdictan-Model: your-openai-model" \
  -d '{"model":"your-openai-model","messages":[{"role":"user","content":"Hello"}]}'

The gateway rejects an unknown provider pin. It does not automatically select a different target. Model and provider pins must be compatible with the declared target.

Verify process health independently of a model request:

curl http://127.0.0.1:41002/healthz

Next steps​