Skip to main content

Connected Gateway Relay

Connected gateways can run on machines behind NAT, firewalls, VPNs, or corporate networks. The built-in relay tunnel lets a registered gateway receive requests for its Verdictan publication without opening an inbound port.

How the relay works​

When a connected gateway has three necessary values, it derives a relay hostname from VERDICTAN_API_URL. These values are an API URL, API token, and runtime registration ID. The gateway then opens an outbound WebSocket connection and keeps it open:

VERDICTAN_API_URL=https://api.eu.verdictan.com
                          ↓ automatic derivation
Gateway ──WSS──▶ wss://relay.eu.verdictan.com/v1/gateway/relay

The relay hostname uses the selected region. The gateway automatically replaces the api. prefix in the API URL with relay.. For example:

API URLDerived relay URL
https://api.eu.verdictan.comwss://relay.eu.verdictan.com/v1/gateway/relay
https://api.us.verdictan.comwss://relay.us.verdictan.com/v1/gateway/relay

You do not have to configure the relay URL. The gateway derives it automatically from the API URL.

  1. The gateway authenticates using its machine token (VERDICTAN_API_TOKEN).
  2. It sends a registration message containing its runtime_registration_id.
  3. The control plane acknowledges the gateway and sends inbound API requests through the WebSocket.
  4. The relay client sends each request to the local gateway HTTP server.
  5. The relay client sends the response back through the WebSocket.
  6. The same channel transfers streams, including SSE for LLM completions.

If an active connection drops, the next retry starts after one second. If a connection fails many times before registration, it uses exponential backoff. The maximum backoff is 30 seconds.

Configuration​

There is no customer relay-enable flag. The relay task starts only when connected mode resolves all three necessary values below:

VariablePurpose
VERDICTAN_API_URLControl-plane URL. The gateway automatically derives the relay hostname from it.
VERDICTAN_API_TOKENMachine token used to authenticate the WebSocket
VERDICTAN_RUNTIME_REGISTRATION_IDOptional registration ID. A connected gateway can register or restore one during startup.

The local gateway HTTP server uses port 41002 by default. The relay client sends requests to http://127.0.0.1:<gateway-port>. Each request has a 30-second timeout.

Monitoring​

Console​

The gateway detail surface shows control-plane heartbeat and telemetry freshness. It does not expose the relay client's WebSocket state in its own status field.

CLI​

verdictan gateway status --name production

This command reports local supervisor or service-manager state. Use the console gateway detail page for control-plane heartbeat freshness and published-hostname status.

Relay heartbeats​

The relay client sends a WebSocket ping each 20 seconds. It sends responses to server pings and reconnects after 45 seconds without a WebSocket message. The runtime telemetry reports control-plane gateway freshness independently.

Troubleshooting​

"relay connection failed"​

  • Verify VERDICTAN_API_URL is set and the host is reachable.
  • Make sure that the firewall allows outbound HTTPS and WSS traffic on port 443.
  • Verify DNS resolution for the API host and the derived relay host, such as relay.eu.verdictan.com.

"authentication failed"​

  • The runtime token can be expired or revoked. It can also have no gateway permissions necessary for the relay endpoint. Replace it through the governed token workflow for that gateway.
  • Verify VERDICTAN_API_TOKEN matches the token assigned to this gateway.

High latency through the relay​

Before you assign the latency to the relay, measure request timing from the API consumer, gateway, and upstream provider. Run the gateway near its upstream provider. Examine gateway metrics and request traces for the slow segment.

Frequent gateway reconnections​

  • Network connection failures cause backoff cycles. Examine the host network.
  • If the backoff becomes 30 seconds, the control plane can be temporarily unavailable. The gateway starts operating automatically when the connection becomes available.

Next steps​