Skip to main content

Gateway Response Cache

Use verdictan cache to examine the response cache that the active VERDICTAN_LLM_CACHE_* environment selects. The command opens the configured backend directly. It does not query a running gateway.

Command summary​

CommandResult
verdictan cache statsShow the selected backend and cache enablement. The filesystem backend also reports entries, size, hit/miss counts, evictions, and warm-up state.
verdictan cache list --top 20List the most recently accessed filesystem entries.
verdictan cache list --top 20 --by-sizeList the largest filesystem entries.
verdictan cache inspect <key>Read metadata for one exact cache key.
verdictan cache clearShows an interactive confirmation before it clears the selected backend.
verdictan cache clear --yesClear without the prompt. Use this command only in an intentional maintenance workflow.

list is filesystem-specific. stats prints a backend summary for each backend, but detailed entry statistics are available only for the filesystem backend. An empty list on a different backend is not evidence that the remote cache has no entries.

Examine the active cache​

Run commands with the same environment used by the gateway:

export VERDICTAN_LLM_CACHE_BACKEND=filesystem
export VERDICTAN_LLM_CACHE_DIR="$HOME/.verdictan/cache"

verdictan cache stats
verdictan cache list --top 10 --by-size

If you do not set VERDICTAN_LLM_CACHE_MAX_BYTES, the filesystem backend uses a 500 MB maximum. The command output is the source of truth for the environment you invoked.

To examine a complete cache key:

verdictan cache inspect '<exact-cache-key>'

The inspection output contains the status, body size, storage time, key version, header count, and content type. It does not show the cached response body. list shows only part of a long key. Do not use that row as input for inspect. Get the complete key from the approved cache diagnostic source.

Clear cache entries safely​

When you clear entries, the operation uses the backend that the command environment selects:

verdictan cache stats
verdictan cache clear
verdictan cache stats

For remote backends, a Cache cleared. message and zero exit status show that the CLI sent the clear operation. They do not prove that each remote delete succeeded. The Redis, Valkey, object-store, and Qdrant adapters can log individual failures without returning them to the command. Use approved backend telemetry and a representative cache miss to verify the result.

Before you clear a shared Redis, Valkey, object-store, or Qdrant backend, make sure that the configured environment is correct. Examine the URL, bucket, prefix, collection, and credentials. --yes removes the interactive confirmation. It does not add a dry run or limit the operation to one gateway.

For a controlled logical rollover without shared-entry deletion, use a cache buster. For each rollover, replace release-current with a new deployment value:

export VERDICTAN_LLM_CACHE_BUSTER="release-current"

The gateway includes the buster in its cache-key behavior. A new value prevents previous entries from matching. This result does not mean that the gateway deleted the previous backend objects.

Configuration checklist​

The cache runtime supports memory, filesystem, Redis, Valkey, S3, GCS, and Qdrant backends. Configure the selected backend through the matching VERDICTAN_LLM_CACHE_* variables. Then complete these steps:

  1. Run verdictan cache stats in the gateway environment.
  2. Make sure that the gateway starts with that same environment.
  3. When your policy allows caching, send a representative request. Then send the same request again.
  4. Examine gateway response metadata and cache metrics rather than inferring a hit from latency alone.
  5. Re-run verdictan cache stats for filesystem counters.

Unless your retention, encryption, and access policies allow it, do not cache sensitive or controlled content.

Troubleshooting​

SymptomCheck
stats shows an unexpected backendCompare the shell environment with the gateway service environment.
Filesystem list is emptyMake sure that the directory, cache enablement, request eligibility, TTL, and cache buster are correct.
Remote backend has no detailed statisticsThis CLI view shows detailed entry statistics only for the filesystem backend. Use the approved backend telemetry.
Remote clear prints success but does not delete entriesExamine CLI and backend logs. Make sure that connectivity and credentials are correct. Verify the selected prefix, bucket, or collection. The command exit status alone is not proof of remote deletion.
A request with the same input missesExamine model, request fields, route, policy eligibility, TTL, key version, and cache buster.
inspect cannot find a keyMake sure that you have the complete, untruncated key and the same cache backend and environment.

Next steps​