Skip to main content

AI Usage SIEM Streaming

Verdictan can stream bounded AI Usage Records to your SIEM or log-management platform.

The record contains the request, response, governance decisions, leakage findings, and financial metadata. Security and compliance teams can examine exposure, exfiltration, and policy violations.

Use request_id, record_id, and trail_event_id to correlate available evidence. Capture limits, redaction, filters, and delivery state affect what a destination receives.

Supported destinations​

Destination typeTarget systemAuth method
splunkSplunk HTTP Event Collector (HEC)HEC token (Authorization: Splunk)
elasticElasticsearch bulk APIAPI key (Authorization: ApiKey)
datadogDatadog logs-intake endpointDD-API-KEY header
generic_https_jsonAn HTTPS JSON collectorBearer token (Authorization: Bearer)

All destinations must use HTTPS endpoints. Non-HTTPS transports are not in scope. Verdictan validates each destination endpoint against its SSRF policy. Verdictan does this validation during configuration and each delivery attempt.

Configure a destination​

Prerequisites​

  • You must have the AI Usage Stream Write permission (AI_USAGE_STREAM_WRITE) on the organization.
  • Your destination must accept HTTPS ingestion.

Console setup​

  1. Go to Settings → AI Usage SIEM Streaming.
  2. Select Add Destination.
  3. Select the destination type (splunk, elastic, datadog, or generic_https_json).
  4. Type the endpoint URL.
  5. Wait for Verdictan to validate the URL against the SSRF policy. The policy rejects private, loopback, and link-local addresses.
  6. Type the destination credential. Verdictan encrypts the credential before storage.
  7. Select the redaction mode. Read Redaction modes.
  8. If you must stream a subset of records, configure event filters.
  9. Save the destination.

SIEM destination form with a synthetic HTTPS endpoint, redacted mode, and event filter

Example SIEM destination form with synthetic data. The form uses redacted mode.

Per-destination setup​

Splunk HEC​

  1. In Splunk, create an HTTP Event Collector token or use an available token.
  2. Get the HEC endpoint URL. A usual URL is https://your-splunk:8088/services/collector/event.
  3. In Verdictan, create a destination with type splunk, the HEC endpoint URL, and the HEC token as the credential.
  4. Verify that Verdictan sends each record as an NDJSON event envelope ({"event": <record>, "sourcetype": "verdictan:ai_usage", "source": "verdictan-api"}) using the Authorization: Splunk <token> header.

Elasticsearch​

  1. In Elasticsearch, create an API key with write access to the target index.
  2. Get the bulk API endpoint. A usual endpoint is https://your-elastic:9200/_bulk.
  3. In Verdictan, create a destination with type elastic, the bulk endpoint URL, and the API key as the credential.
  4. Verify that Verdictan writes each record to the verdictan-ai-usage index. The request must use the Elasticsearch bulk NDJSON format. The format has an index action line and then the record. The request uses the Authorization: ApiKey <key> header.

Datadog​

  1. In Datadog, create an API key or use an available API key.
  2. Get the logs-intake endpoint for your Datadog site. Use this usual US endpoint: https://http-intake.logs.datadoghq.com/api/v2/logs
  3. In Verdictan, create a destination with type datadog, the logs-intake URL, and the API key as the credential.
  4. Verify that Verdictan sends each record to the logs-intake endpoint as a JSON log object (ddsource: verdictan, service: verdictan-ai-usage, with the record in message) using the DD-API-KEY header.

Generic HTTPS JSON collector​

  1. Get your collector's HTTPS ingestion endpoint.
  2. Provision a bearer token that your collector accepts.
  3. In Verdictan, create a destination with type generic_https_json, the endpoint URL, and the bearer token as the credential.
  4. Verify that Verdictan sends a POST request with a JSON object ({"source": "verdictan", "schema_version": "1", "records": [<record>]}). The request uses the Authorization: Bearer <token> and Idempotency-Key headers.

Redaction modes​

Each destination has its own redaction mode. The mode controls how Verdictan delivers request and response content.

ModeBehaviorDefault
RedactedVerdictan replaces request_body and response_body with [REDACTED] before delivery. The record keeps detector findings and SHA-256 hashes.✅ Yes
RawVerdictan transmits captured request and response content without destination redaction. Capture limits continue to apply.No

Select a redaction mode​

  • Use Redacted mode for most deployments. It sends available detector findings without captured request or response content.

    The record contains each detector finding, category, entity type, confidence, and masked location. Investigators can see the detected data and matched policies.

    They can also see the actions that Verdictan took.

  • Organizations can use Raw mode for captured forensic content in their SIEM. Before you enable raw mode, acknowledge the specified data-handling rules.

    The raw_data_handling_acknowledged field records this acknowledgement. Verdictan records the user and timestamp with the acknowledgement.

caution

Make sure that your SIEM controls are correct for unredacted content. Raw mode sends unredacted request and response bodies to the external destination.

The gateway can truncate the request body or response body at the configured capture limit. Examine request_truncated and response_truncated first.

These controls include access, retention, and data-handling agreements.

Event filters​

Each destination can define an optional event_filter. The API evaluates the filter when it captures an AI Usage Record.

The result specifies which active destinations receive outbox rows.

  • Evaluation point: The API evaluates the filter when it enqueues the record. The same transaction stores the record and writes matching delivery rows.
  • No data loss on mismatch: The API always stores the AI Usage Record. The API does not create a delivery row for a destination that does not match.
  • Empty filter = match all: Do not include event_filter to stream each record to the destination. You can also set it to null or {}.
  • AND across fields: When the filter contains multiple fields, all fields must match.
  • One value in arrays: String and http_status fields use exact-match arrays. A record matches when its stored value equals one value in the array.

The API validates the structure during create and update operations. The API rejects unknown fields, empty arrays, empty strings, and out-of-range HTTP status values.

Supported filter fields​

These fields map directly to ai_usage_records columns:

Filter fieldRecord columnMatch type
request_familyrequest_familyexact-match string array
transporttransportexact-match string array
providerproviderexact-match string array
modelmodelexact-match string array
upstream_hostupstream_hostexact-match string array
outcomeoutcomeexact-match string array
gateway_idgateway_idexact-match string array
agent_idagent_idexact-match string array
actor_typeactor_typeexact-match string array
currencycurrencyexact-match string array
sourcesourceexact-match string array (gateway today)
http_statushttp_statusexact-match integer array
streamedstreamedspecified boolean match
scannedscannedspecified boolean match
secrets_detectedsecrets_detectedspecified boolean match
potential_data_leakagepotential_data_leakagespecified boolean match

Examples​

Route only blocked or flagged OpenAI records that detected possible leakage:

{
  "provider": ["openai"],
  "outcome": ["block", "flag"],
  "potential_data_leakage": true
}

Route only streamed MCP traffic:

{
  "request_family": ["mcp"],
  "streamed": true
}

Route only upstream throttling or server errors:

{
  "http_status": [429, 500, 502, 503, 504]
}

AI Usage Record schema​

When capture succeeds, a governed AI interaction creates one AI Usage Record. The record contains these groups:

Envelope​

FieldDescription
schema_versionThis field contains the record schema version.
record_idThis field contains the unique record identifier (UUID).
org_idThis field contains the organization identifier.
sequenceThis field contains the monotonic sequence number for the organization.
captured_atThis field contains the server-owned capture timestamp.
sourceThis field identifies the capture source. The source is gateway.

Identity and tenancy​

FieldDescription
gateway_idThis field identifies the gateway that processed the interaction.
agent_idThis field contains the agent identifier.
subject_token_idThis field identifies the token that the client used for the request.
actor_id / actor_typeThese fields contain the actor identity and type.
session_idThis field contains the session identifier.
correlation_idThis field contains the cross-system correlation ID.
request_idThis field contains the unique request identifier.

Interaction​

FieldDescription
request_familyThis field identifies the request family (Chat, Responses, Messages, WebSocket, MCP).
transportThis field identifies the transport type (buffered, sse, websocket, mcp).
provider / modelThese fields identify the AI provider and model.
upstream_hostThis field contains only the upstream host. It does not contain credentials.
streamedThis field reports if Verdictan streamed the response.
started_at / completed_atThese fields contain the start and completion timestamps.
latency_msThis field contains the total latency in milliseconds.
outcomeThis field contains the governance outcome (allow, block, flag, error).
http_statusThis field contains the upstream HTTP status code.

Content​

FieldDescription
request_body / response_bodyThese fields contain captured request and response content. The gateway can truncate one or the two values.
request_bytes / response_bytesThese fields contain the content byte counts.
request_sha256_raw / response_sha256_rawThese fields contain the SHA-256 hashes of raw content.
request_sha256_redacted / response_sha256_redactedThese fields contain the SHA-256 hashes of redacted content.
request_truncated / response_truncatedThese fields report if Verdictan truncated the content.

Governance​

FieldDescription
matched_policiesThis field contains the policies that matched the interaction.
decisionsThis field contains the applied governance decisions.
applied_mutationsThis field contains mutations that Verdictan applied to the request or response.
applied_redactionsThis field contains the applied redactions.
output_stage_evaluationsThis field contains output-stage evaluation results.
human_oversight / escalation / flagged_reviewThese fields contain oversight references.

Data leakage​

FieldDescription
detectors[]This array contains detector findings. Read the subsequent table.
redaction_mode_appliedVerdictan adds this field to redacted deliveries. The value is redacted when Verdictan removes the request and response bodies.
secrets_detectedThis field reports if Verdictan detected credential or secret patterns.
attachments[]This field contains file metadata (type, size, sha256, scanned).
potential_data_leakageThis Boolean reports if Verdictan detected data leakage.

Each entry in detectors[] includes:

FieldDescription
detectorThis field contains the detector name.
categoryThis field contains the detection category.
entity_typeThis field identifies the detected entity type.
match_countThis field contains the number of matches.
confidenceThis field contains the detection confidence.
matched_policy_idsThis field identifies the policies that triggered.
actionThis field contains the action (redacted, blocked, allowed).
evidence_locationsThis field contains masked location evidence. It contains only offsets and hashes.
sample_hashThis field contains a non-reversible hash of the matched content.

Detector families include PII, HIPAA, PHI, DLP custom rules, and student privacy. They also include three platform scanners. These scanners find credential patterns, secret patterns, and high-entropy content.

An empty detectors[] with scanned: true means that the completed scan found no reportable matches within its configured limits.

Financial​

FieldDescription
prompt_tokens / completion_tokens / cached_input_tokens / total_tokensThese fields contain token counts.
cost / currencyThese fields contain the interaction cost and currency.
wallet_transaction_id / budget_id / reservation_request_id / spend_log_idThese fields contain financial references.

Integrity​

FieldDescription
canonical_sha256This field contains the RFC-8785 canonical hash of the record.
trail_event_idThis field correlates the record to the governance trail.

Delivery guarantees​

  • Delivery guarantee: Verdictan delivers each matching record one or more times. It retries a failed delivery until delivery succeeds or Verdictan quarantines the record.

    Each delivery has a deterministic idempotency key (<record_id>:<destination_id>). The generic HTTPS adapter sends the key in the Idempotency-Key header.

    Each record has a stable record_id and canonical_sha256. Your SIEM can use these values to remove duplicates.

  • Durable scheduling: Verdictan stores an accepted record and its matching destination state together. It does not schedule a filtered destination.

  • Sequenced attempts: Verdictan sends records in sequence for each destination. It applies exponential backoff after failed attempts.

    A retried record can be delivered after subsequent records.

  • Poison-record isolation: Verdictan retries a failed record a maximum of five times. Then, Verdictan quarantines it.

    The failed record does not block subsequent records.

Monitoring delivery health​

The delivery health view shows per-destination status:

  • Pending / sent counts show records that wait for delivery and records that Verdictan delivered.
  • Retryable / quarantined counts show records that await a different attempt. These counts also show records that Verdictan parked after all retries failed.
  • Last delivered / last error shows the most recent delivery that succeeded. It also shows the most recent error from delivery.
  • Attempt history shows recent outcomes for each destination attempt. These outcomes include the HTTP status, error code, and duration.

Gateway policy configuration​

The gateway ai_usage_streaming policy stanza controls capture behavior on the data plane. It does not control redaction or destination routing. You manage redaction and destination routing in the console.

policy-config.yaml
ai_usage_streaming:
  enabled: true
  body_capture_max_bytes: 65536   # 64 KiB per request/response (default)
  mandatory: false                # true = fail-closed if capture enqueue fails
FieldDescriptionDefault
enabledThis field enables capture of AI Usage Records.false
body_capture_max_bytesThis field sets the maximum captured bytes for each request or response body. Verdictan limits the value to 1 MiB.65536 (64 KiB)
mandatoryThis field enables fail-closed behavior. Verdictan denies the AI request if durable capture enqueue fails.false
warning

If the durable enqueue fails, mandatory: true makes the gateway return a 503 error. The gateway does not continue the AI request.

Use this setting when the request must stop after a capture enqueue failure.

Next steps​