Skip to main content

Embedding Detector

The embedding-detector policy compares message content with reference text. The default local mode uses a bag-of-words cosine score. The external mode can call an embedding endpoint when the installed CLI includes that support.

Configuration​

pack:
  name: embedding-detector-example
  version: 1.0.0
  enabled: true

policies:
  chain:
    - embedding-detector

policy:
  embedding-detector:
    backend: local
    similarity_threshold: 0.75
    timeout_ms: 20
    action: block
    categories:
      - label: trade_secret
        reference_text: proprietary manufacturing process internal formula secret recipe
      - label: competitive_intel
        reference_text: competitor pricing strategy acquisition target market positioning

Fields​

FieldTypeDescriptionDefault
backendlocal | externalSelects the backend. external makes HTTP calls only in a build with --features embedding-external.local
endpointstringExternal embedding URL. Only used for backend: external.http://localhost:8080/embed
modelstringExternal embedding model name. Only used for backend: external.all-MiniLM-L6-v2
api_keystringOptional bearer token sent to the external endpoint.unset
similarity_thresholdnumberMinimum cosine similarity that triggers.0.70
timeout_msintegerExternal backend timeout in milliseconds.20
actionredact | blockblock stops the request. redact returns a redact verdict for the gateway redaction path.redact
categoriesarrayMore categories appended to the built-in sensitive categories.[]
categories[].labelstringStable label for the category.necessary
categories[].reference_textstringReference text compared against each segment of the input.necessary

Built-in sensitive categories​

When categories is empty, the detector starts with built-in references for:

  • Social Security numbers
  • credit cards / payment cards
  • medical-record identifiers
  • biometric data terms
  • bank-account style identifiers
  • prompt-injection language
  • export-controlled technical data

External backend note​

For backend: external, do these steps:

  • Make sure that the installed CLI includes external embedding support.
  • Give endpoint and, if necessary, model and api_key.
  • The local backend handles timeout or transport failures.
External errors use local scoring

An external timeout or transport error changes the request to the local bag-of-words check. Use this mode only when that result meets your policy.

Use cases​

Local-only semantic screening​

pack:
  name: local-similarity-screening
  version: 1.0.0
  enabled: true

policies:
  chain:
    - embedding-detector
    - audit-logger

policy:
  embedding-detector:
    backend: local
    similarity_threshold: 0.8
    action: block

Feature-gated external endpoint​

pack:
  name: external-embedding-screening
  version: 1.0.0
  enabled: true

policies:
  chain:
    - embedding-detector

policy:
  embedding-detector:
    backend: external
    endpoint: https://embeddings.example.com/v1/embeddings
    model: approved-embedding-model
    timeout_ms: 200
    similarity_threshold: 0.8
    action: block

How it works​

  1. The detector divides the input into sentence-like segments.
  2. In local mode, it compares each segment with built-in and custom reference text.
  3. In external mode it requests embeddings for the segment and each reference text, subject to timeout_ms.
  4. If the external call fails or times out, the detector falls back to the local path.
  5. The first matching segment sets the policy cause code and verdict.

Best practices​

  • Use categories to extend the built-in defaults. They do not replace the defaults.
  • Use higher thresholds (0.8+) first to decrease false positives.
  • Reserve external mode for cases where you control the build flags and the endpoint.
  • Use block for a deterministic outcome, not a generic redact verdict.

Next steps​