Skip to main content

Document Analyzer

The document-analyzer policy checks documents in request JSON. It supports documents, attachments, input_documents, and files. It can block large documents or denied MIME types. It records extracted and sanitized text in policy details. It does not change the request payload.

Configuration​

pack:
  name: document-analyzer-example
  version: 1.0.0
  enabled: true

policies:
  chain:
    - document-analyzer

policy:
  document-analyzer:
    enabled: true
    sanitize_code: true
    max_document_bytes: 524288
    allowed_mime_types:
      - application/pdf
      - text/plain
      - application/vnd.openxmlformats-officedocument.wordprocessingml.document

Fields​

FieldTypeDescriptionDefault
enabledbooleanDisable the analyzer without removing it from the chain.true
sanitize_codebooleanApply the built-in code patterns to the text copy in policy details.true
max_document_bytesintegerBlock a document when its raw payload exceeds this byte size.524288
allowed_mime_typesstring[]Optional MIME allowlist. An empty list accepts all MIME types.[]

Supported request shapes​

The analyzer looks for arrays in these request keys:

  • documents
  • attachments
  • input_documents
  • files

For each item it reads:

  • name (optional, defaults to inline-document)
  • mime_type or content_type
  • text for inline text documents, or content_base64 / data for encoded bytes

Extraction behavior​

  • application/pdf → heuristic PDF text extraction
  • DOCX MIME types containing wordprocessingml.document → text from word/document.xml
  • Everything else → lossy UTF-8 text conversion of the raw bytes

Code sanitization behavior​

When sanitize_code is true, the policy examines document text for the built-in patterns from the gateway code-sanitizer helper:

  • rm -rf
  • drop table
  • 169.254.169.254
  • curl http://localhost / curl https://localhost
  • chmod 777
  • recursive aws s3 cp

The analyzer replaces matches with [redacted code pattern] in its result copy. It does not write that copy into the request or subsequent policy input.

Use cases​

MIME and size guardrail​

pack:
  name: upload-screening
  version: 1.0.0
  enabled: true

policies:
  chain:
    - document-analyzer
    - audit-logger

policy:
  document-analyzer:
    max_document_bytes: 1048576
    allowed_mime_types:
      - application/pdf
      - text/plain
      - image/png
      - image/jpeg

How it works​

  1. The analyzer reads document entries from the structured request payload.
  2. It blocks immediately if a document exceeds max_document_bytes.
  3. If allowed_mime_types is not empty, it checks each document MIME type.
  4. It blocks a document with a MIME type that is not in the list.
  5. It gets text from each accepted document.
  6. It records the source name, MIME type, character count, and code-block count.
  7. It adds the extracted and sanitized text fragments to policy details.
  8. It does not add these fragments to messages or subsequent policy input.

Best practices​

  • Keep the MIME allowlist small and specified in production.
  • Keep sanitize_code enabled when authorized reviewers must have a sanitized result copy.
  • Do not use the sanitized result as proof that the provider received changed content.
  • This policy does not get remote URLs. It does not parse multipart uploads that are not in the JSON shapes above.

Next steps​