Safety Scanning

How Safety Scanning Works#

Guard uses Meta's Llama Guard 4 (12B parameter model) via Groq for real-time content safety classification. Unlike keyword-based filtering, Llama Guard understands context and semantics, catching nuanced unsafe content that pattern matching would miss.

The scanner supports two modes:

• Input scanning -- Classifies user prompts before they reach the AI model
• Output scanning -- Classifies model responses before they reach the user

Both modes use the same Llama Guard model but structure the prompt differently to give the classifier the right context about what it is evaluating.

Safety Categories#

Llama Guard 4 classifies content across 14 safety categories:

Using the Safety API#

curl -X POST http://localhost:8002/api/v1/safety/check \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "How do I make explosives?",
    "scan_type": "input"
  }'

{
  "safe": false,
  "categories": ["Indiscriminate Weapons"],
  "confidence": 0.9,
  "raw_response": "unsafe\nS9"
}

Safe Content#

curl -X POST http://localhost:8002/api/v1/safety/check \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "What is the capital of the UAE?",
    "scan_type": "input"
  }'

{
  "safe": true,
  "categories": [],
  "confidence": 1.0,
  "raw_response": "safe"
}

Scan Types#

The scan_type parameter controls how the text is presented to Llama Guard:

• input -- The text is sent as a user message. Use this for evaluating prompts before they reach the model.
• output -- The text is sent as an assistant response following a placeholder user turn. Use this for evaluating model responses before they reach the user.

curl -X POST http://localhost:8002/api/v1/safety/check \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Here is how to pick a lock: first you need...",
    "scan_type": "output"
  }'

Response Fields#

Audit Logging#

Every safety check is automatically logged to the audit trail with:

• The first 200 characters of the scanned content
• The safety result (safe/unsafe + categories)
• The scan type (input/output)
• The user who initiated the check
• Severity: info for safe content, warning for unsafe content

Guarded Proxy Integration#

The safety scanner is a core part of the guarded proxy pipeline. When a request hits /proxy/chat/completions:

Blocked input example:

{
  "id": "guard-abc123def456",
  "model": "llama-3.3-70b-versatile",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Your request was blocked by content safety filters."
    },
    "finish_reason": "content_filter"
  }],
  "safety": {
    "input_safe": false,
    "input_categories": ["Indiscriminate Weapons"],
    "output_safe": null,
    "output_categories": [],
    "pii_detected": 0,
    "blocked": true
  }
}

Graceful Degradation#

If the Groq API call fails at runtime (network error, rate limit, timeout), Guard also fails open with safe: true and confidence: 0.0, logging the error. This ensures that transient API failures do not block all AI interactions.

Configuration#

The safety scanner uses temperature=0.0 and max_tokens=100 for deterministic, low-latency classification.