Plugins

Plugin Architecture#

Anar Gateway ships enterprise plugins purpose-built for sovereign government deployments. Plugins hook into the request lifecycle via four interfaces:

• LLMPlugin — Pre/post hooks around AI model calls
• HTTPTransportPlugin — Pre/post hooks around HTTP request handling
• MCPPlugin — Pre/post hooks around Model Context Protocol tool executions
• ObservabilityPlugin — Metrics and trace emission

All custom plugins are compiled into the Gateway binary. There is no dynamic loading or external plugin mechanism.

RBAC#

Role-based access control with API key and JWT authentication.

Authentication Methods#

API Key Authentication

Requests with a Bearer sk-anar-... token are matched against the configured API key map. Each key maps to a role.

curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer sk-anar-admin-key-123" \
  -d '{"model": "llama-3.3-70b-versatile", "messages": [...]}'

JWT Authentication

Standard JWT tokens (three-part dot-separated) have their payload decoded to extract role and sub claims. The Gateway does not verify JWT signatures — signature validation is handled by the issuing auth service or the OIDC plugin.

Default Roles#

Permission Matrix#

Configuration#

{
  "rbac": {
    "enabled": true,
    "default_role": "developer",
    "bypass_paths": ["/health", "/metrics", "/ready"],
    "api_keys": {
      "sk-anar-admin-prod": "admin",
      "sk-anar-dev-team-1": "developer",
      "sk-anar-auditor-ext": "auditor"
    },
    "roles": {
      "analyst": {
        "name": "analyst",
        "permissions": ["execute", "read"]
      }
    }
  }
}

OIDC#

OpenID Connect authentication with JWKS auto-discovery, supporting RSA and EC keys. This plugin validates JWT signatures cryptographically, unlike the RBAC plugin's claim extraction.

Configuration#

{
  "oidc": {
    "enabled": true,
    "issuer_url": "https://auth.example.gov/realms/anar",
    "audience": "anar-gateway",
    "required_scopes": ["ai:access"],
    "jwks_cache_seconds": 3600,
    "bypass_paths": ["/health", "/metrics"],
    "claim_mappings": {
      "subject_claim": "sub",
      "email_claim": "email",
      "role_claim": "role",
      "tenant_claim": "tenant"
    }
  }
}

Claim Mapping Defaults#

How It Works#

1. The plugin fetches the JWKS from the issuer's /.well-known/openid-configuration endpoint
2. JWKS keys are cached for the configured duration and refreshed automatically
3. Incoming Bearer tokens are validated against the cached keys (RSA + EC supported)
4. Claims are extracted and set in the request context for downstream plugins

Guardrails#

Integrates with Anar Guard for real-time content safety scanning on both input and output.

Request Flow#

1. Pre-LLM Hook: Extracts the last user message and sends it to Guard's evaluation endpoint
2. If Guard returns block: The request is short-circuited with a 403 response and the block reason
3. If Guard returns warn: A warning is attached to the context and the request proceeds
4. Post-LLM Hook (when scan_output: true): The assistant's response is scanned for policy violations

Configuration#

{
  "guardrails": {
    "guard_url": "http://guard:8002",
    "policy_ids": ["default-policy", "pii-detection"],
    "scan_output": true,
    "fail_open": true,
    "bypass_models": ["*embedding*", "meta-llama/llama-guard-4-12b"],
    "timeout_ms": 5000
  }
}

Resilience#

The guardrails plugin is designed for high availability:

• Connection pooling: 100 idle connections, 10 per host, 90-second idle timeout
• Fail-open mode: When Guard is down, requests proceed without scanning (configurable)
• Content extraction: Handles chat completions, text completions, and responses API formats

Block Response#

When a request is blocked, the Gateway returns:

{
  "error": {
    "message": "Content violates PII detection policy: UAE national ID detected",
    "type": "guardrails_blocked"
  }
}

HTTP status code: 403 Forbidden.

Audit Logging#

SOC2/GDPR-grade compliance event logging with async batch writing and automatic retention cleanup.

Tracked Events#

Event Schema#

Each audit event records:

Configuration#

{
  "audit": {
    "enabled": true,
    "store_type": "sqlite",
    "store_path": "./audit.db",
    "retention_days": 90,
    "track_actions": []
  }
}

Performance#

• Async batch writer: 4096-event channel, flushed in batches of 100 or every 2 seconds
• SQLite WAL mode: Concurrent reads during writes
• Retention cleanup: Runs hourly, deleting events older than the configured retention period

Cost Tracker#

Per-request cost calculation with built-in pricing for 30+ models.

How It Works#

After every LLM call, the cost tracker:

1. Extracts token usage from the response (prompt_tokens, completion_tokens)
2. Looks up the model's pricing (custom overrides take precedence over defaults)
3. Calculates cost: (prompt_tokens * prompt_price) + (completion_tokens * completion_price)
4. Persists a UsageRecord via async batch writing

Built-in Pricing#

Pricing is included for models from OpenAI, Anthropic, Google, Groq, and Azure. Prices are per token (not per 1K tokens).

Custom Pricing#

Override or add model pricing in the config:

{
  "costtracker": {
    "enabled": true,
    "store_path": "./costs.db",
    "custom_pricing": {
      "my-custom-model": {
        "prompt_price_per_token": 0.000001,
        "completion_price_per_token": 0.000004
      }
    }
  }
}

Query API#

The cost tracker exposes three HTTP endpoints:

Query parameters: model, provider, tenant_id, start_time, end_time, page, limit, sort_by, sort_dir.

S3 Payload Offloading#

Large request/response payloads (base64 images, long documents) are offloaded to S3-compatible object storage to prevent database bloat.

How It Works#

Payloads exceeding threshold_bytes (default 64KB) are stored in object storage. The log entry stores a reference pointer:

{"$ref": "s3://bucket/logs/{id}/{field}.json", "size": 5242880}

Small payloads remain inline with zero behavior change.

Supported Backends#

Configuration#

{
  "objectstore": {
    "backend": "s3",
    "bucket": "anar-gateway-logs",
    "endpoint": "localhost:9000",
    "access_key": "env:MINIO_ACCESS_KEY",
    "secret_key": "env:MINIO_SECRET_KEY",
    "use_ssl": false,
    "threshold_bytes": 65536
  }
}

The env: prefix resolves values from environment variables at runtime.

OpenTelemetry#

Native distributed tracing with HTTP and gRPC export protocols.

Capabilities#

• Trace export to any OpenTelemetry-compatible collector
• Prometheus metrics endpoint for scraping
• Three trace types covering request lifecycle
• Integration with the Anar observability stack (Grafana + Tempo + Loki + Prometheus)

Configuration#

Set the OTEL_EXPORTER_OTLP_ENDPOINT environment variable to enable:

OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318

When the endpoint is not set, the OTel plugin is a no-op with zero overhead.

Plugin Execution Order#

Plugins execute in a defined order during the request lifecycle:

Pre-request (inbound):

1. OIDC (token validation)
2. RBAC (permission check)
3. Guardrails (content safety scan)
4. Routing rules evaluation

Post-response (outbound):

1. Guardrails (output scan, if enabled)
2. Cost tracker (token/cost recording)
3. Audit (event logging)
4. OTel (trace export)