Skip to main content
A
Docs

Architecture

System architecture, service communication, data flow, and infrastructure patterns across the Anar platform.

System Overview

Anar is a microservices platform with 14 products — 1 Go service (Gateway) and 13 Python services. All services are containerized and orchestrated via Docker Compose for development and Kubernetes for production.

Service Communication

All inter-service communication uses REST APIs over HTTP. The Gateway sits at the center:

  • Client → Gateway — External clients hit the Gateway's OpenAI-compatible API for chat completions, embeddings, and model management.
  • Product → Gateway — Internal products route their LLM calls through the Gateway (when GATEWAY_URL is set) for unified cost tracking and safety scanning.
  • Product → Product — Direct service-to-service calls use the AnarServiceClient from the shared library. Example: Docs pushes OCR results to Chat's RAG ingest endpoint.

Data Layer

Each product owns its database:

ProductDatabaseNotes
ChatPostgreSQL + pgvector3072-dim embeddings, conversations, knowledge bases
GuardPostgreSQLScan results, policies, alerts, compliance reports
VoicePostgreSQLTranscriptions, diarization results
GatewaySQLiteCost tracking, audit logs, RBAC (embedded, no external DB)
OthersSQLite (async)Lightweight products use aiosqlite for persistence

Database Strategy

Products that need vector search or complex queries use PostgreSQL. Lightweight products use embedded SQLite to minimize infrastructure requirements — no external database needed.

Authentication Flow

  1. Client authenticates and receives a JWT (HS256-signed)
  2. JWT includes sub (user identifier) and role (admin/manager/user/viewer)
  3. Every API call includes the JWT in the Authorization: Bearer header
  4. The shared auth middleware validates the token and extracts the role
  5. Endpoint-level role checks enforce authorization

The Gateway adds a separate layer with OIDC support for SSO integration and API key authentication for service-to-service calls.

Observability Stack

All 13 Python products call setup_otel() from the shared library, which auto-instruments FastAPI, SQLAlchemy, and httpx with zero code changes. The Gateway has its own native OTel plugin.

  • Traces — Tempo stores distributed traces. Every HTTP request generates a trace with spans for middleware, database queries, and external API calls.
  • Logs — Loki aggregates structured JSON logs from all services.
  • Metrics — Prometheus scrapes the OTel Collector's metrics endpoint. Dashboards track request rates, latencies, error rates, and token usage.

Provider Architecture

The Gateway abstracts provider differences behind the OpenAI API format:

ProviderRegionUse Case
Azure OpenAI (UAE North)UAEProduction sovereign workloads
GroqUS (fast inference)Development, evaluation, fast iteration
Sovereign CloudGCCGovernment-mandated fallback

The Gateway's adaptive load balancer monitors health per provider/model pair using a 4-state machine (Healthy → Degraded → Failed → Recovering) and automatically shifts traffic away from unhealthy providers.

Shared Library

anar_shared (v0.3.0) is a Python package with 63 tests, used by all 13 Python products:

  • Auth — make_auth() and make_require_role() for JWT authentication
  • LLM — Provider-agnostic LLM client with automatic fallback chains
  • OTel — setup_otel() for one-line observability instrumentation
  • Service Client — AnarServiceClient for inter-service communication
  • Guard Telemetry — GuardTelemetry client for sending scan results to Guard
  • Production Hardening — CORS, rate limiting, health checks, error handling

Deployment Topology

Development (Docker Compose)

docker compose up -d                    # All 14 products
docker compose --profile infra up -d    # + LGTM observability stack

Production (Kubernetes)

Kubernetes manifests in infra/k8s/ define:

  • Base — Namespace, LGTM observability stack
  • Products — 14 Deployment + Service pairs, each with OTel environment variables

Pulumi IaC in infra/pulumi/ provides reusable AnarProduct and OTelStack component resources for declarative infrastructure management.

Frontends (Cloudflare Pages)

All 9 product dashboards and the marketing website deploy to Cloudflare Pages via Git integration. Each dashboard is a Vite + React SPA that communicates with its backend API.

IntroductionQuick Start