Anar Chat

Overview#

Anar Chat is a full-stack conversational AI platform built for GCC government organizations. It combines retrieval-augmented generation (RAG) with multi-provider LLM routing, knowledge base management, conversation memory, and role-based access control to deliver accurate, source-grounded answers in Arabic and English.

The backend is a FastAPI service backed by PostgreSQL with pgvector for semantic search. The frontend provides a streaming chat interface with conversation history, knowledge base selection, and document management.

Key Capabilities#

Arabic-First RAG#

Chat processes Arabic documents with specialized text normalization (diacritics removal, alef/teh unification, tatweel stripping) and sentence-aware chunking. Language detection automatically routes Arabic, English, and mixed-language content through the appropriate chunking pipeline, ensuring high-quality retrieval across both languages.

Multi-Provider LLM Routing#

Every chat completion request can target a specific model and provider. Chat supports Groq for fast development inference, Azure UAE North for sovereign production workloads, and GCC sovereign cloud infrastructure as a fallback. When connected to Anar Gateway, all model calls route through the unified proxy for cost tracking and load balancing.

Knowledge Bases and Document Ingestion#

Admins and managers create knowledge bases and upload PDF, DOCX, and TXT documents. The ingestion pipeline extracts text, chunks it with configurable overlap, generates 3072-dimensional embeddings via Azure text-embedding-3-large, and stores vectors in pgvector for cosine similarity search.

LightRAG Graph-Enhanced Retrieval#

When enabled via feature flag, LightRAG builds a knowledge graph on top of ingested documents. At query time, it performs hybrid retrieval that combines graph-based entity and relationship traversal with traditional vector search, producing richer context for complex multi-hop questions.

Streaming SSE Responses#

Chat completions stream token-by-token via Server-Sent Events. Each SSE frame carries a single token and the conversation ID, giving clients real-time output. The full response is persisted to the conversation history after streaming completes.

Conversation Management#

Full conversation lifecycle management with search, filtering, and analytics. The dashboard provides a conversation list with message counts, a real-time analytics page tracking usage patterns, and RAG query/search capabilities for exploring retrieved context.

Guard Telemetry#

When connected to Anar Guard, Chat reports LLM call telemetry (model, tokens, latency) for centralized governance monitoring and cost tracking across the platform.

JWT Authentication with RBAC#

All endpoints (except registration, login, and health) require a valid JWT. Four roles control access: admin has full access, manager can create knowledge bases and upload documents, user can chat and view knowledge bases, and viewer has read-only access.

Architecture#

Chat runs as a single FastAPI process. PostgreSQL stores users, conversations, messages, documents, knowledge bases, and vector embeddings (via pgvector). LLM calls go either directly to a provider or through Anar Gateway when GATEWAY_URL is set.

RBAC Roles#

Quick Start#

Register a user, create a knowledge base, upload a document, and start chatting:

# Register
TOKEN=$(curl -s http://localhost:8001/api/v1/auth/register \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"secure-password-here","role":"admin"}' \
  | jq -r '.access_token')

# Create a knowledge base
KB_ID=$(curl -s http://localhost:8001/api/v1/knowledge-bases \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"UAE AI Policy","description":"AI strategy documents"}' \
  | jq -r '.id')

# Upload a document
curl -s http://localhost:8001/api/v1/documents/upload \
  -H "Authorization: Bearer $TOKEN" \
  -F "file=@strategy.pdf" \
  -F "knowledge_base_id=$KB_ID"

# Chat with the knowledge base
curl -N http://localhost:8001/api/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"message\":\"What is the UAE AI strategy?\",\"knowledge_base_id\":\"$KB_ID\"}"

Configuration#

Key environment variables for Chat:

Observability#

Chat is instrumented with OpenTelemetry via anar_shared.setup_otel(). When the OTEL_EXPORTER_OTLP_ENDPOINT environment variable is set, traces, logs, and metrics are exported to the collector. Without it, instrumentation is a no-op.

The /api/v1/metrics endpoint exposes application-level counters (total requests, errors, chat completions, documents uploaded) and uptime.

MCP Server#

Chat exposes a Model Context Protocol server at /mcp, enabling integration with AI assistants and development tools. The MCP server surfaces all Chat API operations as tool calls.

Test Suite#

366 tests cover auth, RAG pipeline, streaming, knowledge base management, document ingestion, conversation management, analytics, and LLM provider routing.

cd chat/backend && uv run pytest

Next Steps#

• RAG Pipeline -- how documents are chunked, embedded, and retrieved
• Knowledge Bases -- creating and managing document collections
• Streaming -- SSE streaming response format and client integration
• API Reference -- complete endpoint documentation