Configuration

Configuration File#

The Gateway is configured via config.json, which defines providers, API keys, network settings, and the config store. The file is read at startup and can be overridden at runtime via the config store API.

Minimal Example#

{
  "client": {
    "drop_excess_requests": false
  },
  "providers": {
    "groq": {
      "keys": [
        {
          "name": "groq-primary",
          "value": "env.GROQ_API_KEY",
          "models": ["llama-3.3-70b-versatile", "allam-2-7b"],
          "weight": 1.0
        }
      ],
      "network_config": {
        "base_url": "https://api.groq.com/openai/v1",
        "max_retries": 3,
        "retry_backoff_initial_ms": 100,
        "retry_backoff_max_ms": 5000,
        "default_request_timeout_in_seconds": 60
      },
      "custom_provider_config": {
        "base_provider_type": "openai",
        "allowed_requests": {
          "chat_completion": true,
          "chat_completion_stream": true
        }
      },
      "concurrency_and_buffer_size": {
        "concurrency": 50,
        "buffer_size": 200
      }
    }
  },
  "config_store": {
    "enabled": true,
    "type": "sqlite",
    "config": {
      "path": "./config.db"
    }
  }
}

Provider Configuration#

Each provider block contains four sections: keys, network config, custom provider config, and concurrency settings.

Keys#

API keys are defined as an array, allowing multiple keys per provider for load balancing across accounts.

{
  "keys": [
    {
      "name": "azure-uae-north",
      "value": "env.AZURE_OPENAI_API_KEY",
      "models": ["gpt-4o-uaenorth", "text-embedding-3-large-uaenorth"],
      "weight": 1.0
    }
  ]
}

Network Config#

Controls HTTP behavior for upstream provider connections.

{
  "network_config": {
    "base_url": "https://api.groq.com/openai/v1",
    "max_retries": 3,
    "retry_backoff_initial_ms": 100,
    "retry_backoff_max_ms": 5000,
    "default_request_timeout_in_seconds": 60
  }
}

Custom Provider Config#

Configures how the Gateway translates requests for each provider.

{
  "custom_provider_config": {
    "base_provider_type": "azure_openai",
    "allowed_requests": {
      "chat_completion": true,
      "chat_completion_stream": true,
      "embeddings": true
    }
  }
}

Supported request types: chat_completion, chat_completion_stream, embeddings, text_completion, responses.

Concurrency and Buffer Size#

Controls the connection pool per provider.

{
  "concurrency_and_buffer_size": {
    "concurrency": 100,
    "buffer_size": 500
  }
}

Pre-Configured Providers#

The Gateway ships pre-configured for three providers optimized for sovereign GCC deployments:

Azure OpenAI (UAE North)#

Sovereign data residency in the UAE North region. Used for embeddings and production chat.

{
  "azure": {
    "keys": [{
      "name": "azure-uae-north",
      "value": "env.AZURE_OPENAI_API_KEY",
      "models": ["gpt-4o-uaenorth", "text-embedding-3-large-uaenorth"],
      "weight": 1.0
    }],
    "network_config": {
      "base_url": "https://aimodels-uaenorth.cognitiveservices.azure.com/openai",
      "default_request_timeout_in_seconds": 120
    },
    "custom_provider_config": {
      "base_provider_type": "azure_openai"
    },
    "concurrency_and_buffer_size": {
      "concurrency": 100,
      "buffer_size": 500
    }
  }
}

Groq (Fast Inference)#

High-throughput inference for development and production chat workloads.

{
  "groq": {
    "keys": [{
      "name": "groq-primary",
      "value": "env.GROQ_API_KEY",
      "models": [
        "llama-3.3-70b-versatile",
        "allam-2-7b",
        "meta-llama/llama-4-scout-17b-16e-instruct",
        "meta-llama/llama-4-maverick-17b-128e-instruct",
        "meta-llama/llama-guard-4-12b",
        "qwen/qwen3-32b",
        "openai/gpt-oss-120b",
        "llama-3.1-8b-instant"
      ],
      "weight": 1.0
    }],
    "network_config": {
      "base_url": "https://api.groq.com/openai/v1",
      "default_request_timeout_in_seconds": 60
    },
    "custom_provider_config": {
      "base_provider_type": "openai"
    },
    "concurrency_and_buffer_size": {
      "concurrency": 50,
      "buffer_size": 200
    }
  }
}

Sovereign Cloud (Fallback)#

Sovereign cloud infrastructure used as a fallback when primary providers are unavailable.

{
  "sovereign": {
    "keys": [{
      "name": "sovereign-primary",
      "value": "env.CORE42_API_KEY",
      "models": ["gpt-4o", "gpt-4.1", "text-embedding-3-large"],
      "weight": 1.0
    }],
    "network_config": {
      "base_url": "https://api.core42.ai/v1",
      "default_request_timeout_in_seconds": 120
    },
    "custom_provider_config": {
      "base_provider_type": "openai"
    },
    "concurrency_and_buffer_size": {
      "concurrency": 100,
      "buffer_size": 500
    }
  }
}

Environment Variables#

Config Store#

The config store enables runtime configuration changes without restarting the Gateway. It uses GORM for persistence with distributed locking support.

{
  "config_store": {
    "enabled": true,
    "type": "sqlite",
    "config": {
      "path": "./config.db"
    }
  }
}

When enabled, the config store provides APIs for managing virtual keys, teams, customers, budgets, rate limits, and routing rules — all persisted to SQLite (development) or PostgreSQL (production).

Plugin Configuration#

Plugins are configured via their respective JSON blocks in the main config or via environment variables. See the Plugins page for detailed configuration of each plugin.

Example: Enabling RBAC + Audit#

{
  "plugins": {
    "rbac": {
      "enabled": true,
      "default_role": "developer",
      "bypass_paths": ["/health", "/metrics"],
      "api_keys": {
        "sk-anar-admin-key": "admin",
        "sk-anar-dev-key": "developer"
      }
    },
    "audit": {
      "enabled": true,
      "store_type": "sqlite",
      "store_path": "./audit.db",
      "retention_days": 90
    }
  }
}

Docker Configuration#

The Gateway Dockerfile produces a CGO-free static binary running as a non-root user:

FROM golang:1.23-alpine AS builder
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w" -o anar-gateway .

FROM alpine:3.21
RUN adduser -D gateway
USER gateway
EXPOSE 8080
ENTRYPOINT ["./anar-gateway"]

The Docker Compose setup includes a MinIO sidecar for S3-compatible payload offloading:

docker compose up -d

This starts the Gateway on port 8080 and MinIO on ports 9000 (API) and 9001 (console), with persistent volumes for both services.