How Assessments Work
A compliance assessment evaluates an AI system's features against the requirements of one or more regulatory frameworks. The assessment engine maps declared system capabilities to framework requirements, applies weighted scoring, and produces a report with an overall score, per-framework scores, and identified gaps.
Running an Assessment
Submit a system profile with the frameworks you want to assess against:
curl -X POST "http://localhost:8004/api/v1/comply/assess?framework_ids=uae-ai-ethics&framework_ids=sdaia-ai-ethics" \
-H "Content-Type: application/json" \
-d '{
"system_name": "Anar Chat",
"system_type": "conversational_ai",
"risk_level": "medium",
"data_types": ["personal_data", "government_data"],
"deployment_region": "uae",
"features": {
"has_audit_trail": true,
"has_bias_detection": true,
"has_human_oversight": true,
"has_data_encryption": true,
"has_consent_mechanism": false,
"has_explainability": true
}
}'System Profile Fields
| Field | Type | Description |
|---|---|---|
system_name | string | Name of the AI system being assessed |
system_type | string | System category: conversational_ai, classification, recommendation, decision_support |
risk_level | string | Self-declared risk level: low, medium, high, critical |
data_types | string[] | Types of data processed: personal_data, government_data, financial_data, health_data |
deployment_region | string | Where the system is deployed: uae, saudi, qatar, oman, eu |
features | object | Boolean flags for system capabilities |
Feature Flags
| Feature | Description |
|---|---|
has_audit_trail | System logs all AI decisions and interactions |
has_bias_detection | Algorithmic bias detection is implemented |
has_human_oversight | Human review mechanisms exist for AI decisions |
has_data_encryption | Data encrypted at rest and in transit |
has_consent_mechanism | User consent is obtained before data collection |
has_explainability | AI outputs include explanations |
Weighted Scoring
Each framework requirement has a weight that determines its impact on the overall score. Requirements marked as critical have the highest weight, followed by high, medium, and low.
The scoring formula:
- Each met requirement contributes its full weight to the score
- Each unmet requirement contributes zero
- The overall score is the percentage of total weight achieved
overall_score = (sum of met requirement weights / sum of all requirement weights) * 100
For example, if a framework has 12 requirements with total weight of 100, and the system meets 10 requirements totaling weight 78, the score is 78.0.
Scoring Strategy
Focus on critical and high-weight requirements first. A system that meets all critical requirements but misses some low-weight ones will score significantly higher than a system that meets many low-weight requirements but misses critical ones.
Assessment Report
The assessment produces a report with aggregate scores and per-framework breakdowns:
{
"id": "rpt-001",
"system_name": "Anar Chat",
"overall_score": 78.5,
"framework_scores": {
"uae-ai-ethics": 82.0,
"sdaia-ai-ethics": 75.0
},
"total_requirements": 24,
"met_requirements": 19,
"gap_count": 5,
"assessed_at": "2024-01-15T10:30:00Z"
}Gap Analysis
Every unmet requirement is surfaced as a gap with severity and remediation guidance:
curl "http://localhost:8004/api/v1/comply/gaps?report_id=rpt-001&severity=high"[
{
"requirement_id": "req-005",
"framework_id": "uae-ai-ethics",
"requirement": "Consent mechanism for data collection",
"severity": "high",
"status": "not_met",
"remediation": "Implement explicit user consent flow before data collection"
}
]Filtering Gaps
| Parameter | Type | Description |
|---|---|---|
report_id | string | Required — which assessment report to query |
framework_id | string | Filter gaps to a specific framework |
severity | string | Filter by severity: critical, high, medium, low |
Guard Integration
When Anar Guard is deployed alongside the assessed system, Comply can pull live governance data into the assessment. Instead of relying solely on declared feature flags, the assessment engine checks Guard for:
- Policy enforcement status — Are content policies active and enforced?
- PII detection coverage — Is PII scanning enabled for the system's data types?
- Audit trail completeness — Are all interactions logged with full request/response data?
- Alert configuration — Are compliance-relevant alerts configured and active?
This live integration ensures that compliance assessments reflect the actual deployed state of the system, not just what was declared during the assessment.
Tracking Compliance Over Time
Run assessments periodically to track compliance score progression. The Timeline view in the dashboard charts score trends across assessments, making it easy to demonstrate continuous improvement to regulators and auditors.
# List all reports
curl http://localhost:8004/api/v1/comply/reports
# Get a specific report
curl http://localhost:8004/api/v1/comply/reports/rpt-001