Overview
Data sources are the foundation of Insights analytics. Every query, dashboard widget, and report runs against a connected data source. Insights supports CSV file uploads and REST API connectors, with plans for database connectors in future releases.
CSV Upload
Upload CSV files to create queryable data sources. Insights parses the file, detects column types (numeric, text, date), and indexes the data for fast querying.
curl -X POST http://localhost:8010/api/v1/sources \
-H "Content-Type: application/json" \
-d '{
"name": "Ministry Budget Data",
"type": "csv",
"config": {"file_path": "/data/ministry_budgets.csv"}
}'Response:
{
"id": "src-a1b2c3",
"name": "Ministry Budget Data",
"type": "csv",
"status": "connected",
"config": {"file_path": "/data/ministry_budgets.csv"},
"row_count": 0,
"columns": [],
"created_at": "2024-01-15T10:30:00Z"
}After connection, Insights scans the file and populates row_count and columns with detected schema information.
Pre-Loaded Demo Data
Insights ships with two government-specific demo datasets for immediate exploration:
Government Analytics 2024
20 rows covering 5 departments across 4 quarters. Columns include:
| Column | Type | Description |
|---|---|---|
department | text | Department name (Digital, Education, Health, Finance, Infrastructure) |
quarter | text | Q1-Q4 2024 |
budget_aed_millions | numeric | Allocated budget in AED millions |
spending_pct | numeric | Budget utilization percentage |
citizen_satisfaction_pct | numeric | Citizen satisfaction score |
ai_adoption_pct | numeric | AI technology adoption rate |
Government Services Performance
10 rows of service-level metrics:
| Column | Type | Description |
|---|---|---|
service_name | text | Government service identifier |
monthly_requests | numeric | Volume of monthly citizen requests |
avg_processing_days | numeric | Average processing time in days |
digital_adoption_pct | numeric | Percentage of digital vs. in-person usage |
citizen_satisfaction_pct | numeric | Service-specific satisfaction score |
REST API Connectors
Connect external REST APIs as data sources. The connector periodically fetches data from the configured endpoint and indexes it for querying.
Source Selection
When creating queries or reports, specify the source_id to target a specific dataset. Each data source is queryable independently.
Data Source Management
List all connected sources and their status:
curl http://localhost:8010/api/v1/sourcesEach source reports its connection status, row count, detected columns, and last refresh timestamp. Sources in a connected state are available for querying; sources in error state need configuration fixes.
Column Detection
When a CSV is connected, Insights automatically detects column types:
- Numeric columns enable aggregation operations (sum, avg, min, max, count)
- Text columns enable grouping and filtering
- Date columns enable time-series analysis and trend queries
The detected schema determines which natural language questions can be answered and which chart types are suggested.