Technical Architecture
Aceh Agentic RAG + OCR is a self-hosted backend platform for Pemerintah Aceh. It retrieves and answers questions over 100,000+ government documents from JDIH, OpenData, PPID, and SatuData in Bahasa Indonesia and English, while keeping data and inference inside Huawei Cloud.
System Goals
- Serve 10-50 internal concurrent users through API and WebSocket surfaces.
- Support Agentic RAG over PDFs, Word files, spreadsheets, scanned images, TXT, and related government document formats.
- Keep all data sovereignty requirements intact by using self-hosted components only.
- Attach source citations to every generated answer and run a faithfulness check before returning it.
- Apply the Government Policy Guard on every output for anti-SARA, anti-jailbreak, and government communication-policy filtering.
Core Stack
| Layer | Technology | Purpose |
|---|---|---|
| API | FastAPI | Async HTTP/WebSocket API, auth, rate limiting, sessions, validation |
| Model client | LiteLLM | OpenAI-compatible model calls, retries, fallback routing, tracing hooks |
| Orchestration | Custom Python + Pydantic | Router, Planner, Executor, Synthesizer pipeline |
| Model serving | vLLM | Self-hosted LLM, embedding, and reranker endpoints |
| Retrieval store | PostgreSQL + pgvector + full-text search | HNSW semantic retrieval plus lexical keyword retrieval |
| Prompt and observability | Langfuse | Prompt versioning, traces, eval datasets, LLM-as-judge workflows |
| Async ingestion | Kafka | Document-processing event queue |
| Cache | Redis | Session and LLM response cache |
| Cloud platform | Huawei Cloud | APIG, CCE/ECS, RDS, OBS, DCS, DMS, VPC, SWR |
Request Flow
The Router classifies intent, the Planner decomposes complex questions, the Executor calls retrieval and analysis tools, and the Synthesizer returns a cited answer after faithfulness and policy checks.
Retrieval Architecture
Hybrid retrieval is the default path. Semantic retrieval uses pgvector HNSW over chunk embeddings, lexical keyword retrieval uses PostgreSQL full-text search, and RRF fuses both rankings before the reranker re-scores top candidates.
Every final answer must cite the source chunks it uses. Unsupported claims are hallucinations and fail the faithfulness gate.
OCR and Ingestion Architecture
Documents enter through the ingestion pipeline, are stored in OBS, parsed by type, chunked, embedded, and indexed in PostgreSQL. Scanned or image-heavy documents route through OCR before indexing.
OCR confidence is mandatory metadata. Below-threshold extraction must route to human review rather than being silently accepted into the index.
Deployment Footprint
Production separates application, RAG data, model serving, cache, queue, and object storage concerns:
| Component | Production shape |
|---|---|
| FastAPI + orchestrator | 2 application pods, 4 vCPU / 16 GB each |
| LLM serving | A100 80 GB for Qwen3.5-27B |
| Embedding, reranker, OCR | T4 16 GB GPU node |
| RAG database | RDS PostgreSQL + pgvector, 8 vCPU / 32 GB / 1 TB SSD |
| Object storage | OBS buckets for raw documents, Langfuse payloads, and model weights |
| Cache and queue | DCS Redis and DMS Kafka |
Staging can collapse the full pipeline onto a smaller footprint, but still needs a real GPU to validate LLM inference, embeddings, reranking, and OCR.
Operational Guardrails
- No external SaaS or data egress for delivered inference and document handling.
- OSI-licensed open-source components only.
- Prompts live in Langfuse rather than being hardcoded.
- OCR model/version changes require revalidation because confidence scores can drift.
- Context-window overflow must be handled explicitly; silent truncation is not acceptable.
- Ingested documents are untrusted input, so prompt hygiene and the Government Policy Guard are mandatory.