Technical Architecture
Aceh Agentic RAG + OCR is a self-hosted backend platform for Pemerintah Aceh. It retrieves and answers questions in Bahasa Indonesia and English over government sources from JDIH, OpenData, PPID, SatuData and BPS — 44,443 catalogued, of which 35,271 are documents queued for ingestion — and is built to the TOR's requirement of indexing and searching 100,000+ documents. Data and inference stay inside Huawei Cloud.
System Goals
- Serve 10-50 internal concurrent users through HTTP JSON APIs and Server-Sent Events streaming.
- Support Agentic RAG over PDFs, Word files, spreadsheets, scanned images, TXT, and related government document formats.
- Keep all data sovereignty requirements intact by using self-hosted components only.
- Attach source citations to every generated answer and run a faithfulness check before returning it.
- Apply the Government Policy Guard on every output for anti-SARA, anti-jailbreak, and government communication-policy filtering.
Core Stack
| Layer | Technology | Purpose |
|---|---|---|
| API | FastAPI | Async HTTP JSON API, SSE query streaming, auth, rate limiting, sessions, validation |
| Model client | LiteLLM | OpenAI-compatible model calls, retries, fallback routing, tracing hooks |
| Orchestration | Custom Python + Pydantic | Router, Planner, Executor, Synthesizer pipeline |
| Model serving | vLLM | Self-hosted LLM, embedding, and reranker endpoints |
| Retrieval store | PostgreSQL + pgvector + full-text search | HNSW semantic retrieval plus lexical keyword retrieval |
| Prompt and observability | Langfuse | Prompt versioning, traces, eval datasets, LLM-as-judge workflows |
| Async ingestion | Kafka | Document-processing event queue |
| Cache | Redis | Session and LLM response cache |
| Cloud platform | Huawei Cloud | APIG, CCE/ECS, RDS, OBS, DCS, DMS, VPC, SWR |
Request Flow
The Router classifies intent, the Planner decomposes complex questions, the Executor calls retrieval and analysis tools, and the Synthesizer returns a cited answer after faithfulness and policy checks.
Retrieval Architecture
Hybrid retrieval is the default path. Semantic retrieval uses pgvector HNSW over chunk embeddings, lexical keyword retrieval uses PostgreSQL full-text search, and RRF fuses both rankings before the reranker re-scores top candidates.
Every final answer must cite the source chunks it uses. Unsupported claims are hallucinations and fail the faithfulness gate.
OCR and Ingestion Architecture
Documents enter through the ingestion pipeline, are stored in OBS, parsed by type, chunked, embedded, and indexed in PostgreSQL. Scanned or image-heavy documents route through OCR before indexing.
OCR confidence is mandatory metadata. Below-threshold extraction must route to human review rather than being silently accepted into the index.
Deployment Footprint
Production separates application, RAG data, model serving, cache, queue, and object storage concerns:
| Component | Production shape |
|---|---|
| FastAPI + orchestrator | 2 application pods, 4 vCPU / 16 GB each |
| LLM serving | Qwen3.8-27B FP16 on 2x NVIDIA A100 40 GB, tensor-parallel (TP=2) |
| Embedding, reranker, OCR | A30 24 GB GPU node |
| RAG database | RDS PostgreSQL + pgvector, 8 vCPU / 32 GB / 1 TB SSD |
| Object storage | OBS buckets for raw documents, Langfuse payloads, and model weights |
| Cache and queue | DCS Redis and DMS Kafka |
The production LLM pair splits ~54 GB of FP16 weights at ~27 GB per card, providing 80 GB aggregate VRAM and roughly ~22 GB aggregate for KV cache. Huawei's published P3/P3snl A100-40GB line removes the former 80 GB-card availability risk. Residual risk is quota and lead time for 2x A100 40GB in ap-southeast-4 (Jakarta) and confirmation of an NVLink-connected pair for tensor parallelism; PCIe-only pairing adds token-latency overhead. Staging collapses the full pipeline onto one A30 24 GB GPU node for sample data, but still needs a dedicated GPU to validate LLM inference, embeddings, reranking, and OCR.
Operational Guardrails
- No external SaaS or data egress for delivered inference and document handling.
- OSI-licensed open-source components only.
- Prompts live in Langfuse rather than being hardcoded.
- OCR model/version changes require revalidation because confidence scores can drift.
- Context-window overflow must be handled explicitly; silent truncation is not acceptable.
- Ingested documents are untrusted input, so prompt hygiene and the Government Policy Guard are mandatory.