Skip to main content

Retrieval and Answering

Retrieval combines semantic vector search with lexical keyword search, fuses both rankings with RRF, reranks candidates, and passes cited chunks to synthesis. Semantic-only retrieval is explicitly out because government legal and administrative documents depend on exact statute numbers, named entities, and official terms.

:::info Authoritative API contract This page explains retrieval and answer-generation architecture. For exact /search, /query, and /query/stream request fields, response bodies, citation shape, status/error behavior, SSE event order, curl commands, and TypeScript examples, use the Query and Search API reference. :::

Retrieval flow​

Citation model​

Each retrieved chunk carries citation metadata: document identity, document title, chunk identity and ordinal, page number, section path, legal reference, and character span. This architecture page names the provenance model only; the complete citation object and conditional access-control fields are defined in the Query and Search API reference.

Faithfulness gate​

The Synthesizer checks whether each generated claim is supported by cited chunks. Unsupported claims are dropped or the system returns a fallback answer that says it could not verify a faithful answer from retrieved sources.

The support check is figure-aware, not word-overlap only. An exact normalized numeric, currency, percent, or ratio value shared between a claim and a cited chunk grounds terse and cross-language figure answers — an English answer over an Indonesian chunk — that a word-overlap bar drops, because the figure is often the only token shared across languages. Values are normalized across Indonesian and English digit-grouping conventions before comparison, and a number introduced by a structural keyword (Pasal, Nomor, Tahun, ayat, Lampiran) is treated as a clause/statute reference rather than a groundable quantity, so a bare 2 never grounds on Pasal 2. Ratios ground only in the orientation written, and negation or polarity conflicts stay authoritative over any figure match. See src/aceh_rag/services.py _claim_supported_by_chunk/_figure_values.

Tool selection and configured sources​

Retrieval runs through a tool the Planner selects for the query's intent, with hybrid Vector Search as the default and fallback. Legal Check, Summarizer, Cross-Reference, and Excel Query use indexed or uploaded document data. The read-only sql_query tool registers only when a structured SQL source is enabled, so a quantity-shaped question over prose PPID content is answered from the retrieved chunk rather than routed to an unconfigured SQL source and refused. See src/aceh_rag/orchestration.py.

Live OpenData, SatuData, and BPS reads​

When enabled with an egress allow-list, the Open Data tool reads a published figure directly from query-specific CKAN OpenData or SatuData resources and grounds it with dataset identity and read-time metadata. The BPS Statistics tool reads the authoritative BPS WebAPI for indicators such as population, poverty, inflation, or GDP; it shares the open_data_lookup intent and can fall back to the Open Data portal when configured. These tools are disabled by default and must degrade to an honest unavailable/not-enabled note rather than invent a figure. See src/aceh_rag/open_data.py and src/aceh_rag/bps.py.

Policy Guard​

The Government Policy Guard wraps generated answers before they reach users. It filters anti-SARA, jailbreak attempts, and government-communication-policy violations. It cannot be bypassed per query.

Query and search surfaces​

POST /search exposes ranked retrieval chunks for diagnostics and result previews. POST /query runs retrieval plus synthesis and returns a cited answer, while POST /query/stream streams the same answer lifecycle over SSE. This section is an architecture summary; the Query and Search API reference is authoritative for payload fields, status codes, standard errors, SSE terminal behavior, and frontend examples.