Query v2: annotated faithfulness

:::info Live in production POST /query/v2 and POST /query/v2/stream are live in production beside /query since 23 Oct 2026, and /query/v2 is the route to build on going forward. They take the same authentication as /query: the X-API-Key header and the partner-signed JWT in Authorization: Bearer. /query is unchanged. Design: ADR-0051 and ADR-0052. :::

/query withholds any sentence its faithfulness judge does not confirm, and refuses the whole answer when too little survives. /query/v2 runs the same agent with the same tools, scope, history and budget, and the same prompts except one: it asks the agent to write its answer as segments. After the agent finishes, v2 annotates instead of withholding:

1. Citations. The agent returns an ordered list of segments, each with its text, its kind (fact or framing), the passages it cites, and a layout (paragraph, heading, list_item, continue, table_header, table_row). The server renders answer from them as Markdown, so every segment's offsets are exact. A fact segment that cites nothing is marked uncited. Every framing segment (a lead-in such as "Berikut rinciannya:", a greeting, "not found in the documents") is checked in one model call per answer that sees the question, with no word or digit rule in front of it; framing that states a fact needing a citation, or answers the question, is marked uncited, and so is every framing segment when that call fails. A framing segment that cites a passage is checked as a fact against it.
2. Faithfulness. A separate judge checks each cited segment against the passages it cites, shown with the same document identity (legal reference, section, number, year, instrument type, issuer) the agent saw, or for a portal or MCP result its dataset, source, period and the call that produced it. It also sees what the reader sees around the segment (its heading, its list's lead-in, its table header; a table row is checked whole), and never the question. The judge is strict: a segment is supported only when its figure has the same scale (ribu, juta, miliar, triliun), unit, period, entity and condition as the source; the same value written differently (871.667.219.294 and Rp871,67 miliar) is still supported. The result is a mark on the segment; the segment is never removed.
3. Government Policy Guard. It reads the full answer, as on /query. It is the only check that can replace the answer.

The reader verifies a marked segment by opening its citation. A client must render the marks: an unsupported segment shown without its mark looks verified. The rules a client must follow are in Client contract.

Request

The body is /query's body (see Query, Streaming Query, and Search APIs) with one difference: offer_artifacts accepts only false, because /query/v2 does not generate files. include_grounding and include_provenance are JSON-only, as on /query.

Every caller gets the annotations; no opt-in is needed. accept_annotations is accepted and ignored: it was a required opt-in from 21 to 23 Oct 2026, removed by product decision, and stays a known field so clients that still send it keep working. Use /query if the client cannot render the annotations.

curl -sS "$ACEH_API/query/v2" \
  -H "X-API-Key: $ACEH_API_KEY" -H "Authorization: Bearer $PARTNER_JWT" \
  -H "Content-Type: application/json" \
  -d '{"query": "Berapa anggaran pendidikan Aceh tahun 2023?"}'

Prompt playground

POST /playground lets a verified administrator try one agent-editorial prompt without publishing it. The body accepts the same fields as /query/v2 and adds the required editorial_prompt string (1–2,000 characters):

curl -sS "$ACEH_API/playground" \
  -H "X-API-Key: $ACEH_API_KEY" -H "Authorization: Bearer $ADMIN_JWT" \
  -H "Content-Type: application/json" \
  -d '{
    "editorial_prompt": "Answer clearly and concisely in Bahasa Indonesia.",
    "query": "Apa tugas utama Pemerintah Aceh dalam dokumen ini?",
    "document_ids": ["<document UUID>"]
  }'

The route executes the same query-v2 agent, retrieval, tools, access checks, budgets, citations, annotate-v2 faithfulness checks, and Government Policy Guard. Only the editorial prompt is replaced, for that request only. It requires both the shared API key and a verified admin principal; it fails closed with 503 when verified identity is disabled and is rate-limited per administrator.

The response keeps the query-v2 fields that still apply and adds playground_run_id, published: false, and latency_ms. It deliberately omits conversation_id and message_id: no conversation, message, memory, feedback, prompt version, prompt label, or normal answer analytics is published. The server stores only a small administrative receipt containing identifiers, status, counts, and timing—never the candidate prompt, question, answer, or source-passage text. A successful run does not change the prompt used by later /query or /query/v2 requests.

Response

Every /query field that keeps its meaning is kept: answer, conversation_id, message_id, trace_id, history_truncated, history_exchanges_in_view, citations, tool, tool_executed, fallback_used, fallback_reason, and the opt-in grounding, used_docs, tool_call_audit, model_call_log. Removed, because they describe a gate v2 does not have: claim_map, dropped_claims_count, dropped_claim_reasons, provenance_complete. Added: schema (always "annotate-v2", the contract this body follows), claims, framing and faithfulness. Kept with v1's meaning but counted from v2's verdicts: claims_supported, claims_total and grounding_rate, the claim grounding rate (Tingkat keterverifikasian klaim), so one badge renders from either route.

On v2, claims_supported counts supported claims and claims_total counts every claim except from_conversation, which restates the conversation and is checked against no source: an unsupported, unverified, uncited or invalid_citation claim lowers the rate. grounding_rate is null when that total is 0, as on a refusal before any claim was checked. A Policy Guard block publishes no claims, but, as on v1, the rate still counts the claims checked before the block (counts only; no claim text). The same three keys are faithfulness.grounding (on the SSE annotations frame too, and stored in v2_faithfulness). It is not the faithfulness figure (ADR-0042); faithfulness.verification remains the answer-level banner state.

{
  "schema": "annotate-v2",
  "answer": "Berikut datanya. Anggaran pendidikan 2023 adalah Rp 5 miliar. Naik 9 persen dari 2022.",
  "citations": [{"document_title": "Qanun Anggaran", "chunk_id": "…", "document_id": "…"}],
  "fallback_used": false,
  "fallback_reason": null,
  "claims": [
    {"text": "Anggaran pendidikan 2023 adalah Rp 5 miliar.", "citation_keys": ["P1"],
     "citations": [{"citation_key": "P1", "citation": {"chunk_id": "…"}}],
     "verdict": "supported", "unverified_reason": null, "invalid_citation_keys": [],
     "kind": "fact", "layout": "continue",
     "start": 17, "end": 61, "start_utf16": 17, "end_utf16": 61},
    {"text": "Naik 9 persen dari 2022.", "citation_keys": ["P1"],
     "citations": [{"citation_key": "P1", "citation": {"chunk_id": "…"}}],
     "verdict": "unsupported", "unverified_reason": null, "invalid_citation_keys": [],
     "kind": "fact", "layout": "continue",
     "start": 62, "end": 86, "start_utf16": 62, "end_utf16": 86}
  ],
  "framing": [
    {"text": "Berikut datanya.", "layout": "paragraph",
     "start": 0, "end": 16, "start_utf16": 0, "end_utf16": 16}
  ],
  "faithfulness": {"mechanism": "annotate-v2", "judge_status": "complete",
    "framing_check": "complete", "claims_total": 2,
    "supported": 1, "unsupported": 1, "unverified": 0, "uncited": 0,
    "invalid_citation": 0, "from_conversation": 0,
    "unverified_reasons": {"deadline": 0, "budget": 0, "judge_unavailable": 0, "unparseable": 0,
                           "disputed": 0},
    "framing_total": 1, "unlocated_total": 0, "verification": "partial",
    "grounding": {"claims_supported": 1, "claims_total": 2, "grounding_rate": 0.5}},
  "claims_supported": 1,
  "claims_total": 2,
  "grounding_rate": 0.5
}

verdict	Meaning	Required rendering
supported	The judge found the cited passage supports the segment.	The only positive mark: "terverifikasi", with its citation.
unsupported	The judge found the cited passage does not support it, and a recheck of that segment alone confirmed it (an unconfirmed rejection is unverified with reason disputed).	"Belum diverifikasi", for example "tidak didukung sumber"; its citation chip must not read as proof.
unverified	The judge could not decide; unverified_reason says why. Never a pass.	"Belum diverifikasi".
uncited	States a fact but cites nothing (including framing the framing check found checkable).	"Belum diverifikasi", for example "tanpa sumber".
invalid_citation	Every passage it cites is one the agent was not shown.	"Belum diverifikasi"; the citation does not resolve.
from_conversation	Restates this conversation (what the user said, or an earlier verified answer).	Its own neutral mark, "dari percakapan, bukan dokumen". Never styled as supported: a user's own premise echoed back is not confirmed by any document.

Segments that assert nothing (a greeting, "Berikut ringkasannya:", a table's header row) are not claims: they are returned in framing, each with its text, layout and the same four offsets, and carry no verdict. Claims and framing together cover every character of answer except the layout the server writes between segments (whitespace, the ## and - markers, the table pipes and delimiter row); text outside every span would be counted in unlocated_total. answer contains no [P1] labels: a client places a claim's citation chips at its end, from its citation_keys. Every offset is a position in the rendered answer, end exclusive, and answer[start:end] is exactly the span's text, Markdown emphasis included:

Field	Meaning
start, end	Offsets in Unicode code points (Python string indices).
start_utf16, end_utf16	The same span in UTF-16 code units: use these as JavaScript string indices (answer.slice(start_utf16, end_utf16)). They differ from start/end after an emoji or any character outside the Basic Multilingual Plane.
kind	The agent's own fact or framing. A framing claim is framing the framing check found checkable, or framing that cites a passage.
layout	Where the segment sits: paragraph, heading, list_item, continue (same paragraph or item), table_header, table_row (cells separated by |).

faithfulness.verification is the answer-level state, computed by the server:

verification	When
full	At least one claim, every claim supported, and unlocated_total is 0.
partial	At least one claim supported, and anything else: a claim with another verdict (including from_conversation) or unlocated text.
none	No claim supported, which includes a refusal, a greeting with nothing to check, and a turn where nothing was checked.

framing_total counts the framing spans. unlocated_total counts the runs of answer text no claim or framing span covers; a refusal or notice, which carries no annotations, is one such run.

judge_status is complete, unavailable (at least one cited segment is unverified) or skipped (no cited segment). framing_check is the same for the one framing check: unavailable means every segment it had to check is shown uncited, never passed as framing.

unverified_reason is null except on an unverified claim, where it is one of deadline (the request's time ran out before the claim's judge batch returned; batches that had returned keep their verdicts), budget (the request's model calls were spent), judge_unavailable (the judge failed or timed out), unparseable (the judge gave no usable verdict for it) or disputed (the batched judge found the segment unsupported, but the recheck of that segment alone did not confirm it, so it is shown neither as supported nor as unsupported). A failed or timed-out recheck gives its own reason (deadline, budget, …). faithfulness.unverified_reasons counts them. The agent stops early enough to leave the judge its own share of the request's time and model calls.

invalid_citation_keys lists the passages a segment cites that the agent was not shown, or may not cite. A segment with at least one valid citation is judged against its valid ones and keeps that verdict; only a segment whose citations are all invalid is invalid_citation.

fallback_used says whether the API published a refusal, never whether it is trustworthy: false with verification: "none" is a published answer nothing verified. It is true for a refusal: policy_blocked (the Policy Guard), or one of /query's loop-level reasons (query_budget_exhausted, model_timeout, context_budget_exhausted, agent_output_limit (on /query/v2 only when a follow-up's re-asked forced tool round reaches its output ceiling; the could-not-answer-in-time notice is published), mcp_source_unavailable, source_out_of_scope, source_scope_unavailable, unparseable_completion). source_scope_empty is a pre-loop refusal described under Errors. A refusal has no claims. unparseable_completion is also the outcome when the agent's segments are still malformed after one regeneration; malformed output is never published.

context_budget_exhausted means the source material the question needed could not fit the model's context, so the answer was stopped before it was written and nothing was checked. Its faithfulness report carries "checked": false, on the JSON response and the SSE annotations frame alike: render it as "not checked", never as a failed or passed check. Every other response omits checked; do not read its absence as a pass. The notice text is fixed system copy, not a timeout message.

Errors

The same status codes as /query for the same causes (400, 403, 404, 413, 502; 401 for a missing key; 422 for a malformed body), plus 409: a conversation belongs to the route that first answered in it. Continuing a /query conversation on /query/v2, or the reverse, returns 409. A conversation created empty with POST /conversations is claimed by whichever route answers first. Changing a conversation's sources or MCP selection does not release it.

Two causes differ: when the evidence the agent gathered no longer fits the model's context window, /query/v2 answers 200 with fallback_reason: "context_budget_exhausted" where /query answers 413. The same refusal answers when tools ran but none of their results fitted the context. A named selection with no indexed document answers 200 with fallback_reason: "source_scope_empty"; the API publishes fixed Indonesian refusal text before any tool runs or model call, including intent and trend classification, even on resumed recall questions. A selection the caller cannot access also has zero indexed documents and receives this refusal. A 404 from /query/v2 means the tools that ran returned nothing readable, which includes a search that failed or was stopped at the tool deadline. A 413 from /query/v2 carries fixed text, never the model server's error.

Streaming

POST /query/v2/stream sends the annotations before the answer text: tool.started / tool.finished while the agent works, then message.start, trace, citation, then either guard.blocked or passage, annotations, answer.delta, then done. The answer is buffered and guarded before the first frame, so every verdict is final when the text starts.

event: message.start
data: {"conversation_id": "…", "message_id": "…", "schema": "annotate-v2"}

event: citation
data: {"chunk_id": "…", "document_title": "Qanun Anggaran", …, "citation_key": "P1"}

event: passage
data: {"kind": "chunk", "chunk_id": "…", "citation_key": "P1", "text": "…", …}

event: annotations
data: {"mechanism": "annotate-v2", "claims_total": 2, "supported": 1, …,
       "framing_total": 1, "unlocated_total": 0, "verification": "partial",
       "grounding": {"claims_supported": 1, "claims_total": 2, "grounding_rate": 0.5},
       "fallback_used": false, "fallback_reason": null,
       "claims": [{"start": 17, "end": 61, "start_utf16": 17, "end_utf16": 61,
                   "kind": "fact", "layout": "continue", "citation_keys": ["P1"],
                   "verdict": "supported", "unverified_reason": null,
                   "invalid_citation_keys": []}, …],
       "framing": [{"start": 0, "end": 16, "start_utf16": 0, "end_utf16": 16,
                    "layout": "paragraph"}]}

event: answer.delta
data: {"text": "Berikut datanya. Anggaran …"}

event: done
data: {"finish_reason": "stop", "history_truncated": false}

The annotations frame carries the faithfulness counts (including verification) at its top level, fallback_used and fallback_reason, and per claim and per framing span the offsets above, without the text: the answer.delta frames concatenate to exactly answer, so the offsets apply to the streamed text. Every citation frame and every chunk passage frame carries the citation_key it was shown under (a tool passage already did), so each of a claim's citation_keys joins to the source the reader opens to verify it; a key listed in invalid_citation_keys joins to nothing. The event is not /query/stream's faithfulness: that frame counts removed sentences, and a v1 client reading v2's counts under that name would see "nothing dropped". A fault after annotations ends the stream with an error frame; the marks for any text already received have arrived.

Client contract

A client of /query/v2 must follow these rules. They make "verified" the positive state, so no failure (a dropped frame, an unplaced span, a client that ignores a field) can look like a verified answer.

1. Check the contract. Read schema (JSON body, or message.start on the stream) and treat anything other than "annotate-v2" as unverified.
2. Mark verified text positively. Only a supported claim is shown as verified ("terverifikasi"). Every other verdict, and any answer text outside every claim and framing span, is shown as "belum diverifikasi". Framing is not verified either; it needs no mark of its own, but it must not look like the verified mark.
3. from_conversation is neutral. Show it with its own mark, "dari percakapan, bukan dokumen", never styled as supported.
4. Banner. Whenever verification is not "full", show an answer-level notice that stays visible with the answer, saying it is not fully verified.
5. No annotations, no trust. On the stream, an answer whose annotations frame was not received is rendered as "belum diverifikasi" in full, with the banner.
6. fallback_used is not trust. It signals a refusal; an answer without a fallback is only as verified as verification says.

Conversations

Later turns see only what was verified: the history given to the model, the follow-up passages it re-reads, and long-term memory extraction use only supported (and framing or conversation) segments of an earlier /query/v2 answer. An earlier answer with no verified statement is the one exception, for the model's history only: it is quoted whole after a note saying none of it was verified and it is not evidence, so the model knows what the reader saw instead of reading an empty answer. A follow-up reads whole named documents only when it resolves to exactly one earlier document (or one document is selected); when the earlier answer cited several, the agent's searches favour those documents instead (up to six, most recent first, and only documents a supported segment cited); the search text stays the agent's own, with no titles added. The reader's own view of the conversation is the full stored answer.