OCR System
The OCR System converts scanned PDFs, images, handwriting-heavy pages, and structured Indonesian ID documents into machine-readable text before the text enters retrieval. It is self-hosted on Huawei Cloud and does not call external OCR APIs.
:::info Authoritative API contract This page describes how OCR fits into ingestion. Public upload, batch, text-ingestion, document-status, and error contracts are defined in the Ingestion and Documents API reference. Do not treat the architecture tables below as complete request or response schemas. :::
Responsibilities
- Accept OCR-bound documents from the same backend API and ingestion service as born-digital documents.
- Convert uploaded files into page images when required.
- Run OCR through the configured OCR endpoint.
- Preserve page numbers, block type, text spans, confidence, OCR model name, and confidence-comparability metadata.
- Gate low-confidence or non-comparable OCR output into a review state instead of embedding it as indexed content.
- Store OCR-derived text and metadata in PostgreSQL/pgvector and store raw/source files in OBS-compatible object storage.
- Audit OCR operations and review outcomes once the human-review workflow lands.
Upload entry points
Users do not upload directly to an OCR subsystem. They upload to backend ingestion surfaces, and the backend routes files to OCR when the parser detects an image/scanned path. The table summarizes routing responsibility only; use the Ingestion and Documents API reference for fields, status codes, error bodies, curl commands, and TypeScript examples.
| Entry point | Use case | Architecture behavior |
|---|---|---|
POST /documents | Manual multipart upload from a frontend, batch-upload tool, or API client | Stores raw files through the ingestion boundary, tracks document/job state, and schedules one processing path per file |
POST /documents/txt | Inline text ingestion for born-digital text | Bypasses OCR and indexes direct text through the normal ingestion service |
POST /ocr | OCR-only extraction with no ingestion | Runs the same parse-first→OCR-fallback pipeline on an uploaded file and returns the extracted text, per-block confidence, model name, and confidence-comparability inline — nothing is stored, chunked, embedded, or indexed. Lets an operator inspect exactly what OCR produces for a document before deciding to ingest it. |
POST /ingestion/batch | Bulk ingestion from object storage prefix or manifest | Enqueues work over existing object keys; workers fetch bytes from storage |
| Source connectors | JDIH, OpenData, PPID, SatuData ingestion | Connectors call the same API/service layer rather than bypassing ingestion |
The POST /ocr path reuses the ingestion parse/OCR decision (parse-first, then render-and-OCR when the parser output is unusable) but stops at extraction — it is the read-only inspection surface behind the console OCR tab. Fields and error shapes are in the Ingestion and Documents API reference.
End-to-end OCR workflow
Processing pipeline details
1. Upload validation and storage
The upload path validates files before storage, rejects unsupported types, enforces file and batch size limits, and guards against archive-bomb inputs. For S3-compatible storage, the same seam works for local MinIO and Huawei OBS. Partial multipart upload failure triggers compensating object deletion so already-stored objects do not become orphaned storage.
2. Parse-first strategy
The ingestion service uses the born-digital parser first. Docling Slim is the selected boundary for PDF, DOCX, PPTX, XLSX, CSV, MD, and HTML; TXT is parsed locally; scanned images and OCR-bound content route to the OCR service.
3. OCR fallback trigger
OCR runs when a document is an image, when the parser cannot handle an image format, when a PDF/image parse is empty, or when parsed PDF/image text falls below the configured low-quality threshold. Non-image unsupported formats fail as non-indexable rather than being sent to OCR.
4. PDF and image OCR execution
PDF pages are rendered to PNG and processed page by page. Image uploads are processed as a single image. The OCR client receives base64 image payloads and returns text, mean confidence, model name, confidence comparability, and optional blocks.
5. OCR-derived chunk persistence
OCR text is normalized into parsed-document blocks. Chunks carry ocr_confidence, page_number, metadata.ocr_model, metadata.ocr_confidence_comparable_with_prod, and metadata.ocr_confidence_threshold. Chunks marked needs_ocr_review have no embedding and are excluded from semantic and lexical retrieval.
Models and technologies
| Component | Technology | Notes |
|---|---|---|
| OCR model | PaddleOCR-VL GPU profile | Model changes are versioned validation events because OCR confidence scores can shift between model versions. |
| PDF rendering | PDFium | Converts PDF pages to image bytes before OCR. |
| OCR wrapper | FastAPI OCR service behind configurable endpoint | Keeps app code environment-configured rather than hardwired. |
| Model serving | vLLM-backed GPU profile for OCR-VL where configured | GPU profile co-locates OCR with embedding and reranker on an A30 24 GB GPU profile. |
| Storage | Huawei OBS / S3-compatible seam | Raw files and processed artifacts live in object storage. |
| Metadata store | PostgreSQL + pgvector | Stores document/chunk metadata, OCR confidence, statuses, embeddings, lexical index data, conversations, and audit logs. |
Confidence scoring
Each OCR result exposes a mean_confidence. A chunk is indexed only when its gate confidence is at or above ocr_confidence_threshold, defaulting to 0.85; everything below routes to review.
The score fed to that gate must stay comparable across OCR model versions. When the backend's own confidence is not production-comparable — the pinned PaddleOCR-VL profile reports confidence_scores_comparable_with_prod=false and 0.0 block confidence for every page — the ingestion path derives the gate confidence from the OCR output text instead (src/aceh_rag/ocr_quality.py), and records the raw backend score as ocr_raw_mean_confidence for provenance. ADR-0005 owns the derived signal's axes, its calibration, and its known limits.
Error handling
This table describes architectural handling inside OCR and ingestion. Public HTTP status codes, error payloads, and retry-visible status transitions are part of the Ingestion and Documents API reference.
| Failure | Handling |
|---|---|
| Unsupported non-OCR format | Request or document fails before indexing |
| Empty text after parse/OCR | Document fails as non-indexable |
| Archive-bomb or oversized input | Rejected before parsing/OCR exhausts resources |
| Object storage failure | Upload failure is mapped through the ingestion API contract, and already-stored objects are deleted on a best-effort basis |
| Low-confidence OCR | Chunk/document state becomes needs_ocr_review; no embedding is stored |
| Non-comparable backend confidence | Gate confidence is derived from the OCR output text and judged against the same threshold (ADR-0005) |
| Worker transient failure | Drain loop logs, backs off, and continues rather than dying silently |
Human verification and review workflow
The backend produces reviewable states: upload jobs can report needs_review, chunks can be needs_ocr_review, and these chunks are excluded from retrieval until approved.
Target review workflow:
Approval must preserve provenance. The corrected chunk retains the original OCR confidence and model metadata, adds review metadata, and writes an audit event with actor, action, entity, request, metadata, and timestamp details.
OCR architecture
The OCR engine layer is isolated from public clients. Public clients interact with upload/status/query APIs; the worker and OCR service communicate inside the backend/cloud network.