OCR System
The OCR System converts scanned PDFs, images, handwriting-heavy pages, and structured Indonesian ID documents into machine-readable text before the text enters retrieval. It is self-hosted on Huawei Cloud and does not call external OCR APIs.
Responsibilities
- Accept OCR-bound documents from the same backend API and ingestion service as born-digital documents.
- Convert uploaded files into page images when required.
- Run OCR through the configured OCR endpoint.
- Preserve page numbers, block type, text spans, confidence, OCR model name, and confidence-comparability metadata.
- Gate low-confidence or non-comparable OCR output into a review state instead of embedding it as indexed content.
- Store OCR-derived text and metadata in PostgreSQL/pgvector and store raw/source files in OBS-compatible object storage.
- Audit OCR operations and review outcomes once the human-review workflow lands.
Upload entry points
Users do not upload directly to an OCR subsystem. They upload to backend ingestion surfaces, and the backend routes files to OCR when the parser detects an image/scanned path.
| Entry point | Use case | Behavior |
|---|---|---|
POST /documents | Manual multipart upload from a frontend, batch-upload tool, or API client | Stores files, creates a job, enqueues one ingestion message per file, exposes job/document status |
POST /documents/txt | Inline text ingestion for born-digital text | Bypasses OCR and indexes direct text |
POST /ingestion/batch | Bulk ingestion from object storage prefix or manifest | Enqueues one job over existing object keys; worker fetches bytes from storage |
| Source connectors | JDIH, OpenData, PPID, SatuData ingestion | Connectors call the same API/service layer rather than bypassing ingestion |
End-to-end OCR workflow
Processing pipeline details
1. Upload validation and storage
The upload path validates files before storage, rejects unsupported types, enforces file and batch size limits, and guards against archive-bomb inputs. For S3-compatible storage, the same seam works for local MinIO and Huawei OBS. Partial multipart upload failure triggers compensating object deletion so already-stored objects do not become orphaned storage.
2. Parse-first strategy
The ingestion service uses the born-digital parser first. Docling Slim is the selected boundary for PDF, DOCX, PPTX, XLSX, CSV, MD, and HTML; TXT is parsed locally; scanned images and OCR-bound content route to the OCR service.
3. OCR fallback trigger
OCR runs when a document is an image, when the parser cannot handle an image format, when a PDF/image parse is empty, or when parsed PDF/image text falls below the configured low-quality threshold. Non-image unsupported formats fail as non-indexable rather than being sent to OCR.
4. PDF and image OCR execution
PDF pages are rendered to PNG and processed page by page. Image uploads are processed as a single image. The OCR client receives base64 image payloads and returns text, mean confidence, model name, confidence comparability, and optional blocks.
5. OCR-derived chunk persistence
OCR text is normalized into parsed-document blocks. Chunks carry ocr_confidence, page_number, metadata.ocr_model, metadata.ocr_confidence_comparable_with_prod, and metadata.ocr_confidence_threshold. Chunks marked needs_ocr_review have no embedding and are excluded from semantic and lexical retrieval.
Models and technologies
| Component | Technology | Notes |
|---|---|---|
| OCR model | PaddleOCR-VL GPU profile | Model changes are versioned validation events because OCR confidence scores can shift between model versions. |
| PDF rendering | PDFium | Converts PDF pages to image bytes before OCR. |
| OCR wrapper | FastAPI OCR service behind configurable endpoint | Keeps app code environment-configured rather than hardwired. |
| Model serving | vLLM-backed GPU profile for OCR-VL where configured | GPU profile co-locates OCR with embedding and reranker on a T4-class or 24 GB GPU profile. |
| Storage | Huawei OBS / S3-compatible seam | Raw files and processed artifacts live in object storage. |
| Metadata store | PostgreSQL + pgvector | Stores document/chunk metadata, OCR confidence, statuses, embeddings, lexical index data, conversations, and audit logs. |
Confidence scoring
Each OCR result exposes a mean_confidence. The ingestion service compares chunk OCR confidence to ocr_confidence_threshold, defaulting to 0.85. A chunk is indexed only when the OCR confidence is comparable with production thresholds, is at or above threshold, and the text is not low-signal/gibberish.
The system treats confidence comparability as a separate safety property. Dev OCR and uncalibrated OCR-VL layout scores can produce text but still be marked confidence_scores_comparable_with_prod=false; those chunks route to review instead of being treated as production-quality OCR.
Error handling
| Failure | Handling |
|---|---|
| Unsupported non-OCR format | Request or document fails before indexing |
| Empty text after parse/OCR | Document fails as non-indexable |
| Archive-bomb or oversized input | Rejected before parsing/OCR exhausts resources |
| Object storage failure | Upload returns structured 503; partial uploaded objects are best-effort deleted |
| Low-confidence OCR | Chunk/document state becomes needs_ocr_review; no embedding is stored |
| Non-comparable confidence | Same review path as low confidence |
| Worker transient failure | Drain loop logs, backs off, and continues rather than dying silently |
Human verification and review workflow
The backend produces reviewable states: upload jobs can report needs_review, chunks can be needs_ocr_review, and these chunks are excluded from retrieval until approved.
Target review workflow:
Approval must preserve provenance. The corrected chunk retains the original OCR confidence and model metadata, adds review metadata, and writes an audit event with actor, action, entity, request, metadata, and timestamp details.
OCR architecture
The OCR engine layer is isolated from public clients. Public clients interact with upload/status/query APIs; the worker and OCR service communicate inside the backend/cloud network.