Deployment and Operations
The target deployment runs on Huawei Cloud with FastAPI application nodes, GPU inference nodes, PostgreSQL/pgvector, OBS, Redis, Kafka, Langfuse, APIG, and network isolation.
Production footprint
| Component | Production shape |
|---|---|
| Application | 2 ECS/CCE app nodes or pods, 4 vCPU / 16 GB each |
| LLM GPU | A100 80 GB for Qwen3.5-27B target |
| Embed/rerank/OCR GPU | T4 16 GB for embedding, reranker, and OCR |
| Database | RDS PostgreSQL + pgvector, 8 vCPU / 32 GB / 1 TB SSD + backup |
| Object storage | OBS, about 1.2 TB target corpus storage |
| Cache | DCS Redis HA |
| Queue | DMS Kafka |
| Edge/network | APIG, ELB, NAT, VPC, security groups, SWR |
Staging footprint
Staging proves the full LLM to embedding to vector search to rerank to OCR path on the fewest nodes possible. It is not sized for concurrent production load. Preferred staging uses one A100 40 GB; fallback uses two T4 16 GB nodes.
Operational gotchas
- Bad OCR text must be gated before embedding to avoid retrieval poisoning.
- OCR model-version drift makes confidence scores incomparable unless versioned and recalibrated.
- Concurrent vector writes need serialization or transactional safety.
- Context-window overflow must fail or trim explicitly, never silently truncate.
- Documents are untrusted input, so prompt hygiene and Policy Guard are mandatory.