Contextual AI & Medical Vision Pipelines
Suite of context-aware document intelligence pipelines: PDF intelligence (PyMuPDF + spaCy + BERT), passport MRZ segmentation, and prescription OCR.
- PDF entity F1
- 0.91
- MRZ accuracy
- 98.5%
- Rx field accuracy
- 94%
Problem & Solution
The Problem
Domain documents — medical PDFs, passports, prescriptions — each need specialized extraction. Generic OCR misses the structure, and context (e.g., MRZ check digits, dosage patterns) lives outside the pixel-level model.
The Solution
Built a suite of pipelines: PyMuPDF + spaCy + BERT for medical PDF intelligence (entities, sections, context), classical CV + OCR for passport MRZ segmentation and validation, and a fine-tuned OCR pipeline for prescription digitization with dosage parsing.
System Architecture
End-to-end flow from intake to outcome
- 01
PDF Intelligence
PyMuPDFspaCyBERTPyMuPDF extracts text and layout; spaCy + BERT detect medical entities and sections.
- 02
Passport MRZ
OpenCVPaddleOCRImage preprocessing, region localization, OCR, and check-digit validation.
- 03
Prescription OCR
PaddleOCRPythonFine-tuned OCR pipeline with dosage and frequency parsing.
- 04
API
FastAPIFastAPI exposes the pipelines as independent services.
Engineering Challenges & Wins
Optimization results and engineering tradeoffs
Optimization Results
- PDF intelligence: PyMuPDF extraction + spaCy + BERT for medical entity recognition.
- Passport MRZ segmentation and check-digit validation.
- Prescription OCR with dosage and frequency parsing.
Engineering Challenges
- Validating MRZ check digits against OCR output without silent errors.
- Disambiguating medical abbreviations with context-aware BERT models.
Related Projects
End-to-end cheque digitization pipeline using OpenCV for image preprocessing, PaddleOCR for text and amount extraction, and a Flask + Docker orchestration layer.
- Field accuracy
- 96.2%
- Throughput
- 120 img/s
- p95 latency
- 850ms
Mail-server gateway (Postfix Milter) that intercepts every inbound message, runs a fine-tuned BERT core (M-BSCE) through an asynchronous inference pipeline (RabbitMQ + Celery), and adds multimodal threat detection (PaddleOCR for Quishing) with SOAP clawbacks to the upstream mail system.
- Quishing detection
- 98%+
- p95 classification
- <400ms
- Async throughput
- 5k msg/min