Intelligent Document Processing for Insurance: Hybrid OCR, a Local LLM, and Vector Search at 91.3% Accuracy

Insurance Runs on Paper — Even When the Paper Is a PDF

Every insurance policy, endorsement, premium receipt, and claim file has to be read by someone, identified, transcribed into two or three internal systems, and filed somewhere it can be found again eighteen months later. When the document is a phone photo of a national ID card, written in a mix of French and Arabic, that work is slow, inconsistent, and expensive.

I spent sixteen weeks building a system that does it automatically. This is the full engineering write-up: the architecture, the measured numbers, the decisions I would defend, and the ones I would revisit.

Metric	Result	What it replaces
OCR accuracy (weighted)	91.3%	85.9% Tesseract alone / 85.1% EasyOCR alone
Classification F1	93.0% across 9 categories	Manual visual sorting
Entity extraction F1	90.4% across 10 entity types	Manual data entry
Search precision@1	92.1%	Exact-keyword lookup
Pipeline latency (median)	6.8 s/document	15-20 minutes manual
Search latency (median)	0.3 s	Up to 48 h to find an archived contract
Usability (SUS)	78.5 / 100	—
Deployment	100% on-premise	SaaS quoted above 100K MAD/month

The stack in one line: Tesseract + EasyOCR → Phi-3 via Ollama → Sentence Transformers → ChromaDB → MinIO, behind a Flask API.

The Problem: Where Document Handling Breaks at Scale

Morocco's insurance sector is growing — 18.2 billion dirhams in revenue as of 31 March 2025, up 5.5% year-on-year according to ACAPS, the national regulator. More premiums means more paperwork, and the paperwork was already the bottleneck. Interviews with three major insurers produced a consistent picture:

• ~5,000 documents per day arriving at a single company
• 15-20 minutes of agent time per document, end to end
• 68% of agent hours consumed by manual data entry
• 12% error rate on manual keying
• 30% of claims delayed downstream as a consequence
• Up to 48 hours to locate a historical contract in the archive

Throughput ceiling. An experienced agent processes 15-25 documents per hour. That is a hard wall during fiscal year-end or a commercial campaign.

Quality variance. One agent types 15/03/2024, another types 15 mars 2024. The database now holds two formats for one concept, which poisons every downstream automated process and every statistical report.

Retrieval failure. Search is limited to exact fields. When a customer calls and says "my car policy, the one I took out last year after I moved to Rabat", the agent runs four queries across three systems and cross-references by hand.

Why Off-the-Shelf Tools Did Not Fit

Three constraints eliminated the obvious commercial options.

• Language. Documents mix French and Arabic, occasionally with Darija annotations. Standard tooling measured below 65% accuracy on Moroccan national ID cards (CIN).
• Sovereignty. Sending scans of identity documents to a third-party cloud API is a compliance conversation nobody wanted. Local processing was the entry ticket, not a preference.
• Cost. SaaS document-AI platforms were quoted above 100,000 MAD per month. For a mid-size insurer the automation savings evaporate into the licence fee.

System Architecture: Six Modules, One Pipeline

Upload → Type detection → Preprocessing → Dual OCR → Fusion
   → Hybrid classification → NER extraction → Validation
   → Chunking → Embeddings → ChromaDB indexing → MinIO archival

Component	Technology	Rationale
Backend	Python + Flask	Lightweight, native to the ML ecosystem
OCR engine 1	Tesseract 5.3	Fast, excellent on structured French PDFs
OCR engine 2	EasyOCR 1.7	CNN-based, strong Arabic, robust on poor scans
Language model	Phi-3 (3.8B) via Ollama	Best accuracy-per-parameter; runs on consumer GPU
Embeddings	all-MiniLM-L6-v2	384-dim, fast, industry standard
Vector store	ChromaDB	HNSW indexing, native Python, no licence cost
Object storage	MinIO	S3-compatible, self-hosted, AES-256 at rest

Communication uses HTTP/REST with JSON, direct Python calls inside the processing chain, and the S3 API for storage — deliberately boring protocols, because the interesting complexity belongs in the models, not the plumbing.

1. The Hybrid OCR Engine

This is the core contribution, and the insight is simple: the two engines fail in different places.

Tesseract 5.3 excels on clean structured layouts — contracts, certificates, anything with a predictable typographic grid. Fast (4.8 s/page), light on resources. It struggles with Arabic and handwriting, and degrades sharply as image quality drops.

EasyOCR 1.7 is CNN-based and handles French and Arabic simultaneously. Markedly better on identity documents, far more robust on low-quality scans. Slower (7.2 s/page), hungrier for memory.

On Moroccan CIN cards the gap is stark: EasyOCR 89.2% vs Tesseract 78.5%. On structured contracts it inverts: Tesseract 94.1% vs EasyOCR 87.8%. So the system runs both in parallel and picks the winner per document.

Adaptive Preprocessing Comes First

The right preprocessing depends on what is actually wrong with the scan:

Scan quality	Typical source	Baseline	Treatment
High (>300 DPI)	Recent scanner output	95%	Light CLAHE contrast enhancement
Medium (150-300 DPI)	Photocopies	85%	Denoising + Otsu binarisation
Low (<150 DPI)	Phone photos	62%	Text-region detection + perspective correction

Adaptive preprocessing alone reduced OCR errors by 23% before any fusion logic ran.

The Fusion Decision

Score = 0.4 × TextLength + 0.4 × Confidence + 0.2 × ContextualValidation

The reasoning behind the weights: quantity of extracted information and intrinsic reliability are the two dominant signals, so together they carry 80%. Contextual validation — does the text actually contain the business patterns you would expect in an insurance document? — acts as a 20% guardrail against a result that looks statistically fine but is semantically nonsense. In the real implementation scores are normalised before comparison so both engines are judged on the same scale.

Content analysis feeds this: the system measures the ratio of Arabic to Latin characters and biases accordingly. Heavy Arabic favours EasyOCR. Clean structured French favours Tesseract. Poor image quality favours EasyOCR's robustness.

Tesseract Configuration That Mattered

Parameter	Value	Why
PSM (Page Segmentation Mode)	6	Uniform text block — matches insurance forms
OEM (OCR Engine Mode)	1	LSTM engine only
Languages	fra+ara+eng	Multilingual support
Processing DPI	300	Optimal accuracy/speed tradeoff
preserve_interword_spaces	1	Preserves spacing needed for field parsing

EasyOCR ran two specialised readers — Latin (en, fr) and Arabic (ar, en) — with a confidence threshold of 0.5 tuned empirically and a 1920×1920 maximum image size.

OCR Results

Document type	Tesseract	EasyOCR	Hybrid	Time (s)
Contracts (PDF)	91.2%	87.8%	94.1%	4.2
Endorsements	86.7%	84.2%	90.5%	6.1
Certificates	89.3%	83.1%	91.7%	5.3
Claims	84.2%	81.9%	89.1%	7.2
Premium receipts	88.9%	85.4%	92.8%	4.8
Cancellations	87.1%	83.7%	90.9%	5.9
Supporting docs	85.4%	82.3%	89.7%	6.4
Scanned CIN	78.5%	89.2%	92.3%	8.1
Scanned licences	82.1%	85.7%	88.9%	7.9
Weighted average	85.9%	85.1%	91.3%	6.2

The most interesting result is not in that table. It is this one:

Source quality	Tesseract	EasyOCR	Hybrid	Gain
Excellent	94.1%	92.8%	96.7%	+2.6%
Good	89.2%	87.5%	93.1%	+3.9%
Medium	81.7%	83.2%	88.9%	+7.2%
Poor	72.3%	76.1%	81.8%	+9.5%

The fusion helps most exactly where it is needed most. On pristine scans either engine works and the hybrid adds 2.6 points. On the degraded phone photos that make up a real share of insurance intake, it adds 9.5. That asymmetry is the argument for the entire approach.

2. Hybrid Classification: Rules First, Model Second

Stage 1 — Business Rules

Document type	Typical keywords	Distinctive pattern
Insurance contract	souscripteur, prime, garanties	"police d'assurance n°"
National ID card	nationalité, né le, domicilié	"carte nationale d'identité"
Driving licence	catégories, délivré le	"permis de conduire"
Certificate	valide jusqu'au, certificat	"période de validité"

Rules resolve roughly 80% of documents instantly, with no model inference at all.

Stage 2 — LLM Adjudication

When rule confidence falls below 0.8, Phi-3 is called with a constrained prompt:

Analyse this Moroccan insurance document text and classify it as one of:
Contract, Endorsement, Certificate, Claim, Receipt, Cancellation, CIN, Licence.

Text: [document extract]

Respond with the document type only.

• Speed — 80% of traffic never touches the model
• Precision — the LLM handles genuinely ambiguous cases instead of every case
• Control — a business analyst can adjust a rule without retraining anything
• Auditability — for rule-classified documents you can point at the exact matched pattern, which matters in a regulated industry

Classification Results

Type	Precision	Recall	F1	Support
Contract	94.0%	92.1%	93.0%	85
Endorsement	89.0%	88.2%	88.6%	76
Certificate	96.0%	95.8%	95.9%	95
Claim	88.0%	86.9%	87.4%	78
Receipt	93.0%	92.1%	92.5%	89
Cancellation	91.0%	89.6%	90.3%	67
Supporting doc	98.0%	96.7%	97.3%	60
CIN	97.0%	96.2%	96.6%	105
Driving licence	98.0%	97.1%	97.5%	105
Weighted average	93.2%	92.8%	93.0%	760

The errors concentrate exactly where you would expect: endorsements and claims, which share vocabulary with contracts and with each other. An endorsement is a modification to a contract, so it legitimately contains most of a contract's language. Identity documents, with rigid standardised layouts, hit 96-98%.

3. Named Entity Extraction with a Local LLM

Why Phi-3 and Not Something Bigger

Model	Size	NER precision	Latency
BERT-base	110M	86%	320 ms
Llama-2-7B	7B	89%	2,100 ms
Phi-3	3.8B	92%	950 ms

BERT is fast but not accurate enough for production. Llama-2-7B is accurate but the latency is brutal at volume. Phi-3 delivered the highest NER precision in this benchmark at roughly half the parameters of Llama-2 and under half the latency.

The broader principle I would defend: in production systems, "good enough + deployable + fast" beats "optimal + expensive + slow." A model that runs on an RTX 3060 in the client's own server room is worth more than a marginally better model that requires cloud inference and a compliance review.

SYSTEM:   Expert in information extraction from insurance documents
CONTEXT:  Moroccan insurance document, French/Arabic
TASK:     Extract {specific_entities}
FORMAT:   JSON with fields {field_list}
EXAMPLES: {few_shot_examples}
INPUT:    {document_text}

Structured output plus few-shot examples plus a fixed schema — the combination that turns a chatty model into a parser.

Hyperparameters and Their Justification

Parameter	Value	Effect
Temperature	0.1	Near-deterministic; same document produces the same output
Top-p	0.9	Enough flexibility for linguistic variation, still coherent
Max tokens	500	Forces concision; blocks rambling non-answers
Repeat penalty	1.1	Prevents duplicated entity listings
Context length	4096	Fits most complete insurance documents

Temperature at 0.1 is the single most important setting. Extraction is not a creative task — reproducibility matters more than fluency, and an auditor will eventually ask why the same document produced two different answers.

NER Results by Entity Type

Entity	Detection	Precision	Recall	F1	Support
CIN number	98.1%	97.2%	96.8%	97.0%	410
Licence number	95.4%	93.1%	94.7%	93.9%	205
Contract number	96.2%	94.1%	92.8%	93.4%	350
Date of birth	94.1%	92.4%	93.8%	93.1%	410
Client surname	89.1%	92.3%	87.5%	89.8%	760
Dates (general)	91.4%	88.2%	89.7%	88.9%	1,240
Client first name	87.8%	91.7%	85.2%	88.3%	760
Premium amount	87.3%	90.1%	84.9%	87.4%	420
Address	83.2%	85.1%	81.3%	83.2%	760
Place of birth	81.7%	84.2%	79.3%	81.7%	410
Weighted average	91.8%	91.2%	89.6%	90.4%	5,515

Structured identifiers extract almost perfectly — free-text fields do not. CIN numbers follow a rigid format the model can anchor on. Addresses are unbounded natural language with inconsistent abbreviations, arbitrary line breaks, and no canonical form.

Error source	Share	Example
Segmentation	34%	Addresses split across lines, compound surnames truncated
Formatting	28%	Ambiguous dates (03/04 — March or April?), malformed numbers
Contextual ambiguity	23%	Agent name confused with client name
Propagated OCR errors	15%	Character-level misreads flowing downstream

Note the last row: only 15% of extraction failures originate in OCR. The majority are genuine NLP problems — which tells me the next round of improvement belongs in preprocessing and context modelling, not in a better OCR engine.

4. Embeddings and Chunking

all-MiniLM-L6-v2 from Sentence Transformers:

• 384 dimensions — dense enough to be informative, small enough to be fast
• 512 token maximum per input
• Mean pooling across non-masked tokens
• L2 normalisation, which makes cosine similarity a clean distance metric

Chunking Strategy

Parameter	Value	Reasoning
Chunk size	800 characters	Roughly 100-150 tokens, comfortably under the limit
Overlap	100 characters (12.5%)	Prevents information loss at cut points
Cut points	Sentence boundaries	Syntactic segmentation preserves coherence
Metadata retained	Position, length, chunk ID	Enables traceability back to source

overlap_optimal = min(100, 0.15 × chunk_size)

Proportional to chunk size with a hard ceiling, so the ratio holds for small chunks without wasting compute on large ones.

Keeping per-chunk metadata mattered more than expected. When a search result surfaces a chunk, being able to say "this came from page 2, characters 1,600-2,400 of contract X" is the difference between a result a user trusts and one they have to verify manually.

5. Semantic Search with ChromaDB

The Problem with Keyword Search

Query	Keyword search finds	Semantic search finds
contrat voiture	only "voiture"	auto, véhicule, automobile
accident route	those exact words	sinistre, collision, dommage
assurance maison	only "maison"	habitation, logement, domicile
mohamed alami	exact spelling only	Mohammed Allami, M. Alami

That last row is the one agents care about most. Moroccan names transliterate inconsistently — Mohamed, Mohammed, Muhammad — and an exact-match system treats them as three different people.

"Contrat d'assurance automobile pour véhicule Renault"
  → [0.23, -0.45, 0.67, 0.12, ..., 0.89]

"Police auto pour voiture Peugeot"
  → [0.21, -0.43, 0.69, 0.14, ..., 0.87]   ← nearly identical

similarity = (A · B) / (|A| × |B|)          ← cosine, approaching 1 = same meaning

Why HNSW Matters

ChromaDB uses HNSW (Hierarchical Navigable Small World) indexing. Instead of comparing a query against every stored vector — O(N) — HNSW builds a navigable graph where similar documents are linked, and search walks toward the answer in O(log N). At 760 documents the difference is academic. At 500,000 it is the difference between a working product and a timeout.

Technology	Precision@5	Latency	Memory
FAISS	87%	120 ms	1.2 GB
Pinecone	89%	200 ms	Cloud-hosted
ChromaDB	89.3%	110 ms	0.9 GB

Metric	@1	@5	@10	@20
Precision	92.1%	89.3%	85.7%	79.2%
Recall	31.4%	74.8%	91.6%	96.3%
F1	46.8%	81.6%	88.5%	86.9%
NDCG	92.1%	87.4%	84.2%	82.1%

MRR: 0.847 · MAP: 0.823

Read this as two different user needs. "Find me this specific contract" is a precision@1 task — 92.1% means the right document is at the top nine times out of ten. "Show me everything related to this claim" is a recall@20 task — 96.3% means almost nothing is missed.

6. Security, Storage, and GDPR by Design

Handling national ID cards means compliance is not a checkbox at the end.

• Encryption at rest: AES-256 on every stored document in MinIO
• Encryption in transit: TLS 1.3 on all client-server communication
• Log anonymisation: CIN numbers masked before writing — a national ID appears as XXXXXXXX in every log line, preserving traceability without preserving the identifier
• Retention: Moroccan law requires 30-year retention for insurance contracts; the system manages the lifecycle explicitly and performs secure deletion once the statutory period expires
• Access and audit: role-based permissions, mandatory authentication, and a complete audit trail — every action timestamped with user identity and operation
• Sovereignty: no data leaves the building; Phi-3 runs locally through Ollama with no internet connection required and no third-party API in the processing path

Algorithmic Bias

A model trained on documents from one region performs worse on documents from another. The corpus was built with explicit geographic balance — Casablanca-Settat 32.0%, Rabat-Salé-Kénitra 18.0%, Marrakech-Safi 11.9%, Fès-Meknès 10.1%, Tanger-Tétouan-Al Hoceïma 9.0%, other regions 19.0%. Rural documents specifically make up 22% of the corpus, deliberately, because rural scans are systematically lower quality and excluding them would produce a model that quietly fails outside cities.

7. The Dataset

760 documents across 9 categories. Important caveat, stated plainly: this corpus was synthetically generated following statistical distributions derived from real sector data, not scraped from live customer files. That was a privacy decision, and it is the honest framing — it means the benchmarks measure the system against realistic-but-constructed documents, and real-world production numbers should be expected to differ.

Document type	Count	Share
National ID cards	105	13.8%
Driving licences	105	13.8%
Certificates	95	12.5%
Premium receipts	89	11.7%
Contracts	85	11.2%
Claims	78	10.3%
Endorsements	76	10.0%
Cancellations	67	8.8%
Supporting documents	60	7.9%
Total	760	100%

Quality Distribution — Deliberately Harsh

Quality level	Count	Mean DPI	SNR (dB)	OCR legibility
Excellent	183	600	>25	>95%
Good	196	300	20-25	85-95%
Medium	171	150	15-20	70-85%
Poor	210	75	<15	50-70%

210 documents — 27.6% of the corpus — are deliberately poor quality. Degradation was applied programmatically as a product of noise, blur, and skew factors. Benchmarking only on clean scans produces impressive numbers and a system that falls over in week one. The 91.3% headline figure includes those 210 bad documents.

8. Performance: Where the Time Actually Goes

Operation	Min	P50	P90	P99	Max
OCR + preprocessing	1.2	4.8	12.1	28.3	45.7
Hybrid classification	0.1	0.3	0.8	2.1	5.2
NER extraction	0.2	0.7	1.9	4.8	12.1
Embedding generation	0.1	0.2	0.5	1.2	3.1
ChromaDB indexing	0.05	0.1	0.3	0.8	2.1
MinIO storage	0.2	0.8	2.1	5.2	11.3
Semantic search	0.1	0.3	0.7	1.8	4.2
Full pipeline	2.1	6.8	16.2	38.1	72.4

All values in seconds.

OCR consumes roughly 70% of total processing time. Every other stage is effectively free by comparison. That is the optimisation target, and it is why intelligent OCR caching sits at the top of the roadmap. The P99 of 38 seconds is worth noticing too — those are the multi-page, badly-scanned edge cases. Averages hide them; percentiles do not.

Hardware: NVIDIA RTX 3060 (12GB) minimum, 16GB RAM for 50 concurrent documents, 2TB storage for a 500,000-document archive. Consumer-grade, deliberately.

9. Interface and Usability Testing

The best extraction pipeline in the world is worthless if agents will not use it.

Frontend: HTML5/CSS3, JavaScript ES6+, Bootstrap 5, Chart.js. Responsive, WCAG 2.1 Level AA compliant, sub-2-second page loads. Multi-file drag and drop up to 50 documents, real-time preview, per-stage progress bars, advanced filtering, semantic search with relevance scores, highlighted extracted metadata, confidence indicators, CSV/Excel export, and a metrics dashboard.

Test protocol: 7 participants (5 agents, 2 managers), 45-minute sessions, 8 representative workflow tasks, think-aloud protocol with screen recording and post-test questionnaires.

Task	Mean time	Success rate	Satisfaction
Upload documents	45s	100%	4.6/5
Simple search	32s	95%	4.4/5
Advanced search	78s	85%	4.1/5
Validate extraction	56s	90%	4.3/5
Navigate results	28s	100%	4.7/5
Export data	41s	95%	4.2/5
System configuration	124s	75%	3.8/5
Error handling	89s	80%	3.9/5
Overall	62s	90%	4.25/5

SUS score: 78.5/100 — solidly in "good usability" territory on the standard scale.

The two weak rows tell the real story. System configuration and error handling — 75% and 80% success, 3.8/5 and 3.9/5 — are where the interface failed people. Both are admin-facing surfaces that got less design attention than the core agent workflow — a classic and entirely predictable outcome, and one worth naming rather than burying under the 4.25 average.

10. Business Impact

Dimension	Before	After	Change
Processing time	20 min/doc	2 min/doc	10× faster
Error rate	12%	<1%	12× reduction
Contract retrieval	48 hours	30 seconds	~5,760× faster
Audit trail	Manual	Automatic	—
Annual savings	—	675,000 MAD	—

Projected ROI of 198% over three years, break-even at 12 months. These are projections built on measured per-document time savings extrapolated across volume, not realised financials from a production deployment. Treat them as a business case, not an audited result.

Comparison Against Commercial Alternatives

Criterion	This system	Adobe DC	ABBYY	Google Cloud
Arabic/French OCR	Excellent	Moderate	Excellent	Excellent
Classification	Hybrid (rules + AI)	Basic	Advanced	Moderate
Semantic search	ChromaDB / HNSW	No	Proprietary	Vertex AI
On-premise deployment	Yes	No	Limited	No
3-year total cost	680K MAD	1.5M MAD	2.1M MAD	1.8M MAD
Customisation	Total	Limited	Partial	No

Fairness note: this comparison reflects the requirements of this use case, and the cost figures are quoted rather than independently audited. ABBYY in particular is a mature, deeply capable platform, and for an organisation without on-premise constraints or Arabic-language requirements the calculus would likely favour a commercial product with vendor support behind it. The advantage here comes from specialisation and sovereignty, not from being categorically better software.

11. Honest Limitations

Every system write-up should have this section, and most do not.

• Handwriting. Error rate on handwritten documents reaches 23%. Both OCR engines are optimised for print, and claim declarations are frequently handwritten. This is the largest open gap.
• Non-standard layouts. Multi-column documents, heavy tables, and unconventional forms degrade noticeably.
• Long documents. Documents over 20 pages average 42 seconds — acceptable for batch, uncomfortable for interactive use.
• Hybrid document types. Endorsements sit at a 7% classification error rate because they legitimately share most of their vocabulary with contracts. Distinguishing them requires understanding document intent, not just content.
• Cascade dependency. Everything downstream depends on OCR quality. A bad extraction produces a bad classification, bad entities, and a bad embedding.
• Synthetic evaluation data. The 760-document corpus is synthetic. It matches real distributions and includes deliberate quality degradation, but it is not a production dataset.
• Small usability sample. Seven test participants is enough to surface obvious friction, not enough to be statistically robust.

12. Roadmap

• LLaVA integration for visual document analysis — reading logos, stamps, and signatures alongside text
• LoRA fine-tuning of Phi-3 on the nine insurance categories: rank 8, alpha 16, dropout 0.1, targeting q_proj and v_proj, batch size 16, learning rate 3e-4. Expected 75% reduction in GPU memory while retaining ~95% of full fine-tuning performance.
• Intelligent OCR caching to attack the 70% of pipeline time spent on extraction

Full multimodal architecture with late fusion:

P(y|X) = α · P_visual(y|X) + (1 - α) · P_textual(y|X)

where α is learned from per-modality quality — expected 15-20% improvement on degraded documents. Plus handwriting support via visual transformers and ERP/CRM connectors.

• Predictive risk analysis and fraud detection through anomaly detection
• Blockchain-backed immutable document traceability
• Regional expansion across the Maghreb with extended multilingual support
• Cross-sector application — banking, healthcare, public administration

What I Would Tell Someone Building This

• Hybridise where failure modes differ. Two OCR engines only beat one because Tesseract and EasyOCR fail on different inputs. Two models that fail identically give you cost, not accuracy. The same logic drove rules-plus-LLM classification: rules fail on ambiguity, LLMs fail on consistency, and combining them covers both.
• Rules are not legacy technology. 80% of documents classified correctly with zero inference. Reaching for a model when a regex works is expensive, slow, and harder to audit.
• Benchmark on the ugly data. 27.6% of the evaluation corpus is deliberately degraded. That is why the reported numbers are believable and why the system does not collapse on phone photos.
• Deployability is a feature. Choosing Phi-3 over Llama-2-7B prioritised something that runs on hardware the client already owns. Local execution was not just compliance — it made the project viable at all.
• Measure percentiles, not averages. The 6.8s median is the headline. The 38s P99 is the user complaint.
• Instrument the errors, not just the accuracy. Knowing that 34% of NER failures are segmentation problems and only 15% are OCR-propagated told me exactly where the next sprint should go. Aggregate accuracy would have told me nothing.

Final Thought

The interesting result here is not 91.3% OCR accuracy. It is where that 91.3% comes from: a hybrid architecture that gains 9.5 percentage points on bad scans and only 2.6 on good ones, rules that answer 80% of classification questions before a model wakes up, and a 3.8B-parameter model chosen specifically because it runs on hardware the client already owns.

Production AI systems are not won by picking the largest model. They are won by understanding exactly where each component fails, building something that covers the gaps, and then measuring honestly enough to know which gaps are still open.

This work was submitted as my Master's thesis in Artificial Intelligence and Virtual Reality at Université Ibn Tofail, Faculty of Sciences, Kénitra, in partnership with Assure Solutions Morocco.

Built with: Python · Flask · Tesseract · EasyOCR · Ollama · Phi-3 · Sentence Transformers · ChromaDB · MinIO · OpenCV · Docker

References

• Phi-3 Technical Report: A Highly Capable Language Model Locally On Your Phone — Microsoft Research
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks — Reimers & Gurevych
• Efficient and Robust Approximate Nearest Neighbor Search Using HNSW Graphs — Malkov & Yashunin
• ChromaDB — the AI-native open-source embedding database
• Tesseract OCR documentation
• EasyOCR — ready-to-use OCR with 80+ supported languages
• ACAPS — Moroccan insurance and social welfare supervisory authority