I built private AI over a construction company's documents that runs on one computer with a local 4B model. On 128 test questions, 99.2 % of answers contain exactly the expected facts, and every citation opens the passage in the PDF.
Open-source project (MIT license) on a synthetic but deliberately hard corpus: 12 linked documents of a construction company in Slovak and English, two projects with similar numbers and the same subcontractors across contracts. All companies and people are fictional.
Problem
A small construction company has its answers scattered across contracts, budgets, site diaries and manuals. The data must not go to the cloud, and the AI must not make up numbers or dates.
Solution
Hybrid search (vectors + BM25), contextual retrieval, a knowledge graph only for questions about relations between documents, and a reranker. The model never owns a number, time or date: code calculates them, and values from the text are checked against the source. Prompt injection protection runs before every answer.
Results
- Retrieval accuracy (MRR) 76.1 → 96.5 %, nDCG@10 80.0 → 96.9 %
- 99.2 % of answers with exactly the expected facts, 100 % valid citations
- 90 % of unanswerable questions correctly declined, 0.9 % unnecessary refusals
- Prompt injection success rate 16.7 → 0 %, red team 21 of 21 cases without a leak
Stack
- Python, Ollama, Qwen3 4B
- Hybrid search: dense + BM25 (RRF) and a reranker
- Knowledge graph across documents
- Evaluation: 128 questions with gold answers, metric gate in CI
What I learned
Running the graph on every question made results worse (MRR 76.1 → 54.8), so it only runs on questions about relations. The cheapest big gain came from contextual retrieval: one sentence of context per document chunk raised MRR from 76 to 85. The numbers come from small synthetic sets. Treat them as engineering evidence, not a benchmark.
GitHub: construction-graph-rag