AdvAI

The problem

Reviewing a contract against Bolivian civil and commercial codes and case law takes hours, and LLMs tend to hallucinate articles. AdvAI splits a contract into clauses and returns, for each one, a risk level, an analysis and verbatim, verifiable citations.

Highlights

1. 01Hybrid search: BM25 (SQLite FTS5) + bge-m3 embeddings fused with RRF over 3,267 articles and 971 supreme court rulings — retrieval hit@8 went from 42.9% to 80%.
2. 02The model only picks keys from the sources shown and the quoted text is inserted by the system: 82.9% correct citations and 94.6% of clauses with no severe error in internal evaluation.
3. 03Personal data is anonymized before any text reaches the cloud; local LLM (Ollama, qwen3.5 9B) plus free providers with automatic failover.
4. 04Resumable background worker, live progress via SSE, version diffing and DOCX/PDF reports.
5. 05Backend with 69 tests and ≥90% coverage enforced in CI; Next.js 16 + React 19 frontend with a PDF viewer and risk-highlighted clauses.

What I learned

• → 01Measure before optimizing: without an eval set I would have kept tweaking prompts blindly. With metrics, every change became a decision.
• → 02An LLM is not a database: the fix for hallucinated citations wasn’t a better prompt, it was an architecture where the model can’t write the citation.
• → 03Privacy by design: anonymizing before sending text to the cloud shaped everything from the data schema to the prompts.
• → 04Design for graceful failure: providers with failover and a worker that resumes after a restart.