File 04 / 06
Insight.
AI-assisted code review that catches the logic bugs static tools miss.
Year
2023
Role
ML Engineer & Product
Timeline
5 months
Status
Internal tool
Stack
01 — Overview
Insight reads every pull request at Intactic and reviews it like a senior engineer who has infinitely many coffee refills — trained on six years of our own review history, tuned to our conventions and our past mistakes.
02 — The challenge
Generic LLM review noise trained on the whole internet buried real findings under style nitpicks. Static analyzers caught the bugs we were never going to write anyway. The gap — actual logic errors, race conditions, broken invariants — stayed wide open.
03 — Approach
- Fine-tuned on 14,000 of our own resolved review threads, so the model learned our invariants, not the internet’s
- Built a retrieval layer over the codebase and incident history, grounding every comment in how the system actually behaves
- Ranked findings by blast radius and confidence, and suppressed anything below the bar — silence is a feature
- Measured against a hidden set of past real bugs: the ones that reached production before Insight existed
04 — The outcome
Insight now catches roughly one would-be-production bug per week that static tooling never flagged. Reviewer focus shifted from typo-hunting to architecture — which was always the point.