We built an autonomous DeFi security scanner in 4 weeks. Here's what we learned.
The premise was simple: combine traditional static analysis tools with AI reasoning to find vulnerabilities that neither approach catches alone.
The result: 15 specialized scanners, 643 passing tests, and an 82.6% detection rate on the EVMbench benchmark (the GPT-5.3-Codex baseline is 72.2%).
Three things surprised us:
1. Hybrid beats pure-AI. The industry narrative is "AI will replace static analyzers." Our data says the opposite. Slither catches patterns that LLMs hallucinate past. LLMs catch economic logic that static tools can't reason about. Together, they're measurably better than either alone.
2. Local models are competitive. VulnLLM-R-7B running on a laptop via Ollama hits 75-80% detection — for $0 per scan. A 2-week Claude Opus 4.6 audit of Firefox cost $4,000 in API credits. The cost curve for security scanning is collapsing.
3. The bottleneck is precision, not recall. Every AI security tool finds vulnerabilities. The hard part is not flooding developers with false positives. We built multi-model consensus voting and mutation testing specifically to solve this. Precision > recall.
The autonomous hunting pipeline is now operational: discover programs → scan → triage → generate reports. Phase 1 (build the tool) is done. Phase 2 (earn bounties) starts now.
All work is public: github.com/gilchrist-research
---
#DeFi #SmartContractSecurity #AI #CyberSecurity #Web3 #OpenSource #BuildInPublic