Web3Evals @web3evals
Research and evaluation lab for AI in web3 security Joined August 2026-
Tweets8
-
Followers3
-
Following1
-
Likes2
Nice , it seems this applies in code review as well
We ran 90 LLM audits on real codebases with 10 system prompts, from 477 to 7,719 tokens. The 650-token prompt found the most bugs per run. The 7,719-token prompt, with the persona, the methodology and the full vulnerability taxonomy, repeated as few bugs as the bare baseline.
We ran 90 LLM audits on real codebases with 10 system prompts, from 477 to 7,719 tokens. The 650-token prompt found the most bugs per run. The 7,719-token prompt, with the persona, the methodology and the full vulnerability taxonomy, repeated as few bugs as the bare baseline.
One model (DeepSeek V4 Flash), 3 repos, 3 replicates, read-only tools. This is v0; more models and decontaminated repos next. Full write-up, all 90 runs, and the 6-line reporting block that won: web3evals.com/blog/what-a-sy…
Every prompt read the same files and covered all 27 known bug locations. Recall still ranged from 6 to 10. The prompt changed what the model reported, not what it read. The long prompt's extra finds were one-offs: it only pulls ahead once you merge three runs.
We’re Web3Evals — an independent research and evaluation lab for AI in Web3 security. We publish reproducible research, benchmarks, and leaderboards measuring how LLMs and agents audit real-world code. First benchmark: SCAR v1. web3evals.com
pessimist @0xpessimist
1K Followers 902 Following 22. security researcher @Hashlock_, my sensei @0xSorryNotSorry, prev game designer https://t.co/0ZHCkSxkRc
Tarun Koyalwar @KoyalwarTarun
384 Followers 321 Following AI Researcher @ ProjectDiscovery , I do benchmarks, evals and more , views are my own https://t.co/Xx79fGCaxy
Nishant | AresAudits @NishantKoyalwar
112 Followers 389 Following Building @web3evals | Security researcher exploring LLM's, agents & blockchain. 45+ audits • $1.2B+ TVL secured • 2× 1st place 🏆
vitalik.eth @VitalikButerin
8.0M Followers 551 Following I choose balance. First-level balance. mi pinxe lo crino tcati https://t.co/gCQrmCby7PTrends for United States
You might like
