Independent tests of AI security agents and SOC decision models. DFIR results, evidence checks and practical findings. Follow for new benchmark results.secbenchmarks.comJoined September 2026
We’ve completed our first Meta Muse benchmarks.
Spark 1.3 holds the top spot for Muse. However, it still trailed GPT, Gemini and Claude Opus on single-host proof scores.
secbenchmarks.com/blog/2026-10-1…
Our first Gemini 4 Argon Benchmark and it performed incredibly well!
Agentic SOC investigation harness. Run the models against different cases. Track key performance.
If you're interested in Decision Models like Jev and Clef we have some of that too! secbenchmarks.com/blog/2026-10-1…
Each model started with one confirmed part of an attack and assessed connected processes, files, computers and accounts.
The score rewards finding related items and penalizes unrelated additions. The supplied starting point earns no credit.
Small pilot; no unseen test set.
We tested 3 AI decision models on a DFIR task: finding other activity connected to a confirmed attack.
Cloudflare's Clef, Cloudflare's Clef Flash and Jev each ran 8 cases, 3 times.
Here's how they stacked up:
secbenchmarks.com/blog/2026-10-0…
🔎 GPT-6 Astra retrieved sufficient evidence in 36/36 basic DFIR cases.
25/36 ended with a correct decision and accepted proof.
Finding the records doesn’t guarantee a supported conclusion.
Preliminary, one pass per case.
secbenchmarks.com/blog/2026-09-3…
📊 Our first AI security benchmark results are live.
In 36 basic DFIR cases, Claude Opus 5.5 got 33 verdicts right. 12 also met our proof requirements.
Why we score the answer and its evidence separately:
secbenchmarks.com/blog/2026-09-3…
🧭 18/18 verdicts. 12/18 next steps.
Jev 1.13's preliminary SOC results show why “right answer” and “right action” need separate scores.
Our new analysis covers six models and the limits of a six-case test:
secbenchmarks.com/blog/2026-09-3…
@DFIRTraining The timing comparison is useful. Were the web-activity summaries also checked for correct artifact citations and missed domains? We’d like to see speed, factual accuracy and evidence coverage measured separately.
323 Followers 364 FollowingCybersecurity Professor @ NU | Founder of FullStack Platforms
//
Applied AI and Cyber defense is my ongoing research direction
938 Followers 6K Following🙃 Consciousness is - Awareness of attention of any kind of phenomenal experience, or awareness of attention of nothing, or awareness of awareness - 🤔
9K Followers 5K FollowingFounder & CEO of AI Research & Engineering Lab | Weekly @twimlai GenAI meetup | @Cohere_Labs Community Lead & Researcher | Prev @TCS @Infobeans | No dupes ever
995 Followers 4K FollowingAuthor, Passionate about Distributed Systems. Currently AI @Google India, co founded @adalyz_ Opinions here are my own ! He/Him
9K Followers 5K FollowingFounder & CEO of AI Research & Engineering Lab | Weekly @twimlai GenAI meetup | @Cohere_Labs Community Lead & Researcher | Prev @TCS @Infobeans | No dupes ever
120K Followers 514 FollowingMITRE ATT&CK® - A knowledge base for describing the behavior of adversaries. Replying/Following/Re-tweeting ≠ endorsement. @ https://t.co/wt46ArkZVt
306K Followers 77 FollowingPart of @CISAgov, we respond to major incidents, analyze threats, and exchange critical cybersecurity information with partners around the world.
79K Followers 762 FollowingEnd-to-end Cybersecurity consulting team leading the industry, supporting organizations, and giving back. #Hacktheplanet
Blogs, news, webinars, and tools!
18K Followers 644 FollowingHacker, trainer, and guitarist | Black Hills InfoSec #RedTeam | @BreakForge Training | Produces music to hack to at @N0BANDW1DTH
39K Followers 3K FollowingPartner / Threat Intelligence at Microsoft. Previously, Director of Incident Response & Intel Research at Mandiant. Former Chief Technical Analyst at CISA
112K Followers 104 FollowingThe world's leading Digital Forensics and Incident Response provider. This feed updates you on latest DFIR news, events, and training.