Eval Sys @EvalSysOrg
EVAL SYS is a continuously evolving, open-source community! We are on a mission to track and advance model agentic capabilities. mcpmark.ai Joined August 2025-
Tweets21
-
Followers229
-
Following3
-
Likes51
Congrats on the launch of Strata! Thrilled that @Klavis_AI chose MCPMark. š MCPMark now benchmarks not only model agentic performance, but also MCP Services and frameworks. Canāt wait to see what the community builds next ā and always open to partnership!
Congrats on the launch of Strata! Thrilled that @Klavis_AI chose MCPMark. š MCPMark now benchmarks not only model agentic performance, but also MCP Services and frameworks. Canāt wait to see what the community builds next ā and always open to partnership!
A good piece of work from one of our core contributors @JarvisMSUstc . Worth reading.
A good piece of work from one of our core contributors @JarvisMSUstc . Worth reading.
š MCPMark website updated! ā mcpmark.ai On MCPMark, you can now dive into each taskās description, verification, and model performance. Weāve also added model trajectories to each task leaderboard, so you can clearly see how models execute step by step. Manyā¦
We believe MCP servers are shaping the future of software. Thatās why we built MCP Mark: a live benchmark for model mastery in real-world MCP use. Amazing experience collaborating with great friends & building communityāEvalSys will keep driving meaningful work for the community!
We believe MCP servers are shaping the future of software. Thatās why we built MCP Mark: a live benchmark for model mastery in real-world MCP use. Amazing experience collaborating with great friends & building communityāEvalSys will keep driving meaningful work for the community!
Excellent eval, and open weight models holding ground š
Excellent eval, and open weight models holding ground š
1/8) Excited to launch MCPMark (mcpmark.ai) today with NUS TRAIL and @EvalSysOrg! š„³ Itās a highāquality, programāverifiable benchmark for MCP (Model Context Protocol) ā designed to measure model's agentic capability & stability in MCP Use. Not another ālabāā¦
1/8) Excited to launch MCPMark (mcpmark.ai) today with NUS TRAIL and @EvalSysOrg! š„³ Itās a highāquality, programāverifiable benchmark for MCP (Model Context Protocol) ā designed to measure model's agentic capability & stability in MCP Use. Not another ālabāā¦
š šJust launched MCPMark, a challenging MCP benchmark I participated in. Its filesystem section include ops on files, structure exploration, reasoning, and multi-skill tasks. Most models show clear room for improvement, while GPT series excel in precise text manipulation
š šJust launched MCPMark, a challenging MCP benchmark I participated in. Its filesystem section include ops on files, structure exploration, reasoning, and multi-skill tasks. Most models show clear room for improvement, while GPT series excel in precise text manipulation
Excited to be part of @EvalSysOrg and contribute to our debut milestone, MCPMark! Follow us for updates, and come join the teamāweāre just getting started! Githubļ¼ github.com/eval-sys/mcpmaā¦ Websiteļ¼ mcpmark.ai Huggingface trajectory logļ¼ huggingface.co/datasets/Jakumā¦
Excited to be part of @EvalSysOrg and contribute to our debut milestone, MCPMark! Follow us for updates, and come join the teamāweāre just getting started! Githubļ¼ github.com/eval-sys/mcpmaā¦ Websiteļ¼ mcpmark.ai Huggingface trajectory logļ¼ huggingface.co/datasets/Jakumā¦
Sharing some of my thoughts when developing, hope they can help š 1/ Choosing the initial state defines task diversity, difficulty, and usefulness. 2/ State tracking and management is the trickiest stage. Each MCP needs its own isolation strategy. Worth it though: sandboxingā¦
Sharing some of my thoughts when developing, hope they can help š 1/ Choosing the initial state defines task diversity, difficulty, and usefulness. 2/ State tracking and management is the trickiest stage. Each MCP needs its own isolation strategy. Worth it though: sandboxingā¦
Thrilled to see MCPMark officially live! š @AnthropicAI's vision for MCP ā a universal, open standard for AI integrations ā is summed up perfectly by the āUSBāC port for AIā analogy: a single, reliable connector that lets LLMs access tools and data seamlessly. ļæ¼ Withā¦
Thrilled to see MCPMark officially live! š @AnthropicAI's vision for MCP ā a universal, open standard for AI integrations ā is summed up perfectly by the āUSBāC port for AIā analogy: a single, reliable connector that lets LLMs access tools and data seamlessly. ļæ¼ Withā¦
The first EvalSys initiative, in amazing collaboration with NUS TRAIL @michaelqshieh and @lobehub ! ā¢ 127 challenging real-world tasks, 2-3 expert hours each, pass@1 <30% (except GPT-5) ā¢ Solid engineering effort on environment isolation and state tracking More insightsā¦
The first EvalSys initiative, in amazing collaboration with NUS TRAIL @michaelqshieh and @lobehub ! ā¢ 127 challenging real-world tasks, 2-3 expert hours each, pass@1 <30% (except GPT-5) ā¢ Solid engineering effort on environment isolation and state tracking More insightsā¦
Mabel Wang @woneusk
2 Followers 38 Following
Y @mng524353696432
3 Followers 493 Following
Limbicnation @Limbicnation
626 Followers 1K Following Game dev, ML explorer š®š» https://t.co/hAGCOGrQQj #ETH #Solana #UE5 #houdini #b3d #zbrush #AIart #generativeart https://t.co/B7bv3GSWJ9
Ashish Kalya @KalyaAshish
185 Followers 3K Following Product@Tekion, Ex-Product@Sprinklr, Ex-founder
Bill Sun @BillSun_AI
19K Followers 1K Following Cofounder of Gen Alpha(try https://t.co/r38XTz7nSL) and Chief Scientist of @PINAI_IO; Stanford Math PhD, ex-google deepmind AI, 1st to get transformer work on QA
Generative Alpha @aiusd_
18K Followers 170 Following Worldās First Agentic AI Exchange. Our Agentic Money Product at https://t.co/uWmZf7FD7A; Morr about us: https://t.co/mEgIOpaU9c
JY Z @JunYuZzzzz
10 Followers 172 Following
Zoe Wan @ZoeWanyiyi
3 Followers 120 Following
jodan_no1_ꬔéå @littlebabytong
15 Followers 167 Following
Elite AI Talent Acqui... @Elite_AI_Talent
8 Followers 70 Following Connecting untapped AI/ML talent from SoCal universities with the worldās most ambitious companies. | HQ: Redondo Beach https://t.co/VmUsCJoMii
JasonQQQQQ @weiqinghua2015
5 Followers 169 Following
Isa geƧer @GecerIsa
0 Followers 143 Following
Nadav Fux @nadavfux
70 Followers 1K Following
dc @d11cc3s
86 Followers 2K Following č·äŗŗę°č³Ŗ. Infra sherpa. Maximum effort. Details matter. Be excellent to each other. @d11cc3s.bsky.social
Chris M @chrismsyarra
340 Followers 4K Following Exec. Doctorate (c DBA), MBA (Ent&Innov), MAppFin, GradCert Data Science, BSc Digital/IT.
Sankar Nagarajan @nsankar
91 Followers 920 Following Passionate about harnessing the power of Data Science, Machine Learning & AI to solve complex challenges. Principal Data Scientist - Virtana Corp.
Tariq Forde @SynagryLabs
3 Followers 6 Following Founder at SynargyLabs - I help Businesses grow with AI sales systems
AntĆ³n @anton_valletas
340 Followers 992 Following
Vasanth Kumar Nataraj... @anonymous100799
36 Followers 1K Following
Square And Compass @SquareAnCompass
35 Followers 244 Following
Huanxuan Liao @xn_hyacinth
26 Followers 237 Following 3rd year PhD Student @UCAS1978 | Focusing on Long Context Modeling
SĆ©samee @mochi__crossing
62 Followers 359 Following
We live in a time of ... @eemin72
19 Followers 5K Following HUMAN RIGHTS was a lie. FREE SPEECH was a lie. DEMOCRACY was a lie. We live in a time of monstersā¦
Peter Dave HelloĶĶÆļæ½... @PeterDaveHello
683 Followers 3K Following Free and open source software enthusiast, Maintainer and active contributor of few projects.
William Callahan @WilliamCallahan
950 Followers 4K Following Building @aVentureVC (Techstars ā23, a VC/startup research platform - https://t.co/B2EnOfV0UY). CFA. Angel & VC investor.
@b3x @abyouze
14 Followers 414 Following
Philippe šŖšŗ @FdsPhilippe
233 Followers 3K Following You miss 100% of the shots you donāt take. Software engineer
chengjilai @chengjilai
24 Followers 966 Following
šnothingāļø @PlarkoD
0 Followers 47 Following
entanima dev @Entanima
18 Followers 186 Following Building protocols & patterns for provable collaboration. E2EE, data sovereignty, OSS models. Here to share notes and talk shop.
Coder @xyQTEdCSePvhFQo
15 Followers 496 Following
Andrea @sinapsi0
0 Followers 45 Following
Hannie @hannie_liu
67 Followers 569 Following math, models, and memes š¤ ex-founder of AICV startup (Asia open dataš„ & exit)
Michael Jentsch @msoftware
150 Followers 1K Following Weeks of coding can save you hours of planning.
Mouzou @Mouzou07
420 Followers 2K Following
Arvin Xu @arvinxu95
40 Followers 120 Following Design Engineer / Founder and CEO of @lobehub / core member of @AntDesignUI / ex AntGroup
Allison Zhan @AllisonXinyuan
88 Followers 391 Following Investing in early stage AI. First partner to @lobehub , @klavis_ai , @LeptonAI (acq NVIDIA) etc. | co-founder of @EvalSysOrg | Views are my own
LobeHub @lobehub
3K Followers 43 Following Empowering your AI dreams with LobeHubš¤Æ LobeChat Cloud are Public Testing: https://t.co/jPxgXOr17uTrends for United States
You might like
