Picture

Jeff Da @_jeffda

Research Scientist @scale_ai. Research on Reinforcement Learning, Agents, Reasoning. Ex: @allen_ai jeffda.com Joined July 2017
  • Tweets
    126
  • Followers
    402
  • Following
    840
  • Likes
    841
Jeff Da retweeted
alexandr_wang Profile Picture

Alexandr Wang @alexandr_wang

a week ago

New, very needed benchmark from @scale_AI: SWE-Bench Pro Includes: - Multi-file edits - 100+ lines changed on average - Complex dependencies across large codebases Current top model scores: - GPT-5: 23.3% - Claude Opus 4.1: 22.7% - Others drop further (<15%)

vbingliu Profile Picture

Bing Liu @vbingliu

a week ago

New, very needed benchmark from @scale_AI: SWE-Bench Pro Includes: - Multi-file edits - 100+ lines changed on average - Complex dependencies across large codebases Current top model scores: - GPT-5: 23.3% - Claude Opus 4.1: 22.7% - Others drop further (<15%)

Jeff Da retweeted
_zifan_wang Profile Picture

Zifan (Sail) Wang @_zifan_wang

a week ago

Congrats to @_jeffda and @XiangDeng1 on wrapping off their work on SWE-Bench Pro, a benchmark using copyleft repositories and code bases from real startups — shouldn’t appear in training and prior eval sets. Tasks also require much more LoC per reference solution from humans.

vbingliu Profile Picture

Bing Liu @vbingliu

a week ago

Congrats to @_jeffda and @XiangDeng1 on wrapping off their work on SWE-Bench Pro, a benchmark using copyleft repositories and code bases from real startups — shouldn’t appear in training and prior eval sets. Tasks also require much more LoC per reference solution from humans. https://t.co/VMH9leQDDO

Jeff Da retweeted
vbingliu Profile Picture

Bing Liu @vbingliu

a week ago

🚀 Introducing SWE-Bench Pro — a new benchmark to evaluate LLM coding agents on real, enterprise-grade software engineering tasks. This is the next step beyond SWE-Bench: harder, contamination-resistant, and closer to real-world repos.

Jeff Da retweeted
ABosselut Profile Picture

Antoine Bosselut @ABosselut

4 weeks ago

The next generation of open LLMs should be inclusive, compliant, and multilingual by design. That’s why we (@EPFL @ETH_en) built Apertus.

cscsch Profile Picture

CSCS Lugano @cscsch

4 weeks ago

The next generation of open LLMs should be inclusive, compliant, and multilingual by design. That’s why we (@EPFL @ETH_en) built Apertus. https://t.co/yohUnVuwjW

Jeff Da retweeted
_zifan_wang Profile Picture

Zifan (Sail) Wang @_zifan_wang

2 months ago

Congrats @XiangDeng1, @boyuan__zheng, @LiaoZeyi and our collaborators from OSU and UC Berkeley on releasing WebGuard dataset for training browser agents in recognizing potentially high-risk actions.

scale_AI Profile Picture

Scale AI @scale_AI

2 months ago

Congrats @XiangDeng1, @boyuan__zheng, @LiaoZeyi and our collaborators from OSU and UC Berkeley on releasing WebGuard dataset for training browser agents in recognizing potentially high-risk actions. https://t.co/xLGWdyGfL6

Jeff Da retweeted
anisha_gunjal Profile Picture

Anisha Gunjal @anisha_gunjal

2 months ago

🤔 How do we train LLMs on real-world tasks where it’s hard to define a single verifiable answer? Our work at @scale_AI introduces Rubrics as Rewards (RaR) — a framework for on-policy post-training that uses structured, checklist-style rubrics as interpretable reward signals. 🧵

Jeff Da retweeted
alexfabbri4 Profile Picture

Alex Fabbri @alexfabbri4

2 months ago

Excited to share MultiNRC, a new SEAL Leaderboard at Scale AI! MultiNRC is a challenging multilingual reasoning benchmark with native questions in French, Spanish, and Chinese. Leaderboard: scale.com/leaderboard/mu… Paper: tinyurl.com/MultiNRC Data: huggingface.co/datasets/Scale…

scale_AI Profile Picture

Scale AI @scale_AI

2 months ago

Excited to share MultiNRC, a new SEAL Leaderboard at Scale AI! MultiNRC is a challenging multilingual reasoning benchmark with native questions in French, Spanish, and Chinese. Leaderboard: scale.com/leaderboard/mu… Paper: tinyurl.com/MultiNRC Data: huggingface.co/datasets/Scale…

Jeff Da retweeted
SeanHendryx Profile Picture

Sean Hendryx @SeanHendryx

3 months ago

What will the learning environments of the future look like that train artificial super intelligence? In recent work at @scale_AI , we show that training systems that combine verifiable rewards with multi-agent interaction accelerate learning.

_jeffda Profile Picture

Jeff Da @_jeffda

11 months ago

We find that training reward models using a goal-conditioned reward function improves reasoning + general alignment performance!

SeanHendryx Profile Picture

Sean Hendryx @SeanHendryx

11 months ago

We find that training reward models using a goal-conditioned reward function improves reasoning + general alignment performance!

Jeff Da retweeted
hughbzhang Profile Picture

Hugh Zhang @hughbzhang

a year ago

Enabling LLMs to reason more deeply at inference time via search is one of the most exciting directions in AI right now. We introduce PlanSearch, a novel method for code generation that searches over high-level "plans" in natural language as a means of encouraging diversity.

Jeff Da retweeted
karpathy Profile Picture

Andrej Karpathy @karpathy

a year ago

Nice, a serious contender to @lmsysorg in evaluating LLMs has entered the chat. LLM evals are improving, but not so long ago their state was very bleak, with qualitative experience very often disagreeing with quantitative rankings. This is because good evals are very difficult…

alexandr_wang Profile Picture

Alexandr Wang @alexandr_wang

a year ago

Nice, a serious contender to @lmsysorg in evaluating LLMs has entered the chat. LLM evals are improving, but not so long ago their state was very bleak, with qualitative experience very often disagreeing with quantitative rankings. This is because good evals are very difficult… https://t.co/EEqCegELOl

Trends for United States

Boone
11 B posts
#DWTS
7.856 posts
Fried
32,5 B posts
Weaver
9.131 posts
Yankees
66 B posts
Crochet
21 B posts
Chapman
5.578 posts
Volpe
11 B posts
#WWENXT
8.731 posts
Red Sox
17,9 B posts
Bregman
5.397 posts
Yoshida
8.477 posts
Cora
7.963 posts
#RepBX
12,8 B posts
#DWCS
1.300 posts
WNBA
63,8 B posts
#BOSvsNYY
Danielle
14,1 B posts
Duran
22,3 B posts
Devin Williams

You might like

Postdoc @StanfordNLP Building the social brain for AI PhD @LTIatCMU. he/him/他
Hao Zhu 朱昊
@_Hao_Zhu
3K Followers
phd-ing @MIT_CSAIL, working on AI for education | formerly @allen_ai, @harvard ‘20
Alexis Ross
@alexisjross
3K Followers
Incoming assistant professor @UTCompSci, RS @BytedanceTalk, PhD from @uwcse, formerly @allen_ai @AIatMeta @MSFTResearch
Yizhong Wang
@yizhongwyz
6K Followers
Assistant Professor @UWCheritonCS, @CIFAR_News AI Chair @VectorInst, @ReviewAcl Co-CTO | PhD @TTIC_Connect | BS @PKU1898 | Excited about
Freda Shi
@fredahshi
3K Followers
Helping machines make sense of the world. Asst Prof @ICepfl; Before: @stanfordnlp @allen_ai @uwnlp @MSFTResearch #NLProc #AI
Antoine Bosselut
@ABosselut
4K Followers
Ph.D. student @UChicago Ex-SR @google Ex-Scientist @AWS. Ex-RA @jhuCLSP @columbianlp @TsinghuaNLP. Ex-Intern @IBM @AWS. Opinions are my own.
Chenghao Yang
@chrome1996
2K Followers
PhD at UT Austin ex-{uw, isi, facebook} nlper Former intern @SFResearch
Leo Liu
@ZEYULIU10
1K Followers
PhD student @LTIatCMU. SWEing Socially-aware SWE agents @allhands_ai. Previously, @allen_ai, @UWNLP, @Apple, @UCBerkeley; Social Intelligence in language +X.
Xuhui Zhou
@nlpxuhui
1K Followers
Assistant Professor @CSatUSC | Previously Visiting Researcher @facebookai | Stanford CS PhD @StanfordNLP
Robin Jia
@robinomial
4K Followers
姜力炜 • Ph.D. candidate @uwnlp | visiting @stanford | @nvidia @allen_ai 🧊 advancing AI & understanding humans & benefiting society 🏔️ lifetime adventurer
Liwei Jiang
@liweijianglw
2K Followers
PhD student at @uwcse @uwnlp
Alisa Liu
@alisawuffles
3K Followers
Research Scientist at Meta Fundamental AI Research (FAIR)
Scott Yih
@scottyih
2K Followers
Research Lead @USC_ISI and Research Assistant Professor @CSatUSC PhD at CMU ML/NLP @LTIatCMU @CarnegieMellon
Xuezhe Ma (Max)
@MaxMa1987
2K Followers
Head of AI at @magicaltome. Ex-Language Researcher at @FacebookAI. Large LMs and multilingual NLP. @JHUCLSP and @Cornell alumn. https://t.co/WTSCasqDI6
Ves Stoyanov
@vesko_st
2K Followers
Co-founder & CTO @kotoba_tech | Research Assistant Prof. @TTIC_Connect | PhD from @nlpnoah at @UW | IBM PhD Fellow | 孫正義育英財団生 | @Yale Undergraduate
Jungo Kasai 笠井淳...
@jungokasai
2K Followers