Princeton NLP Group @princeton_nlp

Princeton NLP Group led by @prfsanjeevarora @danqi_chen @karthik_r_n nlp.cs.princeton.edu Princeton, NJ Joined August 2020

Tweets

258
Followers

5K
Following

62
Likes

284

Ofir Press @OfirPress

2 months ago

AlgoTune is a benchmark that penalizes expensive models, since we give each model a budget of $1 to solve each task. Cool to see open weight models doing well! x.com/ori_press/stat…

Ori Press @ori_press

2 months ago

AlgoTune is a benchmark that penalizes expensive models, since we give each model a budget of $1 to solve each task. Cool to see open weight models doing well! x.com/ori_press/stat…

0 3 32 6K 2

Download Image

1 1 25 4K 8

What happens if you compare LMs on SWE-bench without the fancy scaffolds? Our new leaderboard “SWE-bench (bash only)” shows you which LMs are the best at getting the job done with just bash. More on why this is important 👇

14 25 205 32K 72

Download Image

Princeton Laboratory for Artificial Intelligence @PrincetonAInews

2 months ago

Shoutout to all the @Princeton researchers participating in @icmlconf #ICML2025 Browse through some of the cutting edge research from AI Lab students, post-docs and faculty being presented this year: pli.princeton.edu/blog/2025/prin…

0 9 47 4K 7

Download Image

Ben Shi @BenShi34

4 months ago

As we optimize model reasoning over verifiable objectives, how does this affect human understanding of said reasoning to achieve superior collaborative outcomes? In our new preprint, we investigate human-centric model reasoning for knowledge transfer 🧵:

6 39 180 23K 137

Download Image

carlos @_carlosejimenez

4 months ago

Improved reasoning increases performance on benchmarks, but are models able to pass their knowledge onto humans? 🧐 We evaluate models’ communication abilities in teaching novel solutions to users! See our new paper!

Ben Shi @BenShi34

4 months ago

6 39 180 23K 137

Download Image

0 1 6 1K 1

Kabir @plodq

5 months ago

Introducing SWE-bench Multilingual: a new eval in the SWE-bench family to test LLM coding abilities in *9* programming languages, fully integrated with SB so it can plug into existing workflows. Claude 3.7 gets 43% on SB Multilingual vs 63% on SB Verified, a 20 pt drop!🧵

2 16 67 15K 25

Download Image

Ofir Press @OfirPress

5 months ago

Join us on May 21st- I'll talk about how we built SWE-bench & SWE-agent and what I'm excited about for the future of autonomous AI systems.

PyTorch @PyTorch

5 months ago

Join us on May 21st- I'll talk about how we built SWE-bench & SWE-agent and what I'm excited about for the future of autonomous AI systems.

3 9 22 21K 7

Download Image

2 3 20 7K 6

Stanford NLP Group @stanfordnlp

5 months ago

Our warmest congratulations to ⁦@danqi_chen⁩, ⁦@stanfordnlp⁩ grad and now Associate Professor at ⁦@PrincetonCS⁩ and Associate Director of ⁦@PrincetonPLI⁩ on her stunning ⁦⁦@iclr_conf⁩ keynote!

6 22 276 36K 15

Download Image

Alex Zhang @a1zhang

5 months ago

Claude can play Pokemon, but can it play DOOM? With a simple agent, we let VLMs play it, and found Sonnet 3.7 to get the furthest, finding the blue room! Our VideoGameBench (twenty games from the 90s) and agent are open source so you can try it yourself now --> 🧵

22 56 413 75K 154

Download Video

Ben Shi @BenShi34

6 months ago

Can language models effectively impersonate you to family and friends? We find that they can: 44% of the time, close friends and family mis-identify Llama-3.1-8b as human… 🧵👇

3 3 8 3K 4

Download Image

Ofir Press @OfirPress

6 months ago

Congrats on the Verified and Multimodal SWE-bench numbers. venturebeat.com/ai/zencoders-c…

0 1 5 1K 4

Ofir Press @OfirPress

6 months ago

We just updated the SWE-bench Multimodal leaderboard. Congrats to Globant, Zencoder, and the Agentless team from UIUC for their strong results.

1 5 29 3K 9

Download Image

Princeton NLP Group @princeton_nlp

6 months ago

Nothing like a sunny hike to welcome spring!

1 6 79 10K 4

Download Image

Alex Wettig @_awettig

7 months ago

🤔 Ever wondered how prevalent some type of web content is during LM pre-training? In our new paper, we propose WebOrganizer which *constructs domains* based on the topic and format of CommonCrawl web pages 🌐 Key takeaway: domains help us curate better pre-training data! 🧵/N

5 58 211 49K 105

Download Image

Ofir Press @OfirPress

7 months ago

This Tuesday (Feb 18), @_carlosejimenez will discuss SWE-bench and the future of codegen evals, as part of the Conference on Synthetic Software in NYC. @KLieret will also be there. RSVP: lu.ma/k2q27yi3

1 2 8 1K 3

Download Image

Kilian Lieret @KLieret

8 months ago

SWE-agent 1.0 is the open-source SOTA on SWE-bench Lite! Tons of new features: massively parallel runs; cloud-based deployment; extensive configurability with tool bundles; new command line interface & utilities.

3 18 61 10K 22

Download Image

Yong Lin @Yong18850571

8 months ago

🚀 Introducing Goedel-Prover: A 7B LLM achieving SOTA open-source performance in automated theorem proving! 🔥 ✅ Improving +7% over previous open source SOTA on miniF2F 🏆 Ranking 1st on the PutnamBench Leaderboard 🤖 Solving 1.9X total problems compared to prior works on Lean…

13 66 274 86K 144

Download Image

Ofir Press @OfirPress

8 months ago

Congrats to o3-mini on setting a new high score on SciCode!! R1 clocks in at an impressive 4.6%, matching Claude 3.5. SciCode is our super-tough programming benchmark written by PhDs in various scientific domains.

10 3 42 6K 9

Download Image

Ofir Press @OfirPress

8 months ago

SciCode is our super tough coding benchmark testing the abilities of LMs to program code based on research in physics/biology/material science/... o1 is the SoTA with 7%. To make it easier to use we're putting it into the Inspect AI format, as a few groups were asking for this.