Jacob Morrison @jacobcares
PhD student @uwnlp, research @allen_ai @ai2_allennlp jacobmorrison.com Seattle Joined April 2009-
Tweets139
-
Followers422
-
Following486
-
Likes24K
this is super cool 👀
Evaluating language models is tricky, how do we know if our results are real, or due to random chance? We find an answer with two simple metrics: signal, a benchmark’s ability to separate models, and noise, a benchmark’s random variability between training steps 🧵
Evaluating language models is tricky, how do we know if our results are real, or due to random chance? We find an answer with two simple metrics: signal, a benchmark’s ability to separate models, and noise, a benchmark’s random variability between training steps 🧵 https://t.co/8sAL5yWbh3
if you’re talking about zoo-related stuff and you start talking about how “zoos need to be abolished” i’m immediately not listening to you
America needs to take open models more seriously. This summer the early lead in open model adoption of the US via Llama has been overtaken by Chinese models. With The American Truly Open Models (ATOM) Project we're looking to build support and express the urgency of this issue.
The bottleneck in AI isn't just compute - it's access to diverse, high-quality data, much of which is locked away due to privacy, legal, or competitive concerns. What if there was a way to train better models collaboratively, without actually sharing your data? Introducing…
The bottleneck in AI isn't just compute - it's access to diverse, high-quality data, much of which is locked away due to privacy, legal, or competitive concerns. What if there was a way to train better models collaboratively, without actually sharing your data? Introducing… https://t.co/mmDhWBp9AZ
Introducing FlexOlmo, a new paradigm for language model training that enables the co-development of AI through data collaboration. 🧵
💡FlexOlmo Recipe 1️⃣ Each data owner trains an expert locally using a shared anchor model 2️⃣ Expert modules from different data owners merge into a single MoE without joint training 3️⃣ At inference, you can control which expert modules along with their data serve particular users…
Can data owners & LM developers collaborate to build a strong shared model while each retaining data control? Introducing FlexOlmo💪, a mixture-of-experts LM enabling: • Flexible training on your local data without sharing it • Flexible inference to opt in/out your data…
Can data owners & LM developers collaborate to build a strong shared model while each retaining data control? Introducing FlexOlmo💪, a mixture-of-experts LM enabling: • Flexible training on your local data without sharing it • Flexible inference to opt in/out your data… https://t.co/Vnaaq6c6If
📑Paper: allenai.org/papers/FlexOlmo ✍️Blog: allenai.org/blog/flexolmo 💻Code: github.com/allenai/FlexOl… 🤗Models: huggingface.co/allenai/FlexOl… This work would not have been possible w/o @AkshitaB93 @notkevinfarhat @Muennighoff @epwalsh @jacobcares Dustin Schwenk @ShayneRedford…
Big paper smell
I’m thrilled to share RewardBench 2 📊— We created a new multi-domain reward model evaluation that is substantially harder than RewardBench, we trained and released 70 reward models, and we gained insights about reward modeling benchmarks and downstream performance!
It's here!!!! Led by @saumyamalik44, RewardBench 2 is more difficult, covers more domains, and has stronger correlation with downstream reward modeling use cases like PPO and best-of-n sampling. If you happen to be hiring students next year, keep an eye out for her application 👀
It's here!!!! Led by @saumyamalik44, RewardBench 2 is more difficult, covers more domains, and has stronger correlation with downstream reward modeling use cases like PPO and best-of-n sampling. If you happen to be hiring students next year, keep an eye out for her application 👀 https://t.co/qDlQGyIJ37
RewardBench 2 is here! We took a long time to learn from our first reward model evaluation tool to make one that is substantially harder and more correlated with both downstream RLHF and inference-time scaling.
Revoking visas to Chinese PhD students is economically shortsighted and inhumane. Most Chinese PhD students stay in the U.S. after graduation (first image, stats from 2022). They're staying and building technology in the U.S., not taking it to China. Immigrant students create…
We’re live on Reddit! Ask us Anything about our OLMo family of models. We have six of our researchers on hand to answer all your questions.
they should facilitate more public tours of stuff. i want to see the new hospital. i want to see the innards of a cruise ship. let me into the docks. anywhere. let me see it
Excited to be at #NAACL2025 in Albuquerque this week! I'll be presenting "OLMES: A Standard for Language Model Evaluations" (arxiv.org/abs/2406.08446)! Work done with my wonderful collaborators at @allen_ai ❤️
Heading to NAACL? With "verification being the key to AI" you should go to the poster session Friday, 9-10:30am to chat with my star colleagues @valentina__py + @jacobcares about RewardBench (and really RewardBench 2, evaluation, and reward models in post-training).
Happening now in hall 2, poster # 492!
Happening now in hall 2, poster # 492! https://t.co/91W2qvDfR0
刘三 @liu_san40433
2 Followers 863 Following
Akari Asai @AkariAsai
19K Followers 870 Following Incoming Assistant Professor @SCSatCMU & research scientist @allen_ai. akariasai @ 🦋
Nachman @nachmanks331
32 Followers 2K Following
Jane @19ohI9uMsHS998
34 Followers 1K Following
Eaglomarf @Eaglomarf8374
25 Followers 867 Following
seth bloomberg @bloomberg_seth
4K Followers 3K Following Investment Partner @UnsupervisedCap | prev: @CrucibleLabs, research @MessariCrypto | Woo Pig 🐽
Tworruif @Tworruif035959
47 Followers 932 Following
degen_bobo 🤖💎 @agostino90
747 Followers 5K Following
Qinyuan Ye @qinyuan_ye
2K Followers 2K Following ☁️ Research Scientist @SFResearch | 🐾 Teaching machines to be versatile and curious. | Prev @nlp_usc
Si Wu 🌊 @siwu_nlp
313 Followers 224 Following PhD candidate working on natural language processing and computational social science @Northeastern. Previously @UCSanDiego. She/her.
Scott McCrae @scottymccrae
182 Followers 1K Following superintelligence, post-training @Meta. helping machines learn :). former founder, @Dropbox, @berkeley_ai
Ao Qu @ao_qu18465
69 Followers 153 Following PhD at @MIT @mitidss @medialab | Building self-evolving AI & multisensory agents
Branson Johns @JohnsBrans58307
29 Followers 2K Following
Dratriv @Dratriv5428306
26 Followers 1K Following
rust reeve @gorklonreeve
128 Followers 8K Following CEO, Chief Designer of SpaceX*1 CEO &product architect of Tesla🚀🚗
Wanru Zhao @Renee42581826
2K Followers 3K Following PhD Student @Cambridge_Uni; Visiting @VectorInst; Intern @MSFTResearch | Prev: @AWS AI Lab | Do not go gentle into that good night 🧗 | https://t.co/MOPcMcPqcc
Aashish Reddy @_AashishReddy
896 Followers 1K Following 20 | Maths & Philosophy @ University of Warwick
Fuuinrui @Fuuinrui201358
16 Followers 609 Following
Teng Xiao @TengX6
105 Followers 600 Following Postdoc at the Allen Institute for AI @allen_ai and @uwnlp. Machine Learning and Reinforcement Learning
Sonia Lueilwitz @SLueilwitz38929
18 Followers 2K Following
Negin Raoof @NeginRaoof_
887 Followers 401 Following Ph.D. student @Berkeley_EECS advised by @AlexGDimakis Ex: SWE @microsoft, collaborator @PyTorch
Macie Wyman @macie40093
16 Followers 2K Following
Johan Obando-Ceron �... @johanobandoc
2K Followers 4K Following Graduate student @Mila_Quebec @UMontrealDIRO | RL/Deep Learning/AI | De Cali/Colombia pal’ Mundo 🇨🇴 | #JuntosProsperamos⚡#TogetherWeThrive| 🌱🌎
Sayan Chakraborty @shockrobortyy
302 Followers 2K Following building @aadilabsinc (prev: @Qualcomm @BrownUniversity, @paytm, @clarisights)
Joy Wongkamjan @joywwong
296 Followers 783 Following PhD student @ UMD, super interest in human-AI interaction in any language tasks!
Zhiyuan Zeng @ZhiyuanZeng_
603 Followers 225 Following PhD-ing @uwnlp @uwcse | Prev. @Tsinghua_Uni @TsinghuaNLP @princeton_nlp
Amanda Bertsch @abertsch72
2K Followers 872 Following PhD student @LTIatCMU / @SCSatCMU, researching long context + decoding | she/her | also @ abertsch on bsky or https://t.co/L4HBUh0R9f or by email (https://t.co/bsHqwIMFPL)
Subramanyam Sahoo @iamwsubramanyam
197 Followers 4K Following Independent AI Safety researcher, M. Tech x Summa Cum Laude @NITHamirpurHP. BASIS Fellow @UCBerkeley, RA @HarvardAISafety. Get Published or Die Trying.
Michael Ryan @michaelryan207
2K Followers 1K Following Incoming PhD Student @stanfordnlp || Working on DSPy 🧩 || Prev @GeorgiaTech @Microsoft @SnowflakeDB
Uma Gunturi @gunturiuma1
353 Followers 2K Following she/her, MLE @IBM Watsonx, affiliated to @IBMResearch, grad @virginia_tech, previously @midasIIITD, @gtcomputing • NLP • HCI Researcher • Opinions are my own!
Tyler Romero @tyleraromero
353 Followers 1K Following https://t.co/rzQWtHMY6E; language modeling research and engineering @allen_ai
Jonathan Frankle @jefrankle
20K Followers 734 Following Chief AI Scientist @databricks via MosaicML.
SatoshiSpark @DayJosefin78169
6 Followers 244 Following 💥 Unlock Massive Daily Potential! Earn 50-100000 USDT daily. Secure platform, rapid resu lts. Farm Smarter, Safer, Faster Now! 💰⚡🛡️
Joanna @AshleyAlle53152
12 Followers 269 Following I’m not afraid to show my true self. Looking for someone who appreciates authenticity as much as I do.
Nishant Balepur @NishantBalepur
765 Followers 508 Following Intern @allen_ai, visitor @nyuniversity, and PhD-esperate @UofMaryland. Aligning and evaluating more helpful #LLMs. Prev @adobe @UofIllinois
Matt Reichel @MattReiche61883
143 Followers 4K Following
James Morrison Rubin @import_jmr
7K Followers 6K Following Product Lead | Google Gemini Prev: Launched @aws Trainium, @alexa99 Echo Show 5 Tweets are my own. Retweets are not endorsements. Joyful Learning Machines
Pasquale Minervini @PMinervini
9K Followers 5K Following Research in ML/NLP at the U of Edinburgh (tenured faculty @InfAtEd @EdinburghNLP), Co-Founder @Miniml_AI, @ELLISforEurope Scholar, https://t.co/5dUI3EFexo
Itay Nakash @itay__nakash
182 Followers 479 Following Research Scientists @ IBM | AI Safety | Agents, LLMs & NLP
Zilu Tang (Peter) @Zilu_Tang_Peter
233 Followers 629 Following Boston University NLP PhD @llamagrp. Researching interpretability and generalization of language models. Made in China. (Biased) Opinions are my own.
Yoo Yeon Sung@ACL2025 @YooYeonSung1
397 Followers 569 Following PhD Student @umdclip @iSchoolUMD; Human-AI Alignment, human-interactable NLP, LLM evaluation, benchmark creation, misinformation
Melissa Pan @melissapan
2K Followers 549 Following CS PhD @UCBerkeley Sky Lab 🐻 Systems & AI & Sustainability 🌍 Prev: @google, @ibm, @CarnegieMellon🐕🦺, @UofT🇨🇦
Omar Khattab @lateinteraction
25K Followers 3K Following Asst professor @MIT EECS & CSAIL (@nlp_mit). Author of https://t.co/VgyLxl0oa1 and https://t.co/ZZaSzaRaZ7 (@DSPyOSS). Prev: CS PhD @StanfordNLP. Research @Databricks.
Foundation for Americ... @JoinFAI
14K Followers 59 Following A think tank focused on advancing technology, talent, and ideas that support a better, freer, and more prosperous future.
Tim O'Brien @_TimOBrien
1K Followers 886 Following Technology law and policy | Lecturer @INSEAD | @PurdueEngineers, @KelloggSchool, @UWSchoolofLaw
Neil Band @neilbband
1K Followers 706 Following PhD student @StanfordAILab @StanfordNLP @Stanford advised by Tatsunori Hashimoto and Tengyu Ma. Prev: @OATML_Oxford @CompSciOxford
Orion Weller @orionweller
2K Followers 951 Following PhD student @jhuclsp interning @AIatMeta FAIR. Prev intern @GoogleDeepMind, @samaya_ai, @allen_ai. Research: LLMs, RAG, and IR
derek guy @dieworkwear
1.4M Followers 956 Following Menswear writer. Editor at @putthison. Bylines at The New York Times, The Financial Times, Politico, Esquire, and Mr. Porter
Governor Newsom Press... @GovPressOffice
629K Followers 1K Following Official account for the press office of @CAGovernor Gavin Newsom #CaliforniaForAll
Zhiqing Sun @EdwardSun0909
19K Followers 1K Following Agents @Meta MSL TBD Lab. previously posttraining research @OpenAI train LLMs to do things: deep research, chatgpt agent, etc. CS PhD @LTIatCMU
Harry Sisson @harryjsisson
329K Followers 3K Following 23 | Democrat | @harryjsisson (1 million+) on Tik Tok | Email: [email protected]
Ranjay Krishna @RanjayKrishna
6K Followers 435 Following Assistant Professor @ University of Washington, Co-Director of RAIVN lab (https://t.co/f0BWKyjoeA), Director of PRIOR team (https://t.co/l9RzTesMSM)
Ion Stoica @istoica05
5K Followers 20 Following Professor at UC Berkeley, co-founder of Databricks, Anyscale, LMArena, Conviva.
Eric @ericmitchellai
9K Followers 562 Following co-lead @openai post-training frontiers team w/ the great @yanndubs; I like ai and music and some other things
Katie Wilson for Seat... @wilsonformayor
2K Followers 312 Following Progressive coalition-builder, mom, running for Mayor of Seattle. It's time for new leadership. Come join the team!
The Needling @TheNeedling
21K Followers 3K Following Seattle's Only Real Fake News (Satire/Parody) Have a hot headline tip? Send it to [email protected]
seth bloomberg @bloomberg_seth
4K Followers 3K Following Investment Partner @UnsupervisedCap | prev: @CrucibleLabs, research @MessariCrypto | Woo Pig 🐽
Qinyuan Ye @qinyuan_ye
2K Followers 2K Following ☁️ Research Scientist @SFResearch | 🐾 Teaching machines to be versatile and curious. | Prev @nlp_usc
Si Wu 🌊 @siwu_nlp
313 Followers 224 Following PhD candidate working on natural language processing and computational social science @Northeastern. Previously @UCSanDiego. She/her.
Kyle Wiggers @Kyle_L_Wiggers
67K Followers 6K Following Ai2 Comms Lead | [email protected] | Pronouns: he/him
Scott McCrae @scottymccrae
182 Followers 1K Following superintelligence, post-training @Meta. helping machines learn :). former founder, @Dropbox, @berkeley_ai
Lakshya A Agrawal @LakshyAAAgrawal
2K Followers 2K Following AI PhD @ UC Berkeley | GEPA Creator (https://t.co/EdPqvzj7k4) | Created https://t.co/YxPZsXZJeS | Past: AI4Code Research Fellow @MSFTResearch | Hobbyist Saxophonist
Ao Qu @ao_qu18465
69 Followers 153 Following PhD at @MIT @mitidss @medialab | Building self-evolving AI & multisensory agents
Ziming Liu @ZimingLiu11
12K Followers 788 Following PhD student@MIT, AI for Physics/Science, Science of Intelligence & Interpretability for Science
Kimi.ai @Kimi_Moonshot
53K Followers 100 Following Built by Moonshot AI to empower everyone to be superhuman.
Danqi Chen @danqi_chen
17K Followers 789 Following Associate professor @princetonPLI @PrincetonCS. Previously: @facebookai, @stanfordnlp, @Tsinghua_Uni https://t.co/hLXgShMIPE
Wanru Zhao @Renee42581826
2K Followers 3K Following PhD Student @Cambridge_Uni; Visiting @VectorInst; Intern @MSFTResearch | Prev: @AWS AI Lab | Do not go gentle into that good night 🧗 | https://t.co/MOPcMcPqcc
Helen Toner @hlntnr
29K Followers 1K Following AI, national security, China. Part of the founding team at @CSETGeorgetown (opinions my own). Author of Rising Tide on substack: https://t.co/LKAoyL00iB
Jiafei Duan@CoRL2025 @DJiafei
4K Followers 946 Following Robotics & AI PhD student @uwcse|Researcher @allen_ai |Ex-@NVIDIA 🇸🇬 Robotics Foundation Models
Zohran Kwame Mamdani @ZohranKMamdani
637K Followers 985 Following Democratic Nominee for Mayor of NYC. Assemblymember. Running to freeze the rent, make buses fast + free, and deliver universal childcare. Democratic Socialist.
Azalia Mirhoseini @Azaliamirh
15K Followers 529 Following Asst. Prof. of CS at Stanford, Google DeepMind. Prev: Anthropic, Google Brain. Co-Creator of MoEs, AlphaChip, Test Time Scaling Laws.
Anna Goldie @annadgoldie
7K Followers 137 Following Senior Staff Research Scientist at @GoogleDeepMind. Prev: @AnthropicAI, @StanfordNLP, @MIT. AlphaChip co-lead.
Teng Xiao @TengX6
105 Followers 600 Following Postdoc at the Allen Institute for AI @allen_ai and @uwnlp. Machine Learning and Reinforcement Learning
Mimansa Jaiswal @MimansaJ
4K Followers 5K Following Currently RS @aiatmeta | LLMs/SLMs Post Training | Data, Evals, Rewards and Agentic System Orchestration
Joy Wongkamjan @joywwong
296 Followers 783 Following PhD student @ UMD, super interest in human-AI interaction in any language tasks!
Alexander Holden Mill... @alex_h_miller
1K Followers 711 Following Research Engineering Manager at FAIR @MetaAI
Zhiyuan Zeng @ZhiyuanZeng_
603 Followers 225 Following PhD-ing @uwnlp @uwcse | Prev. @Tsinghua_Uni @TsinghuaNLP @princeton_nlp
Amanda Bertsch @abertsch72
2K Followers 872 Following PhD student @LTIatCMU / @SCSatCMU, researching long context + decoding | she/her | also @ abertsch on bsky or https://t.co/L4HBUh0R9f or by email (https://t.co/bsHqwIMFPL)
Nishant Balepur @NishantBalepur
765 Followers 508 Following Intern @allen_ai, visitor @nyuniversity, and PhD-esperate @UofMaryland. Aligning and evaluating more helpful #LLMs. Prev @adobe @UofIllinois
Michael Ryan @michaelryan207
2K Followers 1K Following Incoming PhD Student @stanfordnlp || Working on DSPy 🧩 || Prev @GeorgiaTech @Microsoft @SnowflakeDB
Tyler Romero @tyleraromero
353 Followers 1K Following https://t.co/rzQWtHMY6E; language modeling research and engineering @allen_ai
Trends for United States
You might like
