Kevin Lin 林冠言 @nlpkevinl
phd student @berkeleynlp @ucbrise, formerly @ai2_allennlp people.eecs.berkeley.edu/~kevinlin/ Joined September 2017-
Tweets59
-
Followers421
-
Following332
-
Likes927
LLMs can use complex instructions - why can’t retrieval models? We build FollowIR, a training/test set of real-world human retrieval instructions. Our FollowIR-7B is the best IR model for instruct-following, even beating @cohere @OpenAI retrievers 🤯 📝 arxiv.org/abs/2403.15246
Protein language models (pLMs) can give protein sequences likelihood scores, which are commonly used as a proxy for fitness in protein engineering. But what do likelihoods encode? In a new paper (w/ @JacobSteinhardt) we find that pLM likelihoods have a strong species bias! 1/
Do brain representations of language depend on whether the inputs are pixels or sounds? Our @CommsBio paper studies this question from the perspective of language timescales. We find that representations are highly similar between modalities! rdcu.be/dACh5 1/8
release day release day! OLMo 1b + 7b out today 🥳 and 65b coming soon... With OLMo, we are really focused on advancing the study of LLMs. We release **everything**, from toolkit to create its training dataset (dolma) to training & inference code. More details in thread 🧵
LLMs can facilitate student cheating, spread misinformation on the web, and even poison future training datasets. Today, we’re releasing Ghostbuster, a state-of-the-art method for detecting LLM-generated text. Paper: arxiv.org/abs/2305.15047 Try it: ghostbuster.app
It's not the first time! A dream team of @enfleisig (human eval expert), Adam Lopez (remembers the Stat MT era), @kchonyc (helped end it), and me (pun in title) are here to teach you the history of scale crises and what lessons we can take from them. 🧵arxiv.org/abs/2311.05020
For this week’s NLP Seminar, we are thrilled to host @realJessyLin to talk about "Toward Interactive Agents That Use Language"! When: 11/09 Thurs 11am PT (UTC-8) Non-Stanford affiliates registration form (closed at 9am PT on the talk day): forms.gle/UEfq2NeDkZqY2L…
1/ Greetings, loyal followers! I come to you today bearing news of my latest preprint, now on arxiv arxiv.org/abs/2311.01491. In it, we take a deep dive on diffusion models for generating molecular atomic geometries—i.e., the 3D positions of atoms in a molecule.
Introducing MemGPT 📚🦙 a method for extending LLM context windows. Inspired by OS mem management, it provides an infinite virtualized context for fixed-context LLMs. Enables perpetual chatbots & large doc QA. 🧵1/n Paper: arxiv.org/abs/2310.08560 GitHub: github.com/cpacker/memgpt
Dear ACL community, ACL is considering multiple proposals to change its anonymity period policy. It seeks immediate feedback from the community about the proposed changes. Please add your voice until Friday, September 22nd (AOE): aclweb.org/portal/content… #NLProc
Hi all prospective grad students! Our Equal Access to Application Assistance (EAAA) program for @Berkeley_EECS is now accepting applications! Any PhD applicant to @Berkeley_EECS can submit their application for feedback by Oct 8 2023: forms.gle/dHq2EPGrkkdcSu…
New paper at #acl2023nlp! "Modular Visual Question Answering via Code Generation" With @medhini_n @kushaltk1248 @KevinYa33964384 @NagraniArsha @CordeliaSchmid @andyzengtweets @trevordarrell Dan Klein (@berkeley_ai/@GoogleAI)! 📜 arxiv.org/abs/2306.05392 💻github.com/sanjayss34/cod…
Real documents aren’t just plain text – they also have visuospatial layout! Humans generalize flexibly to new layouts (e.g., 1 or 2 column papers). Can LLMs? Our #ACL2023 Findings paper shows current layout-infused models struggle on unseen layouts. arxiv.org/abs/2306.01058 1/6
We're looking forward to welcoming @sea_snell for a presentation on context distillation on December 5th! Big thanks to @_joaogui1 for inviting another great speaker to share their work with us. 🎉 Sign up for the event here! zoom.us/webinar/regist…
Akari Asai @AkariAsai
11K Followers 650 Following Ph.D. student @uwcse & @uwnlp. NLP. IBM Ph.D. fellow (2022-2023). Meta student researcher (2023-) . ☕️ 🐕 🏃♀️🧗♀️🍳
Luca Soldaini 🎀 @soldni
6K Followers 1K Following I like tokens! Lead for OLMo data team at @allen_ai (Dolma 🍇), OSS is fun, @QueerInAI organizer 🤖☕️🍕they/them (views mine, not my employer’s)
Ana Marasović @anmarasovic
4K Followers 604 Following Asst prof @UUtah · Ex @allen_ai @uwnlp postdoc @HD_NLP PhD · she/her 🇭🇷
Kayo Yin @kayo_yin
8K Followers 560 Following PhD student @berkeley_ai @berkeleynlp working on interpretability and signed languages. Former @msftresearch @deepmind @carnegiemellon @polytechnique. 🇫🇷🇯🇵
Tim Dettmers @Tim_Dettmers
29K Followers 821 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.
Sameer Singh @sameer_
7K Followers 2K Following Cofounder @SpiffyAI and Assoc Prof at @UCIrvine, working on reliable LLMs, explanations for AI+ML, adversaries for NLP, and debugging/evaluation.
Alexis Ross @alexisjross
3K Followers 887 Following phd-ing @MIT_CSAIL, interested in NLP for education | formerly nlp @allen_ai, comp sci & philosophy @harvard ‘20
Bill Yuchen Lin 🤖 @billyuchenlin
6K Followers 2K Following Research @allen_ai. I evaluate (multi-modal) LLMs, build agents, and study the science of LLMs. Previously: @GoogleAI & @MetaAI FAIR @nlp_usc
Sam Bowman @sleepinyourhat
35K Followers 3K Following AI alignment + LLMs at NYU & Anthropic. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Sherry Tongshuang Wu @tongshuangwu
5K Followers 1K Following Assist. Prof @SCSatCMU , CS PhD @uwcse. HCI+AI, map general-purpose models to specific use cases! prev. intern @MSFTResearch @GoogleAI @Apple. She/her.
Machel Reid @machelreid
2K Followers 1K Following Research Scientist @GoogleDeepMind Working on LLMs on the Gemini Team; did gemini 1.5 pro
Ofir Press @OfirPress
10K Followers 3K Following I build tough benchmarks for LMs and then I get the LMs to solve them. Postdoc @Princeton. PhD from @nlpnoah @UW. Ex-visiting researcher @MetaAI & @MosaicML.
Lucy Li @lucy3_li
4K Followers 2K Following @UCBerkeley PhD student + @allen_ai. Human-centered #NLProc, computational social science, AI fairness. she/her. https://t.co/rtSSUhWQnL
Gabriel Ilharco @gabriel_ilharco
4K Followers 1K Following Building cool things @xAI. Prev. PhD at UW, Google AI
Zhaofeng Wu @zhaofeng_wu
1K Followers 171 Following PhD student @MIT_CSAIL | Previously @ai2_allennlp | MS'21 BS'19 BA'19 @uwnlp
Niloofar (Fatemeh) Mi.. @niloofar_mire
4K Followers 1K Following Postdoc @uwcse-@uwnlp, Ph.D. from @ucsd_cse /Privacy, ML, NLP, @winlpworkshop chair, @MSFTResearch - Semantic Machines
Kawin Ethayarajh @ethayarajh
3K Followers 728 Following PhD student @StanfordAILab @stanfordnlp Working on machine learning under human incentives.
David Chu @davidchuyaya
62 Followers 59 Following PhD student in distributed systems @UCBerkeley, advised by @joe_hellerstein and @siobhcroo
MIke @MIke71530700
1 Followers 100 Following
Daniel King @danielking36
499 Followers 626 Following Machine Learning Engineer @mosaicml | previously @allen_ai @semanticscholar | @harveymudd | he/him | Black lives matter.
Nicholas Lourie @NickLourie
133 Followers 243 Following I build things. 🤖 Doing a PhD at @nyuniversity (@CILVRatNYU) on better empirical methods for deep learning and data science. Advised by @kchonyc and @hhexiy.
Erik Jones @ErikJones313
263 Followers 137 Following CS PhD Student at @berkeley_ai working on automated evaluation for LLMs
Tianjun Zhang @tianjun_zhang
1K Followers 765 Following Project Lead of RAFT, Gorilla, Berkeley Function Calling Leaderboard, and member of LiveCodeBench, PhD student at Berkeley-AI-Research
laura @laura012747755
421 Followers 5K Following
Zineng Tang @ZinengTang
1K Followers 569 Following PhD in @Berkeley_ai and @BerkeleyNLP. Previously @UNCNLP and @MSFTResearch.
Zhanghao Wu @Michaelvll1
475 Followers 281 Following Building SkyPilot @skypilot_org and Vicuna @lmsysorg | PhD student @Berkeley_EECS @ucbrise
Kevin Mathew T @kevinmathew_
29 Followers 684 Following
Alexander Wan @alexwan55
473 Followers 944 Following CS at Berkeley; @BerkeleyML @BerkeleyNLP; NLP research
LALITH @LALITH_99999
40 Followers 247 Following
Alex Pan @aypan_17
146 Followers 115 Following Berkeley ML/AI PhD Student working on studying and aligning LLMs
Xuhui Zhou @nlpxuhui
688 Followers 430 Following PhD student @LTIatCMU. Previously, @GeorgiaTech, @UWNLP, and @Apple. Social Intelligence in language +X. He/Him.🐳
Ziyang Luo @ChiYeung_Law
586 Followers 2K Following 👨💻CS PhD Candidate @HKBU_NLP 🇭🇰 📘Research on Code Intelligence and LLM/LMMs. 📖Ex Study @UU_University 🇸🇪 💡Ex Intern @MSFTResearch & @WizardLM_AI
Eve Fleisig @enfleisig
375 Followers 332 Following PhD student @Berkeley_EECS | Princeton ‘21 | NLP, ethical + equitable AI, and linguistics enthusiast
最伟大的人 @eengw0755
5 Followers 57 Following
mukesh kumar @mukeshkr165
51 Followers 2K Following Dropped out of college in just two months with zero credits taken(lol)
Charlie Cheng-Jie Ji @charlie_jcj02
75 Followers 506 Following Gorilla LLM, CS & DS @ UC Berkeley, Data 100 Lead TA, Working towards LLM Tool Use, AI safety
Arnav Gudibande @arnavg_
471 Followers 298 Following Research Engineer @perplexity_ai | prev MS @berkeley_ai @berkeleyNLP
Zhiheng LYU @ZhihengLyu
54 Followers 203 Following RA@Berkeley NLP; Yr4-CS@HKU; Currently Seeking PhD Position on 24Fall
Nishant Subramani @nsubramani23
579 Followers 2K Following PhD student at @LTIatCMU // Prev: Predoctoral Researcher at @allen_ai in #NLProc // @BVB supporter // he/him
Woosuk Kwon @woosuk_k
2K Followers 351 Following PhD student at @Berkeley_EECS building @vllm_project
Fei Wang @fwang_nlp
920 Followers 2K Following PhD candidate @USC. PhD Fellow @Amazon. Responsible LLM.
Simon Mo @simon_mo_
339 Followers 303 Following Working on System for ML @ucbrise. Happy to get in touch: https://t.co/ACIbL2HqBr at https://t.co/FWFXdUDDMp (ex-@anyscalecompute)
Nandan Thakur @nandan__thakur
2K Followers 2K Following PhD @uwaterloo | Author of BEIR, Upcoming: TREC-RAG | Prev: @GoogleAI, @UKPLab | Undergrad @BitsPilaniGoa | Interested in IR and NLP 🇨🇦🇩🇪🇮🇳
Danny To Eun Kim @TEKnologyy
327 Followers 930 Following PhD-ing @LTIatCMU working with @841io. MEng @ai_ucl | NLP & IR
Lianmin Zheng @lm_zheng
4K Followers 439 Following CS Ph.D. @ UC Berkeley. Creator of Alpa, Vicuna, and Chatbot Arena. @lmsysorg
Julien Piet @julientpiet
65 Followers 130 Following PhD Student at UC Berkeley in Computer Security X2015 & Corps des Mines
Li Lianjie Anthony @LianJieAnthony
54 Followers 336 Following Medicine + Programming + Red Cross = My Life
Manish Pandey @Manish_GenAI
244 Followers 4K Following Research Engineer Stealth #GraphML, #GeometricDL, #3DComputerVision, #DiffusionModels, #Generative AI #ComputerVision,#ML ,#RL, #LLM, #MultiModal Fusion
Joey Gonzalez @profjoeyg
3K Followers 271 Following Professor @UCBerkeley, co-director of @LMSysorg, and co-founder @RunLLM
Akari Asai @AkariAsai
11K Followers 650 Following Ph.D. student @uwcse & @uwnlp. NLP. IBM Ph.D. fellow (2022-2023). Meta student researcher (2023-) . ☕️ 🐕 🏃♀️🧗♀️🍳
Luca Soldaini 🎀 @soldni
6K Followers 1K Following I like tokens! Lead for OLMo data team at @allen_ai (Dolma 🍇), OSS is fun, @QueerInAI organizer 🤖☕️🍕they/them (views mine, not my employer’s)
Ana Marasović @anmarasovic
4K Followers 604 Following Asst prof @UUtah · Ex @allen_ai @uwnlp postdoc @HD_NLP PhD · she/her 🇭🇷
AK @_akhaliq
310K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo follow on Hugging Face: https://t.co/q2Qoey80Gx
Victor Zhong @hllo_wrld
4K Followers 450 Following ML+NLP assistant prof @UWCheritonCS. Formerly @MSFTResearch @MetaAI, @SFResearch via @MetamindIO, @uwnlp, @StanfordNLP, @eceuoft.
Kayo Yin @kayo_yin
8K Followers 560 Following PhD student @berkeley_ai @berkeleynlp working on interpretability and signed languages. Former @msftresearch @deepmind @carnegiemellon @polytechnique. 🇫🇷🇯🇵
Tim Dettmers @Tim_Dettmers
29K Followers 821 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.
Yoav Artzi @yoavartzi
13K Followers 163 Following Research/prof @cs_cornell + @cornell_tech🚡 / https://t.co/9YnWry7yHs / https://t.co/3VmRSyYm2d / asso. faculty director @arxiv / building https://t.co/f9QkzO5kaC
Greg Durrett @gregd_nlp
6K Followers 752 Following CS professor at UT Austin. I do NLP most of the time. he/him
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Sameer Singh @sameer_
7K Followers 2K Following Cofounder @SpiffyAI and Assoc Prof at @UCIrvine, working on reliable LLMs, explanations for AI+ML, adversaries for NLP, and debugging/evaluation.
Colin Raffel @colinraffel
30K Followers 655 Following nonbayesian parameterics, sweet lessons, and random birds. Friend of @srush_nlp
Alexis Ross @alexisjross
3K Followers 887 Following phd-ing @MIT_CSAIL, interested in NLP for education | formerly nlp @allen_ai, comp sci & philosophy @harvard ‘20
Naomi Saphra @nsaphra
7K Followers 1K Following Waiting on a robot body. ML/NLP. All opinions are universal and held by both employers and family. Same username on every lifeboat off this sinking ship.
Bill Yuchen Lin 🤖 @billyuchenlin
6K Followers 2K Following Research @allen_ai. I evaluate (multi-modal) LLMs, build agents, and study the science of LLMs. Previously: @GoogleAI & @MetaAI FAIR @nlp_usc
Sam Bowman @sleepinyourhat
35K Followers 3K Following AI alignment + LLMs at NYU & Anthropic. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Weijia Shi @WeijiaShi2
5K Followers 968 Following PhD student @uwcse @uwnlp | Visiting Researcher @MetaAI | Undergrad @CS_UCLA | https://t.co/eLBQmgkvym
Jacob Andreas @jacobandreas
14K Followers 958 Following Teaching computers to read. Assoc. prof @MITEECS / @MIT_CSAIL (he/him). https://t.co/5kCnXHjtlY https://t.co/2A3qF5vdJw
David Chu @davidchuyaya
62 Followers 59 Following PhD student in distributed systems @UCBerkeley, advised by @joe_hellerstein and @siobhcroo
Lukasz Kaiser @lukaszkaiser
7K Followers 47 Following
Daniel King @danielking36
499 Followers 626 Following Machine Learning Engineer @mosaicml | previously @allen_ai @semanticscholar | @harveymudd | he/him | Black lives matter.
Nicholas Lourie @NickLourie
133 Followers 243 Following I build things. 🤖 Doing a PhD at @nyuniversity (@CILVRatNYU) on better empirical methods for deep learning and data science. Advised by @kchonyc and @hhexiy.
Brandon McKinzie @mckbrando
2K Followers 2K Following Multimodal LLMs @Apple. Prev: Physics/CS @UCBerkeley.
Mihir Patel @mvpatel2000
3K Followers 385 Following Research Engineer @MosaicML | cs, math bs/ms @Stanford
Erik Jones @ErikJones313
263 Followers 137 Following CS PhD Student at @berkeley_ai working on automated evaluation for LLMs
Tianjun Zhang @tianjun_zhang
1K Followers 765 Following Project Lead of RAFT, Gorilla, Berkeley Function Calling Leaderboard, and member of LiveCodeBench, PhD student at Berkeley-AI-Research
Zineng Tang @ZinengTang
1K Followers 569 Following PhD in @Berkeley_ai and @BerkeleyNLP. Previously @UNCNLP and @MSFTResearch.
Zhanghao Wu @Michaelvll1
475 Followers 281 Following Building SkyPilot @skypilot_org and Vicuna @lmsysorg | PhD student @Berkeley_EECS @ucbrise
Nous Research @NousResearch
18K Followers 29 Following The AI Accelerator Company. https://t.co/vrD0aDJeto
Karina Nguyen @karinanguyen_
12K Followers 650 Following AI research & eng @AnthropicAI, prev. intern @nytimes, @square, @dropbox
Davis Blalock @davisblalock
12K Followers 165 Following Research scientist + first hire @MosaicML. @MIT PhD. I write + retweet technical machine learning content. If you write a thread about your paper, tag me for RT
Alexander Wan @alexwan55
473 Followers 944 Following CS at Berkeley; @BerkeleyML @BerkeleyNLP; NLP research
Jade @Euclaise_
2K Followers 350 Following ⋅ Video game statistician ⋅ Soclib cyberanarchist? ⋅ C, Plan 9, LLMs, etc ⋅ Researcher w/ @NousResearch ⋅ she/they
Alex Pan @aypan_17
146 Followers 115 Following Berkeley ML/AI PhD Student working on studying and aligning LLMs
typedfemale @typedfemale
23K Followers 477 Following a really exciting new account "have you ever though you might be like scott alexander? very smart, but can't do math" - anon
Conference on Languag.. @COLM_conf
2K Followers 6 Following https://t.co/GhGCMEoa4A Abstract submission: March 22, 2024
Xuhui Zhou @nlpxuhui
688 Followers 430 Following PhD student @LTIatCMU. Previously, @GeorgiaTech, @UWNLP, and @Apple. Social Intelligence in language +X. He/Him.🐳
jack morris @jxmnop
11K Followers 761 Following getting my phd in nlp @cornell_tech 🚠 // academic optimist // tweeting from the snack aisle at trader joes
Arnav Gudibande @arnavg_
471 Followers 298 Following Research Engineer @perplexity_ai | prev MS @berkeley_ai @berkeleyNLP
kache (dingboard.com) @yacineMTB
53K Followers 3K Following i'm a swe. go to https://t.co/pWRBfY8kn2 - AI image editing IN YOUR BROWSER! follow to watch a self funded founder beat VC backed AI startups with @dingboard_
Woosuk Kwon @woosuk_k
2K Followers 351 Following PhD student at @Berkeley_EECS building @vllm_project
Nandan Thakur @nandan__thakur
2K Followers 2K Following PhD @uwaterloo | Author of BEIR, Upcoming: TREC-RAG | Prev: @GoogleAI, @UKPLab | Undergrad @BitsPilaniGoa | Interested in IR and NLP 🇨🇦🇩🇪🇮🇳
Lisa Dunlap @lisabdunlap
497 Followers 154 Following PhD student & vibe curator @berkeley_ai and Sky Computing Lab -- for the love of god look at your data
Maithra Raghu @maithra_raghu
17K Followers 476 Following Cofounder and CEO @Samaya_AI. Formerly Research Scientist Google Brain (@GoogleAI), PhD in ML @Cornell.
CLS @ChengleiSi
2K Followers 3K Following vibing @stanfordnlp | real AGI is the friends we made along the way
Simon Mo @simon_mo_
339 Followers 303 Following Working on System for ML @ucbrise. Happy to get in touch: https://t.co/ACIbL2HqBr at https://t.co/FWFXdUDDMp (ex-@anyscalecompute)
Marc Marone @ruyimarone
420 Followers 586 Following PhD student at Johns Hopkins @jhuclsp. Previously @microsoft Semantic Machines, @mstranslator, @GeorgiaTech
Lianmin Zheng @lm_zheng
4K Followers 439 Following CS Ph.D. @ UC Berkeley. Creator of Alpa, Vicuna, and Chatbot Arena. @lmsysorg
Dacheng Li @DachengLi177
620 Followers 476 Following Intelligence. PhD @Berkeley_EECS @lmsysorg @ucbrise @berkeley_ai, Prev. @Google @SCSatCMU.
Naman Jain @StringChaos
896 Followers 903 Following CS PhD @UCBerkeley | Projects - R2E, LiveCodeBench, Chatbot-Arena Coding, RAFT, Data Quality | Past: @AWS @MSFTResearch @iitbombay
Chaitanya Malaviya @cmalaviya11
99 Followers 121 Following PhD student at UPenn | currently @GoogleDeepMind
Julie Kallini ✨ @JulieKallini
601 Followers 338 Following CS PhD @StanfordNLP 🌲 Previously: SWE @Meta, Class of '21 @PrincetonCS
Mimansa Jaiswal @MimansaJ
1K Followers 3K Following MoTS @normativeai. Ex @UMichCSE, 2x @MetaAI, @allen_ai | Speech & NLP | Robustness, Data & Annotations, Evaluation & Interpretability in LLMs
Scaled Cognition @ScaledCognition
20 Followers 0 Following
Roma Patel @996roma
2K Followers 506 Following research scientist @deepmind london. language & rl & interpretability & safety. phd @BrownUniversity '22 with ellie pavlick. (she/her)
Risham Sidhu @RishamSidhu
5 Followers 7 Following CS PhD student focusing on grounded dialogue (NLP) at UIUCExcited to be part of 🦙, more to come! ai.meta.com/blog/meta-llam…
I am honored to share that our recent paper won the Outstanding Paper Award in NSDI’24! The paper explores the policy design of our SkyPilot managed spot for @skypilot_org: Can’t Be Late: Optimizing Spot Instance Savings under Deadlines It would not be possible, if it were not…
New paper from @berkeley_ai on Autonomous Evaluation and Refinement of Digital Agents! We show that VLM/LLM-based evaluators can significantly improve the performance of agents for web browsing and device control, advancing sotas by 29% to 75%. arxiv.org/abs/2404.06474 [🧵]
LLMs can use complex instructions - why can’t retrieval models? We build FollowIR, a training/test set of real-world human retrieval instructions. Our FollowIR-7B is the best IR model for instruct-following, even beating @cohere @OpenAI retrievers 🤯 📝 arxiv.org/abs/2403.15246
📢📢Excited to introduce our new work LiveCodeBench! 📈 Live evaluations to ensure fairness and reliability 🔍 Holistic evaluations using 4 code-related scenarios 💡Insights from comparing 20+ code models 🚨🚨We use problem release dates to detect and prevent contamination
The final layer of an LLM up-projects from hidden dim —> vocab size. The logprobs are thus low rank, and with some clever API queries, you can recover an LLM’s hidden dimension (or even the exact layer’s weights). Our new paper is out, a collaboration between lot of friends!
Google presents: Stealing Part of a Production Language Model - Extracts the projection matrix of OpenAI’s ada and babbage LMs for <$20 - Confirms that their hidden dim is 1024 and 2048, respectively - Also recovers the exact hidden dim size of gpt-3.5-turbo…
We know LLMs hallucinate, but what governs what they dream up? Turns out it’s all about the “unfamiliar” examples they see during finetuning Our new paper shows that manipulating the supervision on these special examples can steer how LLMs hallucinate arxiv.org/abs/2403.05612 🧵
Protein language models (pLMs) can give protein sequences likelihood scores, which are commonly used as a proxy for fitness in protein engineering. But what do likelihoods encode? In a new paper (w/ @JacobSteinhardt) we find that pLM likelihoods have a strong species bias! 1/
Do brain representations of language depend on whether the inputs are pixels or sounds? Our @CommsBio paper studies this question from the perspective of language timescales. We find that representations are highly similar between modalities! rdcu.be/dACh5 1/8
Introduce Archer, our latest efforts to develop better RL algorithms for LM agents. This multi-turn RL alg outperforms all baselines significantly and can achieve up to 100X greater sample efficiency comparing to PPO. It was a pleasure to be part of the team.
How can we train LLM Agents, to learn from their own experience autonomously? Introducing ArCHer, a simple (i.e., small change on top of standard RLHF) and effective way of doing so with multi-turn RL 🧵⬇️ Paper: arxiv.org/abs/2402.19446 Website: yifeizhou02.github.io/archer.io/
LLMs struggling to use new tools? Self-verification improves tool generalization of LLMs! w/ amazing collaborators @jaseweston @JaneDwivedi @robertarail @MariaLomeli_ @shangjingbo
🚨New paper!🚨 ToolVerifier. - Method to generalize to new tools - Self-asks contrastive questions to select between best tools and parameter choices - Fine-tuned on self-built synthetic data - 22% performance improvement over few-shot baseline arxiv.org/abs/2402.14158 🧵(1/4)
📢Excited to release the live Berkeley Function-Calling Leaderboard! 🔥 Also debuting openfunctions-v2 🤩 the latest open-source SoTA function-calling model on-par with GPT-4🆕Native support for Javascript, Java, REST! 🫡 Leaderboard: gorilla.cs.berkeley.edu/leaderboard.ht… Blog:…
Paper with my great collaborators @GoogleDeepMind now accepted to CVPR 2024 🥳 We have run A LOT of experiments 💸 to figure out how to train the strongest video-first encoder while handling efficiently hundreds of frames. Check it out for the best recipe to follow👇
Large multimodal models understand images/clips, but what about longer contexts? We propose a memory-efficient approach for training on long videos and show that our 1B model outperforms LLMs used as information-aggregator over large image captioners. arxiv.org/abs/2312.07395
Was this sequence in the training dataset or not?? In new paper, we study why membership inference attacks show *near-random performance* on LLMs!! We also release a Python package for seamless MIA evaluation!! Paper: arxiv.org/abs/2402.07841 Repo: github.com/iamgroot42/mim…
vLLM v0.3.2 is released with support for OLMo and Gemma! github.com/vllm-project/v…
What happens when RAG models are provided with documents that have conflicting information? In our new paper, we study how LLMs answer subjective, contentious, and conflicting queries in real-world retrieval-augmented situations.
Future LLMs---whether they be RAG models, chatbots, or agents--will have to sift through misinformation, SEO text, and conflicting opinions when reading text. Alex led an interesting analysis of how current LLMs handle such conflicts. TLDR: LLMs love relevance, not style.
What happens when RAG models are provided with documents that have conflicting information? In our new paper, we study how LLMs answer subjective, contentious, and conflicting queries in real-world retrieval-augmented situations.
Remember when @bing’s LLM Sydney threatened @marvinvonhagen for tweeting about its prompt? Our paper shows how such unexpected behavior in LLMs emerges from feedback loops and provides recommendations for evaluation to capture feedback effects. 📰: arxiv.org/abs/2402.06627 1/
release day release day! OLMo 1b + 7b out today 🥳 and 65b coming soon... With OLMo, we are really focused on advancing the study of LLMs. We release **everything**, from toolkit to create its training dataset (dolma) to training & inference code. More details in thread 🧵
This was a cool paper: aclanthology.org/2020.emnlp-mai… Does anyone know if this stuff works now? I feel like an idiot talking to my phone when people are around.
Trends for United States
You might like
