Ahmad Beirami @abeirami
Building safe, helpful, and scalable generative AI @Google | ex-{@AIatMeta, @EA, @MIT, @Harvard, @DukeU} | @GeorgiaTech PhD | زن زندگی آزادی | opinions my own mit.edu/~beirami/ {NYC, BOS, YYZ} Joined December 2018-
Tweets2K
-
Followers4K
-
Following2K
-
Likes8K
Nicely put!
Check out Zhaofeng's work from his internship with us! TL;DR A reward model trained on language S preference data could be used to align a language T LLM. This sometimes works even better than using a reward model trained on language T preference data.
Check out Zhaofeng's work from his internship with us! TL;DR A reward model trained on language S preference data could be used to align a language T LLM. This sometimes works even better than using a reward model trained on language T preference data.
Want to train an aligned LM in a new language 🌏 but don’t have preference data for training the reward model (RM)? 💡 Just use a RM for another language: it often works well, sometimes even BETTER than if you had a RM in your target language! 🤯 arxiv.org/abs/2404.12318
We created reviewing guidelines for @COLM_conf. Not intended to automate the committee work, or dictate constraints. But, to inspire a thoughtful reviewing process, for an exciting and impactful program of the highest possible quality. We have a wonderful program committee ❤️
The review committee's job is to point out the flaws in a paper and give constructive feedback to improve the paper. It's not to speculate how the flaws came about!
The review committee's job is to point out the flaws in a paper and give constructive feedback to improve the paper. It's not to speculate how the flaws came about!
Super excited about our upcoming @icmlconf workshop! Stay tuned for updates 🙌 For details: sites.google.com/view/tf2m
Super excited about our upcoming @icmlconf workshop! Stay tuned for updates 🙌 For details: sites.google.com/view/tf2m
We are happy to announce that the Workshop on Theoretical Foundations of Foundation Models will take place @icmlconf in Vienna! For details: sites.google.com/view/tf2m Organizers: @BerivanISIK, @SZiteng, @BanghuaZ, @eaboix, @nmervegurel, @uiuc_aisecure, @abeirami, @sanmikoyejo
Classical mixture models are limited to positive weights and this requires learning very large mixtures! Can we learn (deep) mixtures with negative weights? Answer in our #ICLR2024 spotlight by @loreloc_ Aleks, Martin, Stefan, Nicolas @arnosolin 📜openreview.net/forum?id=xIHi5…
Classical mixture models are limited to positive weights and this requires learning very large mixtures! Can we learn (deep) mixtures with negative weights? Answer in our #ICLR2024 spotlight by @loreloc_ Aleks, Martin, Stefan, Nicolas @arnosolin 📜openreview.net/forum?id=xIHi5… https://t.co/qDglZQitpU
#icml2024 workshops list is up: icml.cc/virtual/2024/e…
Robustness methods 1) augment data with natural/synthetic perturbations and a consistency loss 2) reweight samples to improve generalization (like DRO) We do it differently! We show significant robustness with a simple tweak of the first layer and loss motivated by comms theory.
Robustness methods 1) augment data with natural/synthetic perturbations and a consistency loss 2) reweight samples to improve generalization (like DRO) We do it differently! We show significant robustness with a simple tweak of the first layer and loss motivated by comms theory.
📢📢📢 Late post, but here we go...! I am thrilled to announce that our work on 𝙚𝒏𝙝𝒂𝙣𝒄𝙞𝒏𝙜 𝙤𝒖𝙩-𝙤𝒇-𝒅𝙞𝒔𝙩𝒓𝙞𝒃𝙪𝒕𝙞𝒐𝙣 𝙧𝒐𝙗𝒖𝙨𝒕𝙣𝒆𝙨𝒔 of deep neural networks has been accepted to 𝘼𝑰𝙎𝑻𝘼𝑻𝙎 2024!
We're organizing the first summer course on LLMs in Armenia this year! We'll cover the foundations of LLMs from first principles through lectures from a great lineup of speakers and hands-on practice sessions. If interested, reach out directly or go to armllm.github.io/2024/.
Gautam Kamath @thegautamkamath
44K Followers 505 Following Assistant Prof of CS @UWaterloo, Faculty @VectorInst, Canada @CIFAR_News AI Chair. Co-EiC @TmlrOrg. I lead @TheSalonML. Privacy, robustness, machine learning.
Clément Canonne @ccanonne_
31K Followers 928 Following Senior Lecturer @Sydney_Uni. Postdocs @IBMResearch, @Stanford; PhD @Columbia. Converts ☕ into puns: sometimes theorems. He/him. @[email protected]
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Dan Roy @roydanroy
45K Followers 2K Following ML / AI researcher, emphasis on theory. Research Director and Canada CIFAR AI Chair, @VectorInst Professor, @UofT (Statistics/CS)
Amin Karbasi @aminkarbasi
8K Followers 2K Following Associate Professor at Yale University, staff research scientist at Google.
Sara Hooker @sarahookr
39K Followers 7K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.
Niloofar (Fatemeh) Mi.. @niloofar_mire
4K Followers 1K Following Postdoc @uwcse-@uwnlp, Ph.D. from @ucsd_cse /Privacy, ML, NLP, @winlpworkshop chair, @MSFTResearch - Semantic Machines
Zachary Lipton @zacharylipton
59K Followers 2K Following Professor: CMU/@acmi_lab, CTO / CSO: @AbridgeHQ, Creator: @d2l_ai & https://t.co/QQt98VNLUp, Relapsing 🎷
Dimitris Papailiopoul.. @DimitrisPapail
11K Followers 971 Following prof @ wisconsin; thinking about transformers; learning in context; babas of Inez Lily
Maxim Raginsky @mraginsky
8K Followers 2K Following father, academic, raconteur, aging wannabe hipster blog: https://t.co/akk6LCvKw6
Prof. Anima Anandkuma.. @AnimaAnandkumar
25K Followers 2K Following Bren Professor @caltech, Fmr Sr Director of #AI research @nvidia, Fmr Principal Scientist @awscloud, AI+Science, PDE, Neural operators. Views my own.
Mathieu Alain @miniapeur
19K Followers 2K Following Researching @ai_ucl. Co-organises @uclcsml and @logconference. FR, EN, trying ES. 🇹🇼🇨🇦🇬🇳🇺🇸🇩🇴🇫🇷🇪🇸🇬🇧🇿🇦
Thomas Steinke @shortstein
9K Followers 454 Following Computer scientist interested in (differential) privacy & related topics, e.g., generalization. @GoogleDeepMind Opinions are mine ©. 🇳🇿
Amir-massoud Farahman.. @SoloGen
5K Followers 2K Following Goal: Understanding the computational and statistical principles required to design AI/RL agents. #MahsaAmini #BlackLivesMatter
Bill Yuchen Lin 🤖 @billyuchenlin
6K Followers 2K Following Research @allen_ai. I evaluate (multi-modal) LLMs, build agents, and study the science of LLMs. Previously: @GoogleAI & @MetaAI FAIR @nlp_usc
Alex Dimakis @AlexGDimakis
13K Followers 2K Following UT Austin Professor. Researcher in Machine Learning and Information Theory. National AI Institute on the Foundations of Machine Learning (IFML) Co-director.
Shaily @shaily99
5K Followers 2K Following PhD @LTIatCMU Prev: @GoogleAI @MSFTResearch. Working on ethics and evaluation in #NLProc. Usually ranting, often about research & DEI. 📚 @readsndrants
Sam Power @sp_monte_carlo
17K Followers 7K Following Lecturer in Maths & Stats at Bristol. Interested in probabilistic + numerical computation, statistical modelling + inference. (he / him)
Pierre Alquier @PierreAlquier
8K Followers 5K Following Professor of Statistics @ESSEC_AP 🇸🇬 // Previously @RIKEN_AIP 🇯🇵 @ENSAEparis 🇫🇷 @ucddublin 🇮🇪 🇪🇺 // random posts about research & birds photos // 🌈
Arya Mazumdar @MountainOfMoon
3K Followers 319 Following Professor @UCSanDiego Dy. Director+AD for Research NSF AI Inst https://t.co/wblPm6DhUX, UCSD Site Lead @encoreinstitut Information Theory, Coding Th., Machine Learning
Eden Narr @EdenNarr56903
66 Followers 5K Following
Tarteau @Tarteau325417
1 Followers 50 Following
Crazy Universe @Crazy_Universe0
94 Followers 1K Following
Rohit Mittal @rohitdotmittal
11K Followers 1K Following Built and sold a fintech startup for 8 figures. Immigrant. Co-founder/CEO @stilt_inc (acq by JGW). YC W16. Raised $350M in debt. Fintech and debt finance guy.
Xuhui Zhang @XuhuiZhangXHZ
4 Followers 230 Following
Maragret Newbert @MaragretN33278
97 Followers 5K Following
Mathieu Blondel @mblondel_ml
9K Followers 421 Following Research scientist at Google DeepMind. Current research interests: differentiable programming, LLMs, Transformers.
Darcey Maco @DarceyMaco14341
56 Followers 5K Following
Bethan Puett @BethanPue
58 Followers 5K Following
Harsh Maheshwari @HarshMheshwari
1K Followers 1K Following Enthusiastic about #GenerativeAI #DataScience 🤖 | Constantly curious learner 🌱 | Applied scientist 2 at @amazon | Writer at @medium | @IITKGP Graduate
Aria Ehret @ehre_ar
66 Followers 5K Following
Hunter Lang @hunterjlang
262 Followers 248 Following phd student at @MIT_CSAIL working in self/weak supervision, nlp with @David_Sontag. he/him
James Smith @jamessealesmith
489 Followers 293 Following Research Scientist at @Samsung_RA. PhD in ML from @GeorgiaTech @mlatgt. I work on efficient model design and training for generative AI.
Aaditya ; @Aaditya26082004
527 Followers 7K Following CS'26 • Machine Learning • Open-Source • Web Dev. • Algorithms • Jai Shree Krishna 🦚🪈
li ii iq j @iq_li80427
54 Followers 311 Following
Ocean Brow @oce_brow
53 Followers 5K Following
Ehsan Naderi @EHSAN__NADERI
3 Followers 89 Following Computer Engineering @ Sharif University Of Technology
CLS @ChengleiSi
2K Followers 3K Following vibing @stanfordnlp | real AGI is the friends we made along the way
【𝕐o𝕦𝕤𝕖�.. @YosGPT
10K Followers 5K Following Programming Engineer & Linux+ | IT & Net+ | CCIE & CISSP | Azure Developer & Multi-Clouds Architect+ | Quantum AI Builder+ | #الحمدلله_على_نعمة_الامارات 🇦🇪 ❤️
روزی، روزگا.. @panflute2013
139 Followers 1K Following
Mohammad Alaggan, Ph... @m_aggan
1K Followers 3K Following Sr. Software Development Engineer at @AWSCloud. Opinions are my own.
Angel Edith @AngelEdith57453
41 Followers 5K Following
INDRAJEET @indrajeet877
424 Followers 2K Following Head of Math Department,Allen Institute Karaikal BTech NITW 2012, Option trader & investor. Math geek, tech-forward, learner Plus Python & Spanish skills.
Xiang Fu @thisisxfu
43 Followers 2K Following Researcher @BUSPH | @BU_CDS | Founder of @ModularNLP | Deep Learning | Data Science |
Brook Trusillo @TrusilBro
74 Followers 5K Following
Pardis Emami-Naeini @PNaeini
911 Followers 405 Following Assistant Professor and DST Scholar at @dukecompsci Ph.D. @SCSatCMU Prev. @uwcse, @MSFTResearch, @intel
kolergy @kolergy
196 Followers 2K Following Test early, physics always win at the end... Machine learning is changing the world! Be prepared
Adhil Parammel @adhil_parammel
111 Followers 1K Following
李伟 @lwi54973215
5 Followers 120 Following
Jitendra Sharma @jkumarsharma998
819 Followers 6K Following Curious about Research in AI. NLP and Computer Vision Interest me. Curious about truth and existence. Views are personal.
Bizlounge @BizLounger
205 Followers 3K Following
T J @tdj11100
319 Followers 4K Following TJ completed a Ph.D. in Physics and then moved into the tech world.
Kassidy Shepard @shepar_kass
56 Followers 5K Following
czxttkl @czxttkl
14 Followers 134 Following
Michael Celentano @mcelentano
136 Followers 98 Following Statistics post-doc @UCBerkeley with @UCB_MillerInst. PhD in statistics from @Stanford.
amirhossein bagheri @amirhossei73915
1 Followers 76 Following
Lara Mutana @mutana28280
75 Followers 5K Following
Pensé FFun @inftyCategory
113 Followers 6K Following
AI Papers Podcast @aipaperspodcast
881 Followers 2K Following A digestible daily update on the latest AI Research Papers. Brought to you by @pocketpodapp
Nirupama Ratna @ratna_kandala
182 Followers 1K Following Ph.D. student in Linguistics @ IIT Hyderabad BS-MS in Systems Biology #NLP#AI#Neuroscience
Preetika Verma @PreetikaVerma15
38 Followers 150 Following
Gautam Kamath @thegautamkamath
44K Followers 505 Following Assistant Prof of CS @UWaterloo, Faculty @VectorInst, Canada @CIFAR_News AI Chair. Co-EiC @TmlrOrg. I lead @TheSalonML. Privacy, robustness, machine learning.
Clément Canonne @ccanonne_
31K Followers 928 Following Senior Lecturer @Sydney_Uni. Postdocs @IBMResearch, @Stanford; PhD @Columbia. Converts ☕ into puns: sometimes theorems. He/him. @[email protected]
(((ل()(ل() 'yoav))).. @yoavgo
46K Followers 2K Following
Behnam Neyshabur @bneyshabur
18K Followers 690 Following Senior Staff Research Scientist @GoogleDeepMind, Interested in reasoning w. LLMs, traveling & backpacking
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Dan Roy @roydanroy
45K Followers 2K Following ML / AI researcher, emphasis on theory. Research Director and Canada CIFAR AI Chair, @VectorInst Professor, @UofT (Statistics/CS)
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Rosanne Liu @savvyRL
33K Followers 966 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDR
Amin Karbasi @aminkarbasi
8K Followers 2K Following Associate Professor at Yale University, staff research scientist at Google.
NeurIPS Conference @NeurIPSConf
111K Followers 35 Following New Orleans, Dec 10-16, 23. https://t.co/ga8aOw615g Tweets to this account are not monitored. Please send feedback to [email protected].
Sara Hooker @sarahookr
39K Followers 7K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.
Niloofar (Fatemeh) Mi.. @niloofar_mire
4K Followers 1K Following Postdoc @uwcse-@uwnlp, Ph.D. from @ucsd_cse /Privacy, ML, NLP, @winlpworkshop chair, @MSFTResearch - Semantic Machines
Jia-Bin Huang @jbhuang0604
51K Followers 285 Following Associate Professor @umdcs; Part-time Research Scientist @Meta. I like pixels.
Zachary Lipton @zacharylipton
59K Followers 2K Following Professor: CMU/@acmi_lab, CTO / CSO: @AbridgeHQ, Creator: @d2l_ai & https://t.co/QQt98VNLUp, Relapsing 🎷
Dimitris Papailiopoul.. @DimitrisPapail
11K Followers 971 Following prof @ wisconsin; thinking about transformers; learning in context; babas of Inez Lily
Maxim Raginsky @mraginsky
8K Followers 2K Following father, academic, raconteur, aging wannabe hipster blog: https://t.co/akk6LCvKw6
Prof. Anima Anandkuma.. @AnimaAnandkumar
25K Followers 2K Following Bren Professor @caltech, Fmr Sr Director of #AI research @nvidia, Fmr Principal Scientist @awscloud, AI+Science, PDE, Neural operators. Views my own.
Ben Recht @beenwrekt
26K Followers 365 Following optimization. machine learning. uc berkeley. I blog at https://t.co/fkJujOPsJb The world won't end.
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Carole-Jean Wu @CarolejeanWu
31 Followers 17 Following
Beidi Chen @BeidiChen
6K Followers 351 Following Asst. Prof @CarnegieMellon, Visiting Researcher @Meta, Postdoc @Stanford, Ph.D. @RiceUniversity, Large-Scale ML, a fan of Dota2.
bilge @bilgeacun
346 Followers 430 Following Research Scientist @MetaAI, PhD @IllinoisCS @Illinois_Alma, @BilkentUniv alum.
Mostafa Elhoushi @m_elhoushi
572 Followers 1K Following Research Engineer at Work. Volunteering for Various Causes after Work. Opinions are my own. 🇵🇸
Eric Wallace @Eric_Wallace_
6K Followers 1K Following Researcher at OpenAI working to make language models more trustworthy, secure, and private.
Hunter Lang @hunterjlang
262 Followers 248 Following phd student at @MIT_CSAIL working in self/weak supervision, nlp with @David_Sontag. he/him
Andrea Bajcsy @andrea_bajcsy
1K Followers 182 Following Assistant Professor @SCSatCMU, @CMU_Robotics | PhD from @Berkeley_EECS | Robots, humans, learning, and safety
Philipp Schmid @_philschmid
16K Followers 651 Following Tech Lead and LLMs at @huggingface 👨🏻💻 🤗 AWS ML Hero 🦸🏻 | Cloud & ML enthusiast | 📍Nuremberg | 🇩🇪 https://t.co/l1ppq3q3hk
Aviral Kumar @aviral_kumar2
2K Followers 338 Following Research Scientist at Google DeepMind. Incoming Assistant Professor of CS & ML at CMU (Fall 2024). PhD from UC Berkeley.
Fahim Tajwar @FahimTajwar10
164 Followers 241 Following PhD Student @mldcmu @SCSatCMU BS/MS from @Stanford
Sasha Orloff @sashaorloff
5K Followers 1K Following CEO @puzzlefin, host of @TurpentineMedia Finance podcast, and tech optimist. YC, ODF, VG alum. I tend to post about accounting, finance and fundraising.
James Smith @jamessealesmith
489 Followers 293 Following Research Scientist at @Samsung_RA. PhD in ML from @GeorgiaTech @mlatgt. I work on efficient model design and training for generative AI.
M.J. Crockett @mollycrockett
15K Followers 2K Following Professor @PsychPrinceton & University Center for Human Values | Cognitive scientist curious about (anti)normativity, technology & the self | They/She 🏳️🌈
Laura Wendel @Lauramaywendel
33K Followers 971 Following Startup Founder & Software Engineer • App in the making • Bookworm •
AI Safety Workshop @ .. @NG_AI_Safety
1 Followers 0 Following Workshop at ICML'24, with a focus on emerging trends in AI and explore the challenges associated with deploying these technologies safely.
Tamay Besiroglu @tamaybes
3K Followers 720 Following Thinking about economics, computing and machine learning @EpochAIResearch @MIT_CSAIL
Anaïs Urlichs @urlichsanais
23K Followers 1K Following 🕸️Newsletter https://t.co/kuJYGTTiYv 🚀she/her Opinions are mine. I am not responsible for anyone not tagged/directly addressed in my tweets feeling addressed.
Pardis Emami-Naeini @PNaeini
911 Followers 405 Following Assistant Professor and DST Scholar at @dukecompsci Ph.D. @SCSatCMU Prev. @uwcse, @MSFTResearch, @intel
Ahmad Al-Dahle @Ahmad_Al_Dahle
4K Followers 53 Following #Girldad of twins. Leading GenAI @ Meta (llama, imagine, meta ai and more)
Mufan (Bill) Li @mufan_li
804 Followers 492 Following Postdoc @Princeton ORFE | Prev: PhD @UofTStatSci @VectorInst
Georgia Gkioxari @georgiagkioxari
9K Followers 412 Following Assistant professor in Computing + Mathematical Sciences @Caltech 🏛️ ∙ Computer vision enthusiast 🤖 ∙ Previously at @metaai 👩🏻💻∙ From 🇬🇷
Arthur Allshire @arthurallshire
1K Followers 379 Following robotics & simulation. incoming PhD @Berkeley_AI. intern @NvidiaAI. prev EngSci @UofT 🇮🇪 🇨🇦 🇦🇺🇨🇭🇨🇿
Zac Kenton @ZacKenton1
1K Followers 1K Following Research Scientist in AI safety at DeepMind. Views are my own and don't represent DeepMind.
Hanna Hajishirzi @HannaHajishirzi
6K Followers 328 Following Associate professor at @uw_cse; senior director at @allen_ai co-leading @allenNLP; AI/NLP researcher at @uw_nlp
Akshitha Sriraman @AkshithaSriram1
3K Followers 218 Following Assistant Professor @CMU_ECE & @SCSatCMU. PhD from @UMichCSE. Research in Software Systems & Computer Architecture. (she/her)
Hossein Talebi @hossTale
32 Followers 7 Following Senior Staff Software Engineer at Google Research working on machine learning, computer vision, and computational imaging.
Kaiyu Yang @KaiyuYang4
2K Followers 774 Following Postdoc @Caltech CMS. Previously: @PrincetonCS, @Tsinghua_Uni. https://t.co/KZiCELQI2D
Varun Vasudevan @DevanVarun
253 Followers 291 Following Computational Scientist. PhD from @ICMEStanford.
David Dohan @dmdohan
8K Followers 1K Following reducing perplexity @openai | past: probabilistic programs, proteins, science & reasoning @ google brain 🧠
Shyam Sankar @ssankar
15K Followers 211 Following CTO @Palantirtech, Chairman @Ginkgo https://t.co/oaEBf1su2x https://t.co/mf1cSAPtef
Palmer Luckey @PalmerLuckey
219K Followers 2K Following I am a technology enthusiast, writer, and modder. Founder of ModRetro, @Oculus VR, and @Anduriltech. Keeping American superheroes safe with autonomous systems.
Noam Brown @polynoamial
34K Followers 611 Following Researching reasoning @OpenAI | Co-created Libratus/Pluribus, the first superhuman no-limit poker AIs | Co-created CICERO | PhD from @SCSatCMU
Elan Rosenfeld @ElanRosenfeld
1K Followers 184 Following Final year ML PhD Student at CMU working on principled approaches to robustness, security, generalization, and representation learning.
Uri Shalit @ShalitUri
6K Followers 1K Following Machine learning researcher, working on causal inference and healthcare applications. Associate prof @TechnionLive @[email protected] @urish.bsky.social
Konrad Rieck 🌈 @mlsec
3K Followers 384 Following Machine Learning and Security, Professor of Computer Science at TU Berlin, @[email protected]
Samir S. Patel @SPatel_v1
770 Followers 165 Following Editor-in-Chief, @QuantaMagazine, Formerly @atlasobscura; Alum and Prof, @columbiajourn
Thomas Lin @7homaslin
10K Followers 1K Following Quanta Books; @QuantaMagazine founder / 1st EIC; former @nytimes, @ScienceWriting; books: (science) https://t.co/ZHAR1Pn4Bi + (math) https://t.co/5WSMip02t1
Junwei @JDI_LINK
400 Followers 5K Following Angel Investor. Ph.D. Computer Vision and Parallel Computing
Tanusree Sharma @Tanusree_Sharma
1K Followers 972 Following Ph.D. Candidate @UofIllinois | incoming assistant professor @ISTatPENNSTATE | Usable Security, Decentralized Governance | Formerly @Google, @maxplanckpress
HaoyueBai @haoyue_bai
933 Followers 838 Following Ph.D. student at Computer Science Department @UWMadisonCS, MPhil @HKUSTCSE.
Ryan David Cotterell @ryandcotterell
9K Followers 1K Following
Secure Learning Lab (.. @uiuc_aisecure
937 Followers 288 Following We are a computer science research group led by Bo Li at UIUC, focusing on responsible and trustworthy machine learning.
never seen a graphic so wrong yet so useful
@yoavgo @kendmil @charusaie Maybe or maybe not but how do you know so much? Are you actually here in the U.S. talking to them? I am here, and visited the Harvard encampment on Wednesday, and still don't know what everyone's mindset is.
My fiance just left town and now he's going to get an alert that I bought The Prince of Egypt on his Prime account lol
An exciting paper from our @AmazonScience CodeWhisperer team on speculative decoding for batched sequence generation. Nearly everybody competes in generating a single output, but, in practice, multiple outputs can be required. This is an overlooked but important scenario!
BASS Batched Attention-optimized Speculative Sampling Speculative decoding has emerged as a powerful method to improve latency and throughput in hosting large language models. However, most existing implementations focus on generating a single sequence.
meanwhile I’m still on free @GoogleColab
First @nvidia DGX H200 in the world, hand-delivered to OpenAI and dedicated by Jensen "to advance AI, computing, and humanity":
Professors never look happier than post a (successful) PhD thesis defense!
Very excited to share our latest work LayerSkip! we started our exploration in how to leverage dynamic compute during generation to enable low latency inference in a single model by using the earlier layers to generate tokens and verify with the remaining ones.
Excited to present our latest research: 🦘LayerSkip! huggingface.co/papers/2404.16… We run a subset of earlier layers of an LLM, & verify/correct using the remaining layers, to achieve upto 🚀2.16x speedup on Llama 7B @AkshatS07 @bilgeacun @bwasti @Ahhegazy77 @BeidiChen @CarolejeanWu
Akshat recently joined our pre-training team to focus on adaptive compute (static graph) approaches to pre-training. Would recommend following him for early results.
Very excited to share our latest work LayerSkip! we started our exploration in how to leverage dynamic compute during generation to enable low latency inference in a single model by using the earlier layers to generate tokens and verify with the remaining ones.
My op-ed in the Crimson today on the Harvard protest and why we as a university are failing in both seeming more important than we are, and not being as important as we could be. thecrimson.com/column/council…
It's been a week since LLaMA 3 dropped. In that time, we've: - extended context from 8K -> 128K - trained multiple ridiculously performant fine-tunes - got inference working at 800+ tokens/second If Meta keeps releasing OSS models, closed providers won't be able to compete.
Transformers Can Represent n-gram Language Models Plenty of existing work has analyzed the abilities of the transformer architecture by describing its representational capacity with formal models of computation. However, the focus so far has been on analyzing the
I agree. Our analysis (arxiv.org/abs/2310.00535) on training dynamics of Transformer shows that self-attention really plays an important role in learning the right representation. More specifically, self-attention dynamics encourages tokens with high co-occurrence to learn first,…
not true, especially for language. if you trained a large & deep MLP language model with no self-attention, no matter how much data you'll feed it you'll still be lacking behind a transformer (with much less data). will it get to the same point? i don't think so. your tokens…
Persuaded or manipulated by AI? Check out this new paper from @GoogleDeepMind on definitions and mitigations. It was a privilege to advise on this important research! #AIethics #PersuasionAI
🔮 New Google DeepMind paper exploring what persuasion and manipulation in the context of language models. 👀 Existing safeguard approaches often focus on harmful outcomes of persuasion. This research argues for a deeper examination of the process of AI persuasion itself to…
Some personal updates: I joined OpenAI a few months ago, working on all things robustness/safety/privacy. Also, we are working to publish more of our safety work. See my first project here below, where we make initial progress on prompt injections and other attacks!
Introducing the Instruction Hierarchy, our latest safety research to advance robustness for prompt injections and other ways of tricking LLMs into executing unsafe actions. More details: arxiv.org/abs/2404.13208
If you are attending @iclr_conf and are interested in privacy regulations, especially in EU, join us on May 11th at the 'Privacy Regulation and Protection in' workshop! Location: Schubert 3, Messe Wien Exhibition and Congress Center pml-workshop.github.io/iclr24/
Controversial opinion: No one should be authoring 30+ papers per year. I'm not criticizing those who do. But there's something wrong with the system if it incentivizes quantity over quality like this.
Q: Senior researchers should be authoring lots of work. A: I doubt the best use of a senior researcher's time is to put 1/n of their time into n projects for the largest possible n. Something has gone wrong if that's what we expect.
One of the many reasons for attending @MicroTas2024
Excited to share a video by Amy Herr on why you should come to microTAS! Amy is the John D. & Catherine T. MacArthur Prof at the University of California Berkeley, renowned expert in microfluidics & former president of the Chemical & Biological Microsystems Society.
Trends for United States
You might like
