Simon Shaolei Du @SimonShaoleiDu
Assistant Professor @uwcse. Postdoc @the_IAS. PhD in machine learning @mldcmu. simonshaoleidu.com Seattle, WA Joined September 2017-
Tweets446
-
Followers6K
-
Following2K
-
Likes5K
I agree. Our analysis (arxiv.org/abs/2310.00535) on training dynamics of Transformer shows that self-attention really plays an important role in learning the right representation. More specifically, self-attention dynamics encourages tokens with high co-occurrence to learn first,…
I agree. Our analysis (arxiv.org/abs/2310.00535) on training dynamics of Transformer shows that self-attention really plays an important role in learning the right representation. More specifically, self-attention dynamics encourages tokens with high co-occurrence to learn first,…
Big congratulations to Avi Wigderson of IAS Princeton for winning the Turing Award in CS. Truly an all-time great in theoretical computer science and discrete math. Also one of the nicest human beings I know --friend and mentor to so many (including me) tinyurl.com/fz5vxxaf
Our LabelBench work has been accepted to the DMLR journal🎉 Super smooth experience and highly recommended for anyone working on data centric ML work. Check out labeltrain.ai: our broader set of label efficient learning work More on LabelBench: x.com/jifan_zhang/st…
Our LabelBench work has been accepted to the DMLR journal🎉 Super smooth experience and highly recommended for anyone working on data centric ML work. Check out labeltrain.ai: our broader set of label efficient learning work More on LabelBench: x.com/jifan_zhang/st…
🆕 @TheLancet Counterfactual #AI models have exciting potential (and challenges) in medicine and life science Why? An explainer, by @suinleelab and me thelancet.com/journals/lance…
A TWO-PLAYER system for online RL fine-tuning.
A TWO-PLAYER system for online RL fine-tuning.
Honored to become a 2024 #SloanFellow. Thanks to @SloanFoundation and to all my students and collaborators for their amazing work!
Honored to become a 2024 #SloanFellow. Thanks to @SloanFoundation and to all my students and collaborators for their amazing work!
“The problem with large language models is that they’re large!” In this @UWITNews story, #UWAllen’s @Tim_Dettmers explains QLora, the @MadronaVentures prize-winning tool from @uwnlp researchers that enables you to take an LLM and "make it your own”: itconnect.uw.edu/making-languag… 1/2
New paper on label-efficient supervised finetuning of LLMs. We address the expensive prompt annotation cost by humans/proprietary LLMs, saving as much as 50% on FLAN V2. Paper: arxiv.org/abs/2401.06692 Work led by: @jifan_zhang @cloudwaysX @BhattGantavya @arnaved 1/
Why Decision Transformer? It doesn't require the Bellman Completeness -- a strong assumption needed by Q-learning
Why Decision Transformer? It doesn't require the Bellman Completeness -- a strong assumption needed by Q-learning
I'm recruiting PhD students @uwcse @uwnlp (bdata.uw.edu). Focus areas include Human-AI collaboration, language agents, LLM safety & applications to mental health, social sciences, education. Apply here: cs.washington.edu/academics/phd/… @uwdatascience @uw_ischool @uwdub
Zihan will present our work on optimal sample complexity for reinforcement learning: arxiv.org/abs/2307.13586
Zihan will present our work on optimal sample complexity for reinforcement learning: arxiv.org/abs/2307.13586
We proved Q* is hard 4 years ago 🤷♂️
#UWAllen is hiring! Join our outstanding scholarly community at @UW shaping the future of computing—and having fun while doing it! Priority will be given to faculty applications received by Nov. 13 (teaching track) & Nov. 15 (tenure track). Please share! cs.washington.edu/faculty_candid…
Now Scan&Snap has a follow-up! 1/ We introduce JoMA (arxiv.org/abs/2310.00535), a joint dynamics for MLP lower and self-Attention layers, in order to better understand (1) how multilayer Transformer with MLP nonlinearity works, and (2) qualitatively explain how hierarchical…
Now Scan&Snap has a follow-up! 1/ We introduce JoMA (arxiv.org/abs/2310.00535), a joint dynamics for MLP lower and self-Attention layers, in order to better understand (1) how multilayer Transformer with MLP nonlinearity works, and (2) qualitatively explain how hierarchical…
Want to get model-based RL to work in diverse, dynamic scenes? Check out @chuning_zhu's latest work (RePo) on model-based reinforcement learning without reconstruction, where we show how to learn world models that scale to dynamic, multi-task environments. A 🧵(1/6)
Super exciting work by @uwcse's @wangshengpkucn's lab published in Nature Machine Intelligence @NatMachIntell! 👇 nature.com/articles/s4225…
Introducing our framework and benchmarks for label-efficient learning. Evaluations of large pretrained models, Semi-SL and active learning have mostly stayed isolated. LabelBench combines all these mutually beneficial techniques to examine the best possible label-efficiency 1/
Gautam Kamath @thegautamkamath
44K Followers 505 Following Assistant Prof of CS @UWaterloo, Faculty @VectorInst, Canada @CIFAR_News AI Chair. Co-EiC @TmlrOrg. I lead @TheSalonML. Privacy, robustness, machine learning.
Yi Ma @YiMaTweets
71K Followers 123 Following Chair Professor in AI, Director of IDS, Head of CS, HKU; Professor of EECS, Berkeley; Author of Book: High-Dim Data Analysis, https://t.co/gwaqMJp8av.
Clément Canonne @ccanonne_
31K Followers 927 Following Senior Lecturer @Sydney_Uni. Postdocs @IBMResearch, @Stanford; PhD @Columbia. Converts ☕ into puns: sometimes theorems. He/him. @[email protected]
Csaba Szepesvari @CsabaSzepesvari
8K Followers 703 Following "If there is not folly in the world, then the world itself is folly. You must understand that mistakes are not always regrets." - Paul Tobin, Bandette🤠
Jason Lee @jasondeanlee
10K Followers 3K Following Associate Professor at Princeton and Research Scientist at Google DeepMind. ML/AI Researcher working on foundations of LLMs and deep learning
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Amin Karbasi @aminkarbasi
8K Followers 2K Following Associate Professor at Yale University, staff research scientist at Google.
Dan Roy @roydanroy
45K Followers 2K Following ML / AI researcher, emphasis on theory. Research Director and Canada CIFAR AI Chair, @VectorInst Professor, @UofT (Statistics/CS)
Behnam Neyshabur @bneyshabur
18K Followers 689 Following Senior Staff Research Scientist @GoogleDeepMind, Interested in reasoning w. LLMs, traveling & backpacking
Prof. Anima Anandkuma.. @AnimaAnandkumar
25K Followers 2K Following Bren Professor @caltech, Fmr Sr Director of #AI research @nvidia, Fmr Principal Scientist @awscloud, AI+Science, PDE, Neural operators. Views my own.
Yuandong Tian @tydsh
16K Followers 801 Following Research Scientist and Senior Manager in Meta AI (FAIR). AI-guided Optimization and Representation Learning. Novelist in spare time. PhD in @CMU_Robotics.
Dimitris Papailiopoul.. @DimitrisPapail
11K Followers 964 Following prof @ wisconsin; thinking about transformers; learning in context; babas of Inez Lily
Animesh Garg @animesh_garg
21K Followers 1K Following Foundation Models for Generalizable Autonomy. Assistant Professor in AI Robotics @GeorgiaTech + @NvidiaAI. prev @Stanford @berkeley_ai @UofTCompSci
Sergey Levine @svlevine
79K Followers 122 Following Associate Professor at UC Berkeley Co-founder, Physical Intelligence
Maxim Raginsky @mraginsky
8K Followers 2K Following father, academic, raconteur, aging wannabe hipster blog: https://t.co/akk6LCvKw6
Zachary Lipton @zacharylipton
59K Followers 2K Following Professor: CMU/@acmi_lab, CTO / CSO: @AbridgeHQ, Creator: @d2l_ai & https://t.co/QQt98VNLUp, Relapsing 🎷
Sam Power @sp_monte_carlo
17K Followers 7K Following Lecturer in Maths & Stats at Bristol. Interested in probabilistic + numerical computation, statistical modelling + inference. (he / him)
Shane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)
Zhengzhong Tu @_vztu
3K Followers 2K Following AI Researcher @GoogleAI | PhD @UTAustin | BS @FudanUni | Intern @GoogleAI | Computer Vision, GenAI | Opinions are my own
Peter Morales @PeterMoralesX
217 Followers 2K Following Founder of funded Stealth AI Startup. Interested in AI development at the edge? DM.
StarlightXYY @huyue82028905
18 Followers 45 Following
rayman @raymanlim
90 Followers 626 Following "The knowledge is there, but you need to utter the incantation to make it manifest."
Sia @SDreaming
12 Followers 1K Following
Sara Lin @S1540295
0 Followers 19 Following
Mao Hong @MaoHong8
0 Followers 20 Following
isynch @funnynoise
8 Followers 78 Following
rambalo987 @rambalo987
6 Followers 63 Following
Pensé FFun @inftyCategory
115 Followers 6K Following
li ii iq j @iq_li80427
56 Followers 309 Following
BS @BS3519241223690
50 Followers 50 Following
Information MDPI @InformationMDPI
1K Followers 2K Following Information (ISSN 2078-2489, #Scopus, #ESCI, #EI Compendex) is an open access journal of information science and technology, data, knowledge and communication.
Alo @Hal90910
0 Followers 2K Following
jj-Theory @TheoryJj34173
20 Followers 171 Following
Elgce @BenQingwei
66 Followers 219 Following Hey, everyone! I am a junior student of Tsinghua University & incoming Ph.D of MMLAB@CUHK. I am interested in Reinforcement Learning and Robotics.
ahad @ahadj0
12 Followers 100 Following
Wenhao Zhan @zhan_wenhao
9 Followers 24 Following PhD Student @ Princeton University • Theoretical Reinforcement Learning • Previously BS @ Tsinghua University
Sichao Liu @ErikLiuSe
44 Followers 289 Following
Keyao Zhan @ZhanKeyao
13 Followers 50 Following Senior Undergraduate @PKU1898 SMS. Incoming PhD student @HarvardBiostats in fall 2024.
FSM @fsm_top
8 Followers 104 Following
Anonymous @Anonymousuomi
0 Followers 67 Following
Hoyeon Chang @hoyeon_chang
278 Followers 640 Following PhD student at KAIST Language & Knowledge Lab Passionate about understanding intelligent systems Also a jazz pianist
Huang Tarik @TarikHuang17731
22 Followers 674 Following
Underthec @Underthec01
133 Followers 962 Following
赫菲斯托斯 @hephaestus93god
4 Followers 213 Following
Kun (Kevin) SUN @Sharp_K_Sun
226 Followers 2K Following Scientist Researcher @ Tübingen University and Professorial Research Fellow @ Fudan University, and interested in LLMs, NLP, and computational cognition .
CollaborativeDynamics.. @CoDynamicsAI
17 Followers 773 Following Boost all aspects of your business with our bespoke B2B AI solutions in prompt engineering, personas and automation. #AI #Automation #GenerativeAI🚀
Zhuokai Zhao @zhuokaiz
1 Followers 21 Following Final-year CS PhD Candidate at @UChicago. Research in data-centric and trustworthy ML. Previously @Meta, @Twitch, @Siemens, @HopkinsEngineer, @ECEILLINOIS.
Wendi Li @windy_lwd
1 Followers 50 Following
Pinpoint @Pinpoint201308
0 Followers 38 Following
Xindong Chen @Dabenmao3
86 Followers 1K Following Postdoc researcher at Tsinghua University. #CellDynamics #Biophysics #Protein-Protein Interactions #AppliedMathematics #Drug-AIGC！
Ritesh Kanchi @rtsh__
170 Followers 686 Following cs & hci @uwcse @makeabilitylab ~ prev @cmuhcii ~ WWDC Scholar ~ https://t.co/4NXNxgstfn
Mehrdad Moghimi @MehrdadM96
9 Followers 144 Following PhD Student at @YorkUniversity, Interested in risk-sensitive #ReinforcementLearning
Jiaming Liu @Jiaming__Liu
274 Followers 830 Following PhD student in ESE，Computational imaging group (CIG)@wustlcig , Washington University in St.Louis (WUSTL).
Que Liu @iDZQueLiu1
0 Followers 364 Following Postdoc Fellow at UC Berkeley and Assistant Professor at Sun Yat-sen University.I study TCS(computational complexity,algorithm math).
kovariance @kovariance
67 Followers 2K Following
Gautam Kamath @thegautamkamath
44K Followers 505 Following Assistant Prof of CS @UWaterloo, Faculty @VectorInst, Canada @CIFAR_News AI Chair. Co-EiC @TmlrOrg. I lead @TheSalonML. Privacy, robustness, machine learning.
Yi Ma @YiMaTweets
71K Followers 123 Following Chair Professor in AI, Director of IDS, Head of CS, HKU; Professor of EECS, Berkeley; Author of Book: High-Dim Data Analysis, https://t.co/gwaqMJp8av.
Clément Canonne @ccanonne_
31K Followers 927 Following Senior Lecturer @Sydney_Uni. Postdocs @IBMResearch, @Stanford; PhD @Columbia. Converts ☕ into puns: sometimes theorems. He/him. @[email protected]
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Csaba Szepesvari @CsabaSzepesvari
8K Followers 703 Following "If there is not folly in the world, then the world itself is folly. You must understand that mistakes are not always regrets." - Paul Tobin, Bandette🤠
Jason Lee @jasondeanlee
10K Followers 3K Following Associate Professor at Princeton and Research Scientist at Google DeepMind. ML/AI Researcher working on foundations of LLMs and deep learning
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Amin Karbasi @aminkarbasi
8K Followers 2K Following Associate Professor at Yale University, staff research scientist at Google.
Dan Roy @roydanroy
45K Followers 2K Following ML / AI researcher, emphasis on theory. Research Director and Canada CIFAR AI Chair, @VectorInst Professor, @UofT (Statistics/CS)
Behnam Neyshabur @bneyshabur
18K Followers 689 Following Senior Staff Research Scientist @GoogleDeepMind, Interested in reasoning w. LLMs, traveling & backpacking
Prof. Anima Anandkuma.. @AnimaAnandkumar
25K Followers 2K Following Bren Professor @caltech, Fmr Sr Director of #AI research @nvidia, Fmr Principal Scientist @awscloud, AI+Science, PDE, Neural operators. Views my own.
Yuandong Tian @tydsh
16K Followers 801 Following Research Scientist and Senior Manager in Meta AI (FAIR). AI-guided Optimization and Representation Learning. Novelist in spare time. PhD in @CMU_Robotics.
Ben Recht @beenwrekt
26K Followers 363 Following optimization. machine learning. uc berkeley. I blog at https://t.co/fkJujOPsJb The world won't end.
Dimitris Papailiopoul.. @DimitrisPapail
11K Followers 964 Following prof @ wisconsin; thinking about transformers; learning in context; babas of Inez Lily
Animesh Garg @animesh_garg
21K Followers 1K Following Foundation Models for Generalizable Autonomy. Assistant Professor in AI Robotics @GeorgiaTech + @NvidiaAI. prev @Stanford @berkeley_ai @UofTCompSci
Rosanne Liu @savvyRL
33K Followers 965 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDR
NeurIPS Conference @NeurIPSConf
111K Followers 35 Following New Orleans, Dec 10-16, 23. https://t.co/ga8aOw615g Tweets to this account are not monitored. Please send feedback to [email protected].
Sergey Levine @svlevine
79K Followers 122 Following Associate Professor at UC Berkeley Co-founder, Physical Intelligence
Maxim Raginsky @mraginsky
8K Followers 2K Following father, academic, raconteur, aging wannabe hipster blog: https://t.co/akk6LCvKw6
Xinran Gu @hmgxr128
208 Followers 91 Following Master student at the Institute for Interdisciplinary Information Sciences of Tsinghua University
Jesse Dodge @JesseDodge
3K Followers 2K Following Senior Research Scientist at AI2 @ai2_allennlp. Responsibly open work on the science of AI and AI for science. Environmental impact of AI. he/him 🏳️🌈
Giannis Daras @giannis_daras
4K Followers 399 Following Computer Science Ph.D. student, @UTAustin working with @AlexGDimakis. Research Scientist Intern @nvidia. Ex: @google, @explosion_ai, @ntua
Lining Yao @lining_yao
3K Followers 636 Following Assistant Professor @UCBerkeley @Cal_Engineer @BerkeleyME/ Director of Morphing Matter Lab / alum @mit @medialab @cmuhcii/ design sustainable morphing materials
Runtian Zhai @RuntianZhai
327 Followers 216 Following PhD @SCSatCMU. I study representation learning, why big models generalize, and out-of-distribution(OOD).
Shayne Longpre @ShayneRedford
4K Followers 997 Following PhD @MIT. Prev: @Google Brain, @apple ML, @stanfordnlp. 🇨🇦 Interests: AI/ML/NLP, Data-centric AI, transparency & societal impact
Stanley H. Chan @stanley_h_chan
7K Followers 137 Following Professor | computational imaging | machine learning | Purdue ECE
Shuangning Li @ShuangningLi
781 Followers 217 Following Postdoc @HarvardStats | PhD @Stanford Statistics
Bin Yu @bbiinnyyuu
352 Followers 7 Following Professor of Statistics, EECS and Comp. Bio. at UC Berkeley
Rebecca Barter @rlbarter
3K Followers 569 Following Data Scientist and Educator with a PhD in Statistics from UC Berkeley. I like exploring messy data and explaining things. https://t.co/EgnymIovaf
Hexiang (Frank) Hu @Hexiang_Hu
504 Followers 400 Following Research Scientist @GoogleDeepmind | Vision & Language | Gemini
@vfsglobalcare @vfsglobalcare
116K Followers 1 Following Official 24x7 Customer Care handle for VFS Global. World's largest outsourcing & technology services specialist for Governments and Diplomatic missions.
Bit查理⚔️ @BitBtcX
14K Followers 405 Following #btc OG since 2018 ｜Alpha Degen｜#btc #eth $pepe｜推文仅做记录｜挖掘一级项目｜曾经的天通苑房产中介，白手起家现A8-5身家｜带领无数小弟实现A7-A8｜心有猛虎，细嗅蔷薇
K线教主 @Paris13Jeanne
65K Followers 2K Following 二级/撸毛/Builder/ 21年币安带单平台TW季度第一/ 极端动保/缠论爱好者 币安:https://t.co/JzHvBmXBy0
Charles Qi @charles_rqi
6K Followers 218 Following Autopilot and AI @Tesla | Prev: Research Scientist & Manager @Waymo | Postdoc @FAIR, PhD @Stanford | COO at the Lighthouse Mentorship Program.
Character.AI @character_ai
115K Followers 13 Following Download the official #CharacterAI Mobile App for 𝗙𝗥𝗘𝗘: https://t.co/2QsT1bAhLu
Anca Dragan @ancadianadragan
8K Followers 177 Following AI safety & alignment at Google DeepMind • associate professor at UC Berkeley EECS • proud mom of an amazing 2yr old
Danijar Hafner @danijarh
14K Followers 868 Following Building AI that makes autonomous decisions using world models, artificial curiosity, and temporal abstraction @DeepMind
Rocky Duan @rocky_duan
779 Followers 84 Following Building @CovariantAI, CTO. Previously @OpenAI, @UCBerkeley PhD. 2024 Forbes 30 Under 30.
Lily Liu @calilyliu
15K Followers 3K Following co founder @anagramxyz, president @solanafndn, contributor @osmosiszone, superfan @superteamdao
Mickel Liu @mickel_liu
99 Followers 235 Following research visiting @uwnlp, Prev: @PKU1898, @uoftengineering RL + LLM
AK @_akhaliq
309K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo follow on Hugging Face: https://t.co/q2Qoey80Gx
jiang tao @csdncto
10K Followers 1K Following Founder, CEO of CSDN, No.1 Chinese Developer Community
Mohit Bansal @mohitban47
9K Followers 650 Following Parker Distinguished Professor, UNC Chapel Hill (@unc). Director https://t.co/5qlPVgnrlN (@uncnlp). Prev: @Berkeley_AI, @TTIC_Connect @IITKanpur #NLP, #CV, #AI, #ML
Fei Liu @feiliu_nlp
689 Followers 290 Following Associate professor @EmoryUniversity. Working on large language models, automatic summarization, natural language generation, and various aspects of AI.
Jiaqi Li @Jiaqi97Li
20 Followers 55 Following William H. Kruskal Instructor @UChicago '24 | Ph.D. Candidate in Statistics @WUSTL '24 | Statistical Learning, Time Series, Neuroimaging | Piano & Cello Player
零下二度 @jackli727
32K Followers 372 Following 腾讯大粤网前健康频道主编，白桦林散文诗撰稿人，经典语录微博大V。长期旅居海外，关注时政，热爱旅行和加密货币。励志，乐观，高逆商。短中线合约狙击手，长线现货价值投资者，攻防有序，道法自然。
EudemoniaCC @EudemoniaCC
28K Followers 1K Following 北极光创投NLVC Crypto 📧 [email protected]｜@TsingHua_Uni studying｜@THUBA_DAO 2023Summer VP/BD Lead/Research/Hackathon
Nathan Lambert @natolambert
25K Followers 688 Following Figuring out AI @allen_ai, "rl boi" DM me papers. Writes @interconnectsai, talks @retortai Has phd and some credentials
Aditi Raghunathan @AdtRaghunathan
1K Followers 18 Following Assistant professor at CMU @SCSatCMU @CSDatCMU | Machine learning
Yangqing Jia @jiayq
12K Followers 263 Following Founder @leptonai. @UCBerkeley alumni. ex @google & @facebook. ex vp @AlibabaGroup. Open source work on caffe, @pytorch, @tensorflow, & @onnxai.
花果山大圣 @shengxj1
39K Followers 748 Following 不爱上班的程序员, 靠谱的前端讲师， 卖课为生，前端 Web3 程序员英语 私教, 独立开发 努力做最好的程序员讲师 合作微信：itdasheng168
AutoGen @pyautogen
4K Followers 38 Following OSS library for agentic AI apps and research 🤖🤖 GitHub: https://t.co/LliIsorLuY Discord: https://t.co/2iE2O7QV6A Research: https://t.co/TeOUTAZrbd
Christian Szegedy @ChrSzegedy
32K Followers 2K Following #deeplearning, #ai research scientist. Opinions are mine.
main @main_horse
8K Followers 473 Following AGI Believer. Haven't applied @OpenAI. Likes are not always endorsement.
Anirudh Goyal @anirudhg9119
5K Followers 487 Following Gemini ♊ Spent time at @Berkeley_EECS, @MPI_IS, @DeepMind.
Vidya Muthukumar @v__muthukumar
89 Followers 70 Following Assistant Professor of ECE and ISyE at Georgia Tech. Interested in anything involving game theory, statistical and/or online learning. Co-organizer of @let4all.
Ben Grimmer @prof_grimmer
3K Followers 433 Following Assistant Professor @JohnsHopkinsAMS, Optimization, PhD @CornellORIE Mostly here to share pretty maths/3D prints, sometimes sharing my research
Morris @Morris_LT
20K Followers 341 Following 某不知名Web3.0创业公司 - 创始人&CEO&首席擦屁股执行官！ #Bitcoin #Đogecoin #Web3 #Gamefi
Neale Mahoney @nealemahoney
8K Followers 689 Following Professor @StanfordEcon. Incoming Director @SIEPR. Former @WhiteHouse National Economic Council. Watches soccer, reads history, listens to all types of music.
Nicolas Papernot @NicolasPapernot
10K Followers 665 Following Security and Privacy of Machine Learning @Uoft @VectorInst @Google 🇫🇷🇪🇺🇨🇦 Co-author https://t.co/VJF39DQPCu; @CentraleLyon + @PSUEngineering alumnus. Opinions mine
Jiquan Ngiam @JiquanNgiam
467 Followers 171 Following Building @Lutra_AI Previously: Google Brain, Coursera, Stanford ML GroupI agree. Our analysis (arxiv.org/abs/2310.00535) on training dynamics of Transformer shows that self-attention really plays an important role in learning the right representation. More specifically, self-attention dynamics encourages tokens with high co-occurrence to learn first,…
not true, especially for language. if you trained a large & deep MLP language model with no self-attention, no matter how much data you'll feed it you'll still be lacking behind a transformer (with much less data). will it get to the same point? i don't think so. your tokens…
We often hear about the theory-practice gap. At this workshop we will take a thorough look at this. Is there a gap? What is the nature of the gap? Who made it? Is it good to have the gap? If not, how to close it? I think this is super important for the healthiness of the field!
🧵 Thrilled to announce the #ICML RL workshop 'Aligning RL Experimentalists and Theorists'! We will have several talks and a panel delivered by a super lineup of speakers: @white_martha, @ShamKakade6, @yayitsamyzhang, Dylan Foster, Niao He, @svlevine, and @MengdiWang10. 1/3
My 2018 hobby proj supports a core part of the modern large-scale training pipeline in fairscale. The true power of #oss
I feel some $TSLA investors don’t fully understand the significance of Tesla's upcoming shareholder vote on reinstating Elon Musk's 2018 CEO compensation package. It’s a very important vote. This is a long post, but it needs to be. First, some background: Elon Musk's 2018…
Always great to see students succeed! Congrats Mufan.
I’m excited to announce that in July 2025 I will be joining @UWaterloo as an Assistant Professor in the Department of Statistics and Actuarial Science! Until then, I will continue at Princeton as a DataX Postdoc Fellow, working with Boris Hanin. I have many exciting projects…
A good day to give 2 guest lectures in Caltech in @yisongyue and @georgiagkioxari’s class! Shameless plugging llama3 advertisement in the introduction slides :) I hope my academia friends don’t mind :)
Thanks for the shout out! We'll be updating the course website with materials as the term progresses, and hope others find it useful. We're also thrilled to have guest lectures by @tydsh, @denny_zhou, @KaiyuYang4!
LLama3 is released with strong performance!
Introducing Meta Llama 3: the most capable openly available LLM to date. Today we’re releasing 8B & 70B models that deliver on new capabilities such as improved reasoning and set a new state-of-the-art for models of their sizes. Today's release includes the first two Llama 3…
Joins us for Machine Learning in Computational Biology (MLCB) 2024 mlcb.github.io Submission deadline: June 15 Conference dates: Sept 5 & 6
I’m excited to announce that in July 2025 I will be joining @UWaterloo as an Assistant Professor in the Department of Statistics and Actuarial Science! Until then, I will continue at Princeton as a DataX Postdoc Fellow, working with Boris Hanin. I have many exciting projects…
Now that it is official, my amazing student Chris Harshaw is joining the statistics department @Columbia as an Assistant Professor. Super proud of him. chrisharshaw.com
10, duh
On a scale of 10, how important do you think is the knowledge of probability and statistics when it comes to learning/understanding machine learning and data science?
是不是00后们已经不知道wap网页这回事了？ 当年诺基亚和摩托罗拉时代，正规网页几乎不可能在手机打开。 当年手机浏览器只能支持十几个字符宽、图片极小极模糊、排版极其简单的wap网页，跟html+css+js的网页完全不同。 国内当时所有门户网站都有wap或者3g门户，当时最大的作用就是下载游戏、铃声、…
Yes Paul (@pliang279), that bubbly is yours! 🥳🎉Congratulations on your very successful dissertation defense! (on the "Foundations of Multisensory Artificial Intelligence" in the @mldcmu ,@SCSatCMU, @CarnegieMellon ).
wrote this down more formally so that I can get it off my mind... arxiv.org/abs/2404.09946 If you find the original tweets lack context/background but find the topic interesting, the note might be helpful
At CISS hearing nice talks on model-based RL. MBRL has the reputation of bad "error compounding", but I realize recently that its theoretical root may be different from what ppl think... The problem may not be error accumulation over *time*, but the one-step error itself! 1/
I am excited for this upcoming talk by Andrew about "optimally" exploring given some offline data! Bonus: We'll hear about the gap between verifiable and unverifiable learning! I hope to see you tomorrow!
Tomorrow (Tuesday) 5pm UTC, Andrew Wagenmaker will present about "Leveraging Offline Data in Online Reinforcement Learning". Hosted by Csaba & Vlad.
And that's a wrap ! @ElanRosenfeld , it's been a pleasure and privilege to watch you grow and develop as a researcher. Congratulations on an impressive body of work --- I'm certain you are just getting started and you'll do great things.
Congrats @ElanRosenfeld on a great thesis that moves our understanding of distribution shift forward! w/ @risteski_a @boazbaraktcs @ShalitUri
Check out NPO, a simple objective for LLM unlearning.
LLM unlearning was mostly based on variants of gradient ascent (GA), susceptible to catastrophic forgetting. We propose Negative Preference Optimization (NPO), demonstrating efficient unlearning on TOFU benchmark. w/ @RuiqiZhang0614 @ Licong Lin, @yubai01. arxiv.org/abs/2404.05868
Upcoming Guest Lectures: Apr. 11 (Thu): Invited talk in Stanford NLP. Apr. 18 (Thu): 2 Guest lectures in Caltech. Apr. 24 (Wed): Remote guest lecture in UChicago. Upcoming travel plans: May. 1 (Wed) - May. 5 (Sun): New York May. 6 (Mon) - May. 12 (Sun): Vienna
Trends for United States
You might like
