-
Tweets1K
-
Followers56K
-
Following491
-
Likes5K
In AI research there is tremendous value in intuitions on what makes things work. In fact, this skill is what makes “yolo runs” successful, and can accelerate your team tremendously. However, there’s no track record on how good someone’s intuition is. A fun way to do this is…
The first lecture of our @Stanford CS25 V4 Transformers course (cs25.stanford.edu) is now released! Check it out here: youtube.com/watch?v=fKMB5U…. We (the instructors) gave a brief intro and overview of the history of NLP, Transformers and how they work, and their impact. We…
nothing gets my heart rate up like waiting for eval results on new models to come in
Congrats @YiTayML on this launch. It is impressive that a small team can train a strong model so quickly. What I also like is that the PR is not full of unfounded hype. Just plainly states the model's benchmark scores and you can immediately try out the model yourself for free.
Congrats @YiTayML on this launch. It is impressive that a small team can train a strong model so quickly. What I also like is that the PR is not full of unfounded hype. Just plainly states the model's benchmark scores and you can immediately try out the model yourself for free.
Flan-2 is published in JMLR jmlr.org/papers/v25/23-…. I think it's a nice piece of history. The work scaled instruction tuning with respect to model size and finetuning tasks, which both improved performance. Our MMLU was 75%, SOTA when the paper came out in Oct 2022. Our…
In 2022, a model with 70%+ MMLU score, would cost 20 dollars per 1M tokens (instructGPT 3.5). Today it costs less than $1! It is perfectly reasonable to expect that in say five years, you will be able to use a model with 90%+ MMLU score for just a few cents per 1M tokens.
In 2022, a model with 70%+ MMLU score, would cost 20 dollars per 1M tokens (instructGPT 3.5). Today it costs less than $1! It is perfectly reasonable to expect that in say five years, you will be able to use a model with 90%+ MMLU score for just a few cents per 1M tokens.
This new hallucinations eval by GDM friends is in the right direction in many ways: 1. Tackles the scenario of extremely long-form responses, which is a harder but more realistic setting 2. Extracts the number of relevant facts, then browses to verify each individual fact 3.…
This new hallucinations eval by GDM friends is in the right direction in many ways: 1. Tackles the scenario of extremely long-form responses, which is a harder but more realistic setting 2. Extracts the number of relevant facts, then browses to verify each individual fact 3.…
Cheesy realization: studying history underscores how special this current moment in AI is. In past eras, the great powers of the world fought religious wars, sailed to unexplored lands, and built the first industrial cities. Now we will race to build artificial intelligence. So…
Had a bit of a fanboy moment today meeting @bryan_johnson, who has been super inspirational to me in prioritizing my health. I asked him about the best way to balance career and spending time on health. His advice is that while many people give up sleep to work more, sleeping…
My mental model of Sora is that it is the “GPT-2 moment” for video generation. GPT-2, which came out in 2018, could generate paragraphs of text that are coherent and grammatically correct. GPT-2 wasn’t able to write an entire essay without making mistakes like being inconsistent…
My typical day as a Member of Technical Staff at OpenAI: [9:00am] Wake up [9:30am] Commute to Mission SF via Waymo. Grab avocado toast from Tartine [9:45 am] Recite OpenAI charter. Pray to optimization Gods. Learn the Bitter Lesson [10:00am] Meetings (Google Meet). Discuss how to…
An incredible skill that I have witnessed, especially at OpenAI, is the ability to make “yolo runs” work. The traditional advice in academic research is, “change one thing at a time.” This approach forces you to understand the effect of each component in your model, and…
A key insight from chain-of-thought is around the idea of information density. Language models can only do so much with a single forward pass, and so the amount of compute the language model can use must be scaled proportional to how hard a prompt is to solve. What is…
One thing in AI research that I have finally recognized with clarity is the idea of “inertia bias”: continuing to do something when it’s not the best option. The most basic instance of inertia bias is the feeling of “I already spent time implementing X, so let me continue trying…
There’s no adrenaline rush like launching a massive gpu training
For most companies, hiring more people is strictly better. However, this is often not true in AI research. AI research is often bottlenecked by compute, and when this is the case, hiring more researchers can be counter-productive. I remember back at Google Brain, my manager once…
Yi Tay @YiTayML
29K Followers 97 Following chief scientist / cofounder @RekaAILabs 🫠 past: research scientist @google brain 🤯 currently learning to be a dad 🍼(((ل()(ل() 'yoav))).. @yoavgo
46K Followers 2K FollowingPercy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | PianistDelip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈Eric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0pRosanne Liu @savvyRL
33K Followers 966 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDRHorace He @cHHillee
23K Followers 449 Following Working at the intersection of ML and Systems @ PyTorch "My learning style is Horace twitter threads" - @typedfemaleKyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).Sam Bowman @sleepinyourhat
35K Followers 3K Following AI alignment + LLMs at NYU & Anthropic. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.Akari Asai @AkariAsai
11K Followers 650 Following Ph.D. student @uwcse & @uwnlp. NLP. IBM Ph.D. fellow (2022-2023). Meta student researcher (2023-) . ☕️ 🐕 🏃♀️🧗♀️🍳Shane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)Graham Neubig @gneubig
31K Followers 586 Following Associate professor at CMU, studying natural language processing and machine learning.Jacob Andreas @jacobandreas
14K Followers 958 Following Teaching computers to read. Assoc. prof @MITEECS / @MIT_CSAIL (he/him). https://t.co/5kCnXHjtlY https://t.co/2A3qF5vdJwFelix Hill @FelixHill84
9K Followers 777 Following Research Scientist, Deepmind I try to think hard about everything I tweet, esp on 90s football and 80s music None of my opinions are really someone else'sTim Dettmers @Tim_Dettmers
29K Followers 820 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.Ethan Caballero is bu.. @ethanCaballero
8K Followers 2K Following ML PhD student @Mila_Quebec ; previously @GoogleDeepMindThomas Wolf @Thom_Wolf
68K Followers 4K Following Co-founder and CSO @HuggingFace - open-source and open-scienceSara Hooker @sarahookr
39K Followers 7K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.Behnam Neyshabur @bneyshabur
18K Followers 690 Following Senior Staff Research Scientist @GoogleDeepMind, Interested in reasoning w. LLMs, traveling & backpackingcoffee & AI @realcoffeeAI
44 Followers 600 Following김동현 @GguVK7y5wlgfgqP
3 Followers 150 FollowingAVINASH VIS @AVIS408
1 Followers 119 Followingybtsdst @ybtsdst_hz
26 Followers 1K Followingnavj @pranav_jad
1 Followers 54 Followingsq z @s1833815
2 Followers 19 FollowingWendy @Wendy419642709
5 Followers 398 FollowingJay Bee @JayBee123460856
212 Followers 1K Following文 杨 @wenyang1034488
2 Followers 95 FollowingDakota Durfee @dakota_durfee
26 Followers 72 FollowingShivin Verma @verma_shivin
0 Followers 106 FollowingSiddharth @randforestgump
34 Followers 797 Following dare mighty things. analytics @NortheasternCOE, MA. machine learning and spatiotemporal stats. chelsea fc 💙 ex-Fidelity and ZoomRxMarcelo Tomaz @MarceloTomaz
28 Followers 51 FollowingSonakshi Chauhan @ChauhanSon8200
12 Followers 36 FollowingHfnRtd @HfnRtd
0 Followers 63 FollowingSaumil Patel @saumilp_
2K Followers 999 Following 🚀 Co-Founder & CEO @ https://t.co/2rmaJyjkus (YC S21) | 🤖 SWE In a symbiotic relationship with AI | Sova (iykyk) | e/acchua @james1024y
0 Followers 138 FollowingMichieO @michlite
110 Followers 789 FollowingMr.Li 李先生 @FelixLee2022
14 Followers 105 Following Shenzhen,China. Business travel in USA. Mobile phone/ tablet/ IOT etc.brianways @brianways65584
2 Followers 195 FollowingTari @Tari14918197
9 Followers 193 FollowingAmit Kumar Singh Yada.. @Aksy01021999
184 Followers 720 Following he/him | ECE PhD candidate at Purdue University, VIPER Lab | Director Medalist, BTech- IIT Gandhinagar | Ex @Enphase | Ex-Rakuten ResearchClaire McTaggart @McTaggartClaire
230 Followers 983 Following Founder of @SquarePegHires, a data driven hiring platform.Kaory,miya◦◦◦�.. @OeZWewKMTt11927
3K Followers 7K Following Bonjour🫶🦋✨ Je suis un Japonais ✨sérieux et plein de curiosité🦋 Sérieusement, j'en ai trop dit💦 Optimiste 🌺✨ Français🇫🇷✨Étudier📚🖋️✨HolyDifficult @HolyDifficult
9 Followers 146 FollowingShuminWang @ShuminWang7
1 Followers 29 FollowingHiếu Nguyễn @HiuNguy71624401
0 Followers 141 FollowingSamy C. @Samy_Gen_AI
9 Followers 35 Following I share insights on AI, No-Code and Product mindset | Digital Strategy Consultant | Worked with fortune 500 companieselbert @elbert866777443
14 Followers 40 FollowingViviana @Viviana75842443
2 Followers 161 FollowingSebastián Uría @SebastinUra1
99 Followers 539 FollowingAakash Rana @AakashRana000q
1 Followers 16 FollowingKanhaiya @kanhaiya2wit
30 Followers 125 FollowingJan @Jan___Lucas
14 Followers 170 FollowingAlexander Morosow @alex5m6
3 Followers 35 Following Head of Creative Engineering & Software Architect @refikanadol studio | @datalandmuseum | simplify omnidirectional motionLily_Anne@ @LilyAnnne_Gucci
683 Followers 503 Following Entrepreneur💻 Vietnamese American🇺🇸🇻🇳 - Texas 🦬, Free girl 👸👸, Active member of the charity community for children 👶❤️Yi Tay @YiTayML
29K Followers 97 Following chief scientist / cofounder @RekaAILabs 🫠 past: research scientist @google brain 🤯 currently learning to be a dad 🍼(((ل()(ل() 'yoav))).. @yoavgo
46K Followers 2K FollowingPercy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | PianistEric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0pRosanne Liu @savvyRL
33K Followers 966 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDRAnthropic @AnthropicAI
261K Followers 26 Following We're an AI safety and research company that builds reliable, interpretable, and steerable AI systems. Talk to our AI assistant Claude at https://t.co/aRbQ97uk4d.Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGzSam Bowman @sleepinyourhat
35K Followers 3K Following AI alignment + LLMs at NYU & Anthropic. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.Akari Asai @AkariAsai
11K Followers 650 Following Ph.D. student @uwcse & @uwnlp. NLP. IBM Ph.D. fellow (2022-2023). Meta student researcher (2023-) . ☕️ 🐕 🏃♀️🧗♀️🍳Shane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)Christopher Manning @chrmanning
126K Followers 115 Following Director, @StanfordAILab. Assoc. Director, @StanfordHAI. Founder, @stanfordnlp. Prof. CS & Linguistics, @Stanford. IP @aixventureshq. 🇦🇺 Do #NLProc & #AI. 👋Graham Neubig @gneubig
31K Followers 586 Following Associate professor at CMU, studying natural language processing and machine learning.Jacob Andreas @jacobandreas
14K Followers 958 Following Teaching computers to read. Assoc. prof @MITEECS / @MIT_CSAIL (he/him). https://t.co/5kCnXHjtlY https://t.co/2A3qF5vdJwFelix Hill @FelixHill84
9K Followers 777 Following Research Scientist, Deepmind I try to think hard about everything I tweet, esp on 90s football and 80s music None of my opinions are really someone else'sTim Dettmers @Tim_Dettmers
29K Followers 820 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.Ethan Caballero is bu.. @ethanCaballero
8K Followers 2K Following ML PhD student @Mila_Quebec ; previously @GoogleDeepMindThomas Wolf @Thom_Wolf
68K Followers 4K Following Co-founder and CSO @HuggingFace - open-source and open-scienceSara Hooker @sarahookr
39K Followers 7K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.Dwarkesh Patel @dwarkesh_sp
54K Followers 699 Following Being pretrained Host of Dwarkesh Podcast https://t.co/3SXlu7fy6N https://t.co/rEhnfYywXY https://t.co/hQfIWdM1UnJared Quincy Davis @jaredq_
643 Followers 308 Following Founder and CEO, Foundry. @mlfoundry Orchestrating Compute. Fmr Research Scientist @DeepMind, Deep Learning Team. CS PhD @Stanford. ML, Distributed SystemsJason Weston @jaseweston
9K Followers 568 Following Research @MetaAI+NYU. Pretrain+FT: NLP from Scratch (2011). Multilayer attention+position embed+LLM: MemNets (2015). Recent (2023+):Sys 2 Attn, Self-Rewarding..Brian Ichter @brian_ichter
1K Followers 178 Following Research Scientist at Google Brain, interested in robotics and AIHelen Qu @_helenqu
226 Followers 66 Following supernovae / cosmology / machine learning ✨ incoming research fellow @FlatironCCA, prev: PhD @physatpenn ‘24, BSE @CIS_Penn '17NetMind.AI @NetmindAi
29K Followers 92 Following NetMind Power is a decentralized platform aimed at democratizing AI computing power. Telegram: https://t.co/cYOXxXdzRT ; Discord: https://t.co/YStJyP1T1iKai Zou @anMe_kz
4K Followers 40 Following Founder and CEO at https://t.co/YPVwP0HF5C, https://t.co/QRm3Mj3azx, https://t.co/rG5uII6TfJIan Osband @IanOsband
8K Followers 365 Following Research scientist at OpenAI working on decision making under uncertainty.Ben Kuhn @benskuhn
7K Followers 290 Following Care a lot and try hard • making language models safer @AnthropicAI • prev CTO @WaveSenegal 🐧❤️Richard Socher @RichardSocher
101K Followers 970 Following CEO @youSearchEngine Investing at @aixventuresHQ Before: Stanford Adj Prof in AI/NLP, Chief Scientist at Salesforce, MetaMindYutong Bai @YutongBAI1002
3K Followers 397 Following EECS Rising Star, 2023 Apple Scholar, Visiting PhD @berkeley_ai, Intern @GoogleAI Brain team @MetaAI (FAIR Labs), CS PhD @JHUCompSciWei Xu @cocoweixu
9K Followers 1K Following CS professor @GeorgiaTech @gtcomputing @ICatGT @mlatgt. Natural language processing, machine learning, social media research.Keisuke Sakaguchi @KeisukeS_
1K Followers 437 Following Assoc. Prof. at Tohoku University, Sendai 🇯🇵. Natural Language Processing, Machine Learning, Psycho&Neurolinguistics. ex. @allen_ai @jhuclsp @NAIST_MAIN_ENYoung @yjkim362
343 Followers 262 Following Principal Researcher, Large language models, NLP, @Microsoft GenAICade Gordon @CadeGordonML
755 Followers 587 Following Working at the intersection of Bio x ML🧬 | @BerkeleyML Prev: @BigHatBio | LAION-5B & open_CLIP | ML Intern @CohereAI | research @UICCS,Chuck Ganapathi @chuckganapathi
1K Followers 461 Following President & COO @GainsightHQ | former Founder & CEO, @tact_ai | former SVP & GM Products at @salesforceJinYeong Bak @NoSyu
707 Followers 669 Following 박진영/JinYeong Bak/朴秦永 Leader @ https://t.co/fOvQXsEjo6 Conversation Modeling Researcher who is not good at talkingLawrence H. Summers @LHSummers
326K Followers 706 Following Charles W. Eliot University Professor and President Emeritus at Harvard. Secretary of the Treasury for President Clinton and Director of NEC for President ObamaPika @pika_labs
116K Followers 53 Following Video on command. Website: https://t.co/G5bjmrMQsx Discord: https://t.co/bX68ThPTQH About: https://t.co/atvdcgbe9SDemi Guo @demi_guo_
22K Followers 693 Following Co-founder & CEO @pika_labs | ex @StanfordAILab @HarvardChelsea Sierra Voss @csvoss
10K Followers 1K Following engineeress ✨ Member of Technical Staff @openai serious play // notice your curiosityYash Dagade @YashDagad
17 Followers 189 Following Tinkerer(always). Reader(sometimes). Writer(rarely). Funny?Marin-Llobet @Arnauya
516 Followers 1K Following Hi, this is Arnau. I’m PhDing @hseas @harvard !! brain machine interfaces, neuroai, neuromorphics… previously @UPCTelecos.Liv Boeree @Liv_Boeree
254K Followers 495 Following Looking for the win/wins in life. Not a fan of Moloch traps. Brand new podcast out now, link below👇Bret Taylor @btaylor
139K Followers 2K Following Co-Founder @SierraPlatform. Board @OpenAI @Shopify.Kevin Scott @kevin_scott
28K Followers 692 Following Chief Technology Officer @Microsoft; Host of #BehindTheTech podcast https://t.co/05oKfZqU3e; Author of "Reprogramming the American Dream"Rosie @RosieCampbell
6K Followers 869 Following Forever expanding my nerd/bimbo Pareto frontier. Policy Frontiers team lead @OpenAI.Voyage AI @Voyage_AI_
2K Followers 164 Following Building embedding/vectorization models, customized for your domain and company, for better retrieval quality https://t.co/MEAhTpBQqdAleksander Madry @aleks_madry
31K Followers 166 Following Head of Preparedness at OpenAI and MIT faculty (on leave). Working on making AI more reliable and safe, as well as on AI having a positive impact on society.Tri Dao @tri_dao
18K Followers 364 Following Incoming Asst. Prof @PrincetonCS, Chief Scientist @togethercompute. Machine learning & systems.Nat McAleese @__nmca__
3K Followers 305 Following Superalignment by models helping humans help models help humans at OpenAI. Previously @DeepMind. Views my own.Tao Xu @txhf
6K Followers 888 Following Learning Machine at OpenAI, previously Airbnb, Quora, Facebook and Microsoft.Giambattista 'Gb' Par.. @giambattista92
2K Followers 445 Following Reasoning about reasoning to understand understanding Research scientist at @OpenAI ML PhD @MPI_IS & @ETH Zurich Prev also @DeepMind and Google XSorry but this is actually Top 3 benchmarks to *not" use.
Agree. Here are the top three LLM benchmarks I would recommend: 1. Open LLM leaderboard 2. MT-Bench 3. AlpacaEval
@RuiboLiu I worked on 3-5 projects at once during my PhD and also during my time at google. I don't think it's about what you care about but more of individual preference.
I believe any PhD who cares quality more than quantity works in single thread mode.
Out of curiosity, do AI PhDs normally work (lead) on several projects simultaneously? I have never managed to work on more than one project during my PhD and I tried to convince my students not to do so. The paradigm might have already changed, so I am asking here.
I don't think it's productive or effective for a PhD student to ever lead more than 1 project simultaneously. If anything, I think leading 0.5 projects is even better (see SWE-bench & SWE-agent which Carlos and John co-led) Focusing is really important.
Out of curiosity, do AI PhDs normally work (lead) on several projects simultaneously? I have never managed to work on more than one project during my PhD and I tried to convince my students not to do so. The paradigm might have already changed, so I am asking here.
@OfirPress I would disagree, I definitely think there are students who lead multiple projects effectively. Focus is good, but focus doesnt imply efficacy. Diversified portfolios have their own value.
Very excited to see this come out:
Do models need to reason in words to benefit from chain-of-thought tokens? In our experiments, the answer is no! Models can perform on par with CoT using repeated '...' filler tokens. This raises alignment concerns: Using filler, LMs can do hidden reasoning not visible in CoT🧵
It feels like once you hyper specialise in AI and engineering/science you somehow lose all ability to reason with the finance/business like tax laws. They both require similar reasoning skills sometimes but it's almost as though my brain just shuts off and cannot process. 🫠
Bad sleep for me is usually from: 1. Eating late 2. Too much/wrong kinds of food 3. Skipping 30 min wind down before bed 4. Stimulants too late in the day 5. Bed/room temp too hot/cold 6. Disruptions: noise, others
No one will remember what you tweeted; they will remember what you built.
Bets are basically experiment preregistration
In AI research there is tremendous value in intuitions on what makes things work. In fact, this skill is what makes “yolo runs” successful, and can accelerate your team tremendously. However, there’s no track record on how good someone’s intuition is. A fun way to do this is…
@RuiboLiu @_jasonwei @hwchung27 damn i wish i never left google so i could still find out how wrong i was at predicting LLM chess.
Reminder that your bedtime is your most important appointment of the day. Respect yourself and be on time.
@YiTayML @_jasonwei I just checked that doc. The most closest guess was from @hwchung27 actually. Everyone else was just so wrong ...
Cool piece from the Financial Times comparing hallucinations in LLMs to hallucinations in humans! People often complain about how LLMs frequently hallucinate, but it’s easy to forget that humans hallucinate a lot as well. For example, if you read some article and then later tell…
phi is a good litmus test to tell who understands LLMs and who doesn't.
people think UL2 is an encoder-decoder. if you think it is, you haven't read the paper. The UL2 objective is agnostic to architecture.
the problem of being a xoogler is that you refer to RSUs as GSUs all the time.
Flash is OP!
サイズを制限したなかで特に良い汎用モデル ①Google:Gemini 1.5 Pro ②Meta:LLaMA 3(8B、70B) ③Anthropic:Claude 3(Haiku、Sonnet) ④Reka:Reka Flash(21B)