Aryaman Arora @aryaman2020
phding @stanfordnlp. likes language(s). aryaman.io 🌲 Joined December 2018-
Tweets11K
-
Followers4K
-
Following2K
-
Likes27K
random thought: for LoRA/SFT, you host different model endpoints for different tasks. for ReFT, you can host a single model endpoint for different tasks. When serving, these interventions elicit different behaviors. They are batched and composable to elicit new abilities.
This weekend, we’re Natural Lake Processing! #NLProc
@srush_nlp Agree with the post that there is a distinction (and often implicit conflation) of behavioral and mechanistic induction heads. Having a behavior definition seems more natural to me, followed by specific computational implementations of that def (eg on a transformer) 🧵
did 20 hour training run (1) script ran evals on wrong split of the benchmark (2) model trained great but did not save properly conclusion: 20 hours wasted lol
does anyone have an interpretability explanation for why MoEs are better than FLOP-matched dense models
🚨New Paper🚨 We propose 1⃣CultureBank🌎 dataset sourced from TikTok & Reddit 2⃣An extensible pipeline to build cultural knowledge bases 3⃣Evaluation of LLMs’ cultural awareness 4⃣Insights into culturally-aware LLMs Project: culturebank.github.io Data: shorturl.at/hrtwP
san francisco is the greatest city in the world and the spiritual center of western civilization
i wonder if REFT / RepEng really is the future 🤯 cc @voooooogel
i wonder if REFT / RepEng really is the future 🤯 cc @voooooogel
Yesterday we had our first paper reading session in the @ai4bharat discord covering LoRA, DoRA and REFT (@aryaman2020). This is to be the first of many. Recordings: drive.google.com/drive/u/0/mobi… Link to discord: discord.com/invite/KSkKswzh Join us!
not so sure about how to interpret these results yet; but it shows (1) ReFT works on Llama-3; and (2) finetuned Llama-3 is pretty strong. LoRAs/PEFTs should finish at this point as well? would be interesting to compare🧐
very interesting that every frontier lab interp team is working on sparse autoencoders (SAEs) and ~ no one in academia is
New coding benchmark! We introduce the USACO benchmark (princeton-nlp.github.io/USACOBench/), which tests the limits of LLM intelligence in creative, grounded, algorithmic scenarios. Key takeaways below 👇
New coding benchmark! We introduce the USACO benchmark (princeton-nlp.github.io/USACOBench/), which tests the limits of LLM intelligence in creative, grounded, algorithmic scenarios. Key takeaways below 👇 https://t.co/voqfXQk540
need to take a week off to read Will Merrill papers
Sā́mapriyaḣ @avzaagzonunaada
10K Followers 7K Following Statistics post-doc. Interested in descriptive, historical & contact linguistics. Ossetic, Circassian, Kashmiri-Kishtwari, Coptic, Quileute.matt 🍉 @matt_boot_
12K Followers 3K Following language enjoyer. Egyptian Arabic enthusiast. he/هو/er/él. see my pinned tweetSameer ⌬ @dxrsam_0
2K Followers 1K Following Linguistics, mythology, mathematics, etc. Hobbyist. 𐫅𐫡 𐫀𐫏𐫗 𐫢𐫁 𐫘𐫏𐫀𐫝 𐫃𐫯 𐫃𐫢𐫤 𐫡𐫀𐫝 𐫖𐫞𐫘𐫇𐫅Alexander Jabbari @yakabikaj
18K Followers 596 Following Professor of sundry Persianate things. Tweeting about philology (Persian, Urdu, Semitic), language, literature, history, Muslim and Jewish stuff, MarxismA. Z. Forеman: Serio.. @azforeman
12K Followers 1K Following Historical linguist, grad student, medievalist, 1st amendment geek, translator. I write poems. I talk like dead people for fun. My pronouns are ergative.Luca Soldaini 🎀 @soldni
6K Followers 1K Following I like tokens! Lead for OLMo data team at @allen_ai (Dolma 🍇), OSS is fun, @QueerInAI organizer 🤖☕️🍕they/them (views mine, not my employer’s)चेदिराड.. @Saatvata
13K Followers 9K Following स्खलितानि भविष्यन्ति ममैतानि क्षमस्व भोः। नात्र घोषयितव्यं मे यत्किञ्चिदवशिष्यते॥Sagir Ahmed สกิ.. @SagirAhmed98
1K Followers 617 Following Insta: @sagir_langs Eastern Indo-Aryan, Assamese, Sylheti, Bodo-Garo, Tai & other languages, History, Linguistics, Scripts (esp Brahmic) 🇮🇳 he/সি/ꠢꠦ/बि/উৱাঁইTal Linzen @tallinzen
16K Followers 893 Following Professor @nyuling and @NYUDataScience, research scientist @GoogleAIShaily @shaily99
5K Followers 2K Following PhD @LTIatCMU Prev: @GoogleAI @MSFTResearch. Working on ethics and evaluation in #NLProc. Usually ranting, often about research & DEI. 📚 @readsndrantsNathan Schneider @complingy
4K Followers 1K Following Computational Linguist and Professional Nerd at Georgetown University he/him pronouns, ALL the prepositions @[email protected] @complingy.bsky.socialMarijn van Putten @PhDniX
29K Followers 842 Following Historical Linguist; Working on Quranic Arabic and the linguistic history of Arabic and Tamazight. Game designer @team18kKayo Yin @kayo_yin
8K Followers 560 Following PhD student @berkeley_ai @berkeleynlp working on interpretability and signed languages. Former @msftresearch @deepmind @carnegiemellon @polytechnique. 🇫🇷🇯🇵d̶͔̾o̴̔͜u̶͕̕.. @likethemagician
2K Followers 3K Following math👨🏫 linguist 🐝kpr, @deatherhixon's worse ½ ~ oqe/ghaH/𓇓𓅱/ⲛⲧⲁϥ/𒀀𒉈/𒋗𒌑/bí/ēme/tic/yè/yená/hã/à/re/are’/paluṟu/eh/𑀘𑁆𑀧𑀺/𐤡𐤦𐤳/𐬵𐬋/ᜐᜒᜌ/ia/ཁོ་ན/இவன்Maarten Kossmann @ait_kisou
3K Followers 645 Following Berber linguistics - linguistique amazighe - ⵜⵎⵓⵙⵏⵉ ⵏ ⵜⵎⴰⵣⵉⵖⵜsleepy linguist ✨�.. @l_linguist
2K Followers 2K Following ☕☕☕ Transylvanian, Cisatlantic, {lingu-; philolog-; oriental}ist by education, playground translanguageress by vocation. Translator from/into EN/DE/IT HungarianShruti Rijhwani @shrutirij
4K Followers 499 Following * Research Scientist @GoogleDeepMind * #NLProc research * PhD from @LTIatCMU * Amateur woodworker, scuba diver, foosball playerSameer ud Dowla Khan .. @SameerudDowla
1K Followers 1K Following phonetician of phonation | intonationologist | fag of color | overfeeding aunty | cat servant | i study speech sounds other than consonants and vowels | they/heMSS @sajwan_mellow
13 Followers 228 FollowingManling Li @ManlingLi_
3K Followers 421 Following Postdoc @Stanford, Incoming Assistant Professor @Northwestern, PhD @UIUC. Working on Knowledge Foundation Models, especially for Multimodal data (Language + X).Anu Chowdhury @a_chowdhury99
53 Followers 482 Following Data Science & AI @MorningstarInc | @Uni_of_Cambridge @Edinburgh_Uni gradWei-Lin Chen @wlchen0206
19 Followers 221 Following Incoming Ph.D. student @CS_UVA Prev. @ National Taiwan University @AIResearchAISTParth Sarthi @parthsarthi03
420 Followers 635 Following Researching Language Models @StanfordNLP, @StanfordAILab Prev: @ContextualAIHugh Zhang @hughbzhang
1K Followers 522 Following open source ai @scale_AI. co-created @gradientpub.Zhaoyang Wang @wangwan83764204
316 Followers 4K Following CS PhD student at Uni of Birmingham in the United Kingdom. Research interests: Automated Machine Learning, Online Learning, and Reinforcement Learning 🏳️🌈Lisa N. Berg Montgome.. @lisan_berg
2 Followers 30 FollowingElon Musk @EMusk50470
0 Followers 167 Following Entrepreneur 🚀| Spacex • CEO & CTO 🚔| Tesla • CEO and Product architect 🚄| Hyperloop • Founder 🧩| OpenAI • Co-founder 👇🏻| Build A 7-fig IG BusinessYihuai Hong @YihuaiH91773
25 Followers 157 Following CS Undergraduate interested in NLP research @SCUT previously Research Intern in @UCLMaheep Chaudhary | �.. @ChaudharyMaheep
41 Followers 509 Following MS @NTU || Collab w/ Stanford || Ex-MIT Driverless, UIUC.Sanjay @Sanjay_0508
121 Followers 869 Following 19 y/o • Data • AI/ML • Prev LFX '23 mentee @CloudNativeFdnTannon Kew @tannonk
130 Followers 239 Following phd student at @cl_uzh, working on controlled conditional text generation, text simplification and leveraging LLMsMark J. @MarkJ97517270
49 Followers 269 FollowingPedram Akbarian @pedakb
113 Followers 507 Following PhD Student @UTAustin | ML Theory • Optimization • StatisticsAjay Jain @ajayj_
6K Followers 3K Following Co-founder @genmoai. Co-created denoising diffusion (DDPM), DreamFusion, Dream Fields. Ex Ph.D. @berkeley_ai, @googleai, @facebookai, @nvidiaai, @mitVi @AvimanyuRoy3
581 Followers 2K Following 🍎🕊/🦦☕️/😴🛌/he/him Shouting into the Void (TM) GPU poor peasantCarraamoo Gaheedhaa @carraamoo
46 Followers 55 FollowingKB @katiebowles_
644 Followers 5K Following Advancing AI for Healthcare at Scale at @AbridgeHQ | $150M Series C 🚀 | We're Hiring!Daniel Johnson @_ddjohnson
2K Followers 576 Following Researcher at @GoogleDeepMind. PhD student at @VectorInst / @UofT. Building tools to study neural nets and find out what they know. He/him.adddddd @anandnew38
339 Followers 5K Followingusman sheikh @usmansheikh06
3 Followers 423 FollowingAidan @kool_aids3
9 Followers 71 Following. @lolkek221337
71 Followers 346 Followinggotem @gautam_sharda_
292 Followers 742 Following VLMs, search, rl, open-endedness. swe intern, @google gemini code assist, embedding models research. building @radah_aiRobonomous @realpolity101
2K Followers 2K Following controls,robotics & UAVs//Algos,CV//usual sh!tposter XD //Ryan Boyle @_RyanBoyle_
1K Followers 5K Following Tech Enthusiast 👨🏼💻 Aspiring ML Engineer. Frequent Traveler 🌎 Based in Philly & LA, Soon → SF 🌉Alpaca 🦙 (e/delve) @LeetAlpaca3
215 Followers 3K Following Servant of the future. If I blocked you, I probably didn’t mean to. (Big block list from long ago) Tweets are my own.Dilip Venkatesh @dipsivenkatesh
27 Followers 423 FollowingManBeast — e/acc @beastMan3b
551 Followers 2K Following Building https://t.co/6C5iNtfvWe . Time traveling Quantum realm King. Robo Lord. David Bloggin @ mediumSameera Lanka @samspacelanka
136 Followers 540 Following Machine Learning Scientist @Microsoft Previously @MSFTResearch, @NCStateTony Sun @tonysunai
7 Followers 272 FollowingYutong Zhang @zhangyt0704
12 Followers 92 Following cs master student @Stanford | previously undergrad @UofIllinoisalphaomega @alphaom39676319
7 Followers 59 FollowingSā́mapriyaḣ @avzaagzonunaada
10K Followers 7K Following Statistics post-doc. Interested in descriptive, historical & contact linguistics. Ossetic, Circassian, Kashmiri-Kishtwari, Coptic, Quileute.matt 🍉 @matt_boot_
12K Followers 3K Following language enjoyer. Egyptian Arabic enthusiast. he/هو/er/él. see my pinned tweetAbhishek Avtans अ�.. @avtansa
10K Followers 273 Following Lecturer (Indic language/s), Leiden University (Netherlands), KHS (Agra), JNU (Delhi), Hindi, Urdu, Linguistics, Lit, भाषा, भाषाविज्ञान, साहित्य 🇮🇳 🇳🇱 ✍️🚲(((ل()(ل() 'yoav))).. @yoavgo
46K Followers 2K FollowingSameer ⌬ @dxrsam_0
2K Followers 1K Following Linguistics, mythology, mathematics, etc. Hobbyist. 𐫅𐫡 𐫀𐫏𐫗 𐫢𐫁 𐫘𐫏𐫀𐫝 𐫃𐫯 𐫃𐫢𐫤 𐫡𐫀𐫝 𐫖𐫞𐫘𐫇𐫅Alexander Jabbari @yakabikaj
18K Followers 596 Following Professor of sundry Persianate things. Tweeting about philology (Persian, Urdu, Semitic), language, literature, history, Muslim and Jewish stuff, MarxismA. Z. Forеman: Serio.. @azforeman
12K Followers 1K Following Historical linguist, grad student, medievalist, 1st amendment geek, translator. I write poems. I talk like dead people for fun. My pronouns are ergative.Luca Soldaini 🎀 @soldni
6K Followers 1K Following I like tokens! Lead for OLMo data team at @allen_ai (Dolma 🍇), OSS is fun, @QueerInAI organizer 🤖☕️🍕they/them (views mine, not my employer’s)चेदिराड.. @Saatvata
13K Followers 9K Following स्खलितानि भविष्यन्ति ममैतानि क्षमस्व भोः। नात्र घोषयितव्यं मे यत्किञ्चिदवशिष्यते॥Yoïn van Spijk @yvanspijk
18K Followers 715 Following /'dʒowɪn/ • historisch taalkundige | historical linguist • dialectoloog | dialectologist • See my Tumblr for all graphics and videos: https://t.co/vDE6Wth705Sagir Ahmed สกิ.. @SagirAhmed98
1K Followers 617 Following Insta: @sagir_langs Eastern Indo-Aryan, Assamese, Sylheti, Bodo-Garo, Tai & other languages, History, Linguistics, Scripts (esp Brahmic) 🇮🇳 he/সি/ꠢꠦ/बि/উৱাঁইTal Linzen @tallinzen
16K Followers 893 Following Professor @nyuling and @NYUDataScience, research scientist @GoogleAIShaily @shaily99
5K Followers 2K Following PhD @LTIatCMU Prev: @GoogleAI @MSFTResearch. Working on ethics and evaluation in #NLProc. Usually ranting, often about research & DEI. 📚 @readsndrantsNathan Schneider @complingy
4K Followers 1K Following Computational Linguist and Professional Nerd at Georgetown University he/him pronouns, ALL the prepositions @[email protected] @complingy.bsky.socialDanish Pruthi @danish037
7K Followers 628 Following Faculty at Indian Institute of Science, Bangalore. PhD from @LTIatCMU.Marijn van Putten @PhDniX
29K Followers 842 Following Historical Linguist; Working on Quranic Arabic and the linguistic history of Arabic and Tamazight. Game designer @team18kDr. Hadas Kotek 🦄�.. @HadasKotek
6K Followers 1K Following Linguist in tech | NLP, data, safety | former academic | AltAc advocate | cat lover | feminist | she/her | demi | 🏳️⚧️ ally | BLM | powered by coffee & spiteGraham Neubig @gneubig
31K Followers 588 Following Associate professor at CMU, studying natural language processing and machine learning.Kayo Yin @kayo_yin
8K Followers 560 Following PhD student @berkeley_ai @berkeleynlp working on interpretability and signed languages. Former @msftresearch @deepmind @carnegiemellon @polytechnique. 🇫🇷🇯🇵Manling Li @ManlingLi_
3K Followers 421 Following Postdoc @Stanford, Incoming Assistant Professor @Northwestern, PhD @UIUC. Working on Knowledge Foundation Models, especially for Multimodal data (Language + X).Parth Sarthi @parthsarthi03
420 Followers 635 Following Researching Language Models @StanfordNLP, @StanfordAILab Prev: @ContextualAIHugh Zhang @hughbzhang
1K Followers 522 Following open source ai @scale_AI. co-created @gradientpub.Maheep Chaudhary | �.. @ChaudharyMaheep
41 Followers 509 Following MS @NTU || Collab w/ Stanford || Ex-MIT Driverless, UIUC.Daniel Johnson @_ddjohnson
2K Followers 576 Following Researcher at @GoogleDeepMind. PhD student at @VectorInst / @UofT. Building tools to study neural nets and find out what they know. He/him.Garry Tan @garrytan
433K Followers 4K Following President & CEO @ycombinator —Founder @Initialized—PM/designer/engineer who helps founders—YouTuber—San Francisco Democrat accelerating the boom loop—e/accYutong Zhang @zhangyt0704
12 Followers 92 Following cs master student @Stanford | previously undergrad @UofIllinoisJohn Carmack @ID_AA_Carmack
1.1M Followers 241 Following AGI at Keen Technologies, former CTO Oculus VR, Founder Id Software and Armadillo AerospaceHaotian Ye @haotian_yeee
92 Followers 79 Following CS PhD at @Stanford | BS at Yuanpei College, @PKU1898. Working on AI for Science | Generative AIVyoma Raman @VyomaRaman
28 Followers 203 FollowingDylan Patel @dylan522p
39K Followers 684 Following SemiAnalysis Boutique AI & Semiconductor Research and Consulting DMs are open for consulting, quotes, or to talk shopNikhil Prakash @nikhil07prakash
365 Followers 2K Following CS Ph.D. @KhouryCollege | Prev: Visiting Scholar @ MPI-SP @maxplanckpress; Intern @kixlab_kaist @SamsungBlr @iitrpr @HasuraHQAlexander Khazatsky @SashaKhazatsky
287 Followers 15 FollowingStanford AI Club @stanfordaiclub
88 Followers 4 Following Stanford’s premier student-led club focused on AI research and development.Anikait Singh @Anikait_Singh_
126 Followers 264 Following PhD Student @StanfordAILab, Previously Student Researcher @GoogleDeepMind, Undergraduate @Berkeley_AI Deep Learning, Reinforcement Learning, Robotics.Teortaxes▶️ @teortaxesTex
7K Followers 1K Following Ours is the age of unaligned utilitarians. Other problems are relatively unimportant, but sometimes I tweet about them anyway. (кто/кого)Pete @epwalsh
52 Followers 88 Following Research Engineer at @allen_ai. Lead engineer for OLMo pretraining.Josh Cason @TheGrizztronic
458 Followers 2K Following e/wfh | ⛏️ Train/Predict ETL @Demandbase (2k+ models) | Computational Linguist, MS @UW | Self Dev 🎨🎵 😂 💪 | Ex IBM Watson NLP | Inventor | ❤️∃∀ | FL Life 🐊Alexa R. Tartaglini @ARTartaglini
430 Followers 480 Following Research scientist @ the Human & Machine Learning Lab, NYU // Interested in representational alignment, interpretability, & more esoteric thingscarolyn zou @cqzou
317 Followers 259 Following hci/ai @northwesternu @comdatasci, soon @stanford phdNaman Jain @StringChaos
900 Followers 903 Following CS PhD @UCBerkeley | Projects - R2E, LiveCodeBench, Chatbot-Arena Coding, RAFT, Data Quality | Past: @AWS @MSFTResearch @iitbombayMarius Mosbach @mariusmosbach
714 Followers 877 Following Postdoc @Mila_Quebec & @mcgillu | NLP researcherSanjeev Arora @prfsanjeevarora
21K Followers 32 Following Director, @PrincetonPLI and Professor @PrincetonCS. Seeks math/conceptual understanding of deep learning and large AI models.latent space princeps @lumpenspace
3K Followers 481 Following ruthless osmopolitan, autist suprematist, ai influencee. 🕰️🌍🔜🪦;🌑🤼🤰🏻;👇⏱️|🐙,…| likes and retweets are pledges of undying loyalty.Ashwinee Panda @PandaAshwinee
945 Followers 605 Following PhD @princeton, @Cal alum, currently working on LLMsYuki @y_m_asano
3K Followers 683 Following Assistant Professor for Computer Vision and Machine Learning at the QUVA Lab, @UvA_Amsterdam. Previously @Oxford_VGG, @OxfordAI, interned @facebookaiMichael Bernstein @msbernst
16K Followers 2K Following @Stanford, Associate Professor of Computer Science. I design (better) social tech.Andrew Ng @AndrewYNg
1.0M Followers 913 Following Co-Founder of Coursera; Stanford CS adjunct faculty. Former head of Baidu AI Group/Google Brain. #ai #machinelearning, #deeplearning #MOOCsAmeen Patel @Ameen_ml
107 Followers 717 Following Staff ML Engineer @togethercompute Interested in deep learning and distributed systemsLiza Pertseva @LizaPertseva1
7 Followers 31 FollowingCas (Stephen Casper) @StephenLCasper
3K Followers 1K Following #AI safety & responsibility. PhD Candidate @ #MIT_CSAIL.Mert Yuksekgonul @mertyuksekgonul
4K Followers 676 Following (he/him) Computer Science PhD Candidate @Stanford @StanfordAILabKrista Opsahl-Ong @kristahopsalong
651 Followers 352 Following CS PhD candidate @Stanford @StanfordAILab || Prev @Google @MicrosoftEric J. Michaud @ericjmichaud_
1K Followers 773 Following PhD student at MIT. Trying to make deep neural networks among the best understood objects in the universe. 💻🤖🧠👽🔭🚀Jeremy Howard @jeremyphoward
222K Followers 5K Following 🇦🇺 Co-founder: @AnswerDotAI & @FastDotAI ; Hon Professor: @UQSchoolITEE ; Digital Fellow: @StanfordMaximilian Mozes @maximilianmozes
201 Followers 483 Following Member of Technical Staff @cohere. PhD @UCL/@ucl_nlp. Previously: @GoogleAI/@SpotifyResearch. He/Him.Pindar and the Rigved.. @ioplokamos
653 Followers 413 Following Comparative Antiquities, Classics and Indo-European Studies❤️🔥👑Tweeting My Dissertation📖🧵Craftsmen of words🛕🏛️ https://t.co/u320K5QlXwTracingWoodgrains @tracewoodgrains
22K Followers 1K Following Storyteller. CFP @TheBARPod. Pursue excellence. Eng/中文Anjali Menon @elegantunivers9
5K Followers 3K Following Founder @ https://t.co/14mH96cmxK : Building tech to stop the youth mental health crisis. Prev Vice President ifonly(acq. @mastercard) @magic (YC) Board @cwclubJames Bradbury @jekbradbury
11K Followers 8K Following Compute at @AnthropicAI! Previously JAX, TPUs, and LLMs at Google, MetaMind/@SFResearch, @Stanford Linguistics, @Caixin.typec (new acct) @typeclonghouse
1K Followers 425 Following you know who i am (help me find my friends, permab& on main) "The Mother Teresa of Incels"Ahmed Ahmed @AhmedSQRD
401 Followers 795 Following CS PhD @Stanford - Funding @KnightHennessy @NSF- 🇸🇩 - tweets include history & politicsI've historically been very critical of EA, but this is just about the worst critique of EA possible. It argues that they were secretly, nefariously talking about a Hidden Agenda in "core EA," on... *checks notes* publicly visible online forums and all of their in-person events.
🚨 Effective Altruism's Bait-and-Switch: From Global Poverty to AI Doomerism 🚨 The Effective Altruism founders planned – from day one – to mislead donors and new members in order to build the movement's brand and community. aipanic.news/p/effective-al…
It will be a bittersweet goodbye to DC/Georgetown and the wonderful people here (and Tatte 🤤) 🥺🥹 but it's also time to move onto the next chapter and check out what's in store for me 🛒🤩🤗 [4/4]
2⃣ Prior to joining Pitt, I will do a postdoc at @MaiNLPlab directed by @barbara_plank 😇 Excited about this opportunity and to be moving to Munich 🇩🇪 later this year 🕺🏼 I'm determined to get my "used-to-be-B1" German back (perhaps with a Bavarian touch 😉) [3/4]
I will be joining the wonderful colleagues at Pitt Linguistics (@pittprofdude - most not on X) and especially Dan Villarreal and Na-Rae Han to to further develop and advance the compling program at Pitt! Also looking forward to getting to know folks @PittCompSci and CMU🤗 [2/4]
today's our last day of classes, so let me share 2 exciting updates 🥁 1⃣ Starting Fall 2025, I will begin as an Assistant Professor of Computational Linguistics at the University of Pittsburgh @PittTweet 🤠 Looking forward to exploring Pittsburgh (send all your recs)! [1/4]
Everything in the world is about FLOPS except FLOPS. FLOPS are about power.
This new writeup by @cHHillee uncovered some very unexpected reasons for why we can never reach the theoretical TFLOPS advertised by accelerator vendors thonking.ai/p/strangely-ma… spoiler: it's all about the power. Make sure to read it!
Jelani makes an excellent point I would go further and extend this to the graduate level with fellowships from both government (e.g. NSF, NDSEG) and industry (e.g. Google, Meta). Marginal benefit for students/faculty is often greater elsewhere. - signed by Cornell/Stanford alum
Also, donors: donate to state schools. Eg Berkeley graduates more undergrads per year than the total 4-year undergrad population of Stanford, and with a much smaller endowment/budget. Higher % Pell grant recipients (almost double), higher % first-gen (50% higher). 1/
I'm in the Baltimore/DC area this week to share some cool stuff we've been cooking! I'll give this talk at UMBC on Monday, Georgetown on Tuesday, UMD on Wednesday, and as a poster at MASC at JHU on Friday. If you're around and want to chat please ask! Happy to come say hi! 😃
A wealth of results that many people will find valuable. GPT-3.5 vs GPT-4 with & without DSPy optimizers. Optimized GPT-3.5 calls are consistently more accurate here than uncompiled GPT-4 calls.
Authors @AugustinToma @RonaldXie1 @SPalayew @BoWang87 won all three sub-tasks of MEDIQA-CORR 2024 with their programs. Check out their thread (above) and paper (below). The image shows the added gains by optimizing the language programs in DSPy. arxiv.org/pdf/2404.14544
1. I worked my ass off during my undergraduate at a tier 3 college. This mostly involved memorization. A lot of time wasted. 2. I spent a year of my Masters being depressed. A lot of time wasted. 3. I spent most of my PhD being depressed and borderline suicidal. A lot of time…
Still coping with how I wasted my first 2 years at college doing nothing 🥲 My prime years, going to waste Rebounding now ofc, but it would be nice to have had that headstart 🫠 anyone relate? lol
Can you figure out how many interacting circuits are involved in a behavior just by looking at loss curves? Maybe! In this cool paper, @Aaditya6284 et al. study the emergence of circuits in isolation by retraining models with certain activations "clamped" to post-training values
In-context learning (ICL) circuits emerge in a phase change... Excited for our new work "What needs to go right for an induction head (IH)?" We present "clamping", a method to causally intervene on dynamics, and use it to shed light on IH diversity + formation. Read on 🔎⏬
there's a lot of pressure to date when you're young but nobody talks about the people who could be doing their best work in their twenties but end up tangled in things that lead to absolutely nothing
@albfresco you don't know how bad gpt-2 is if you think that.
large language model buzzwords • it's not a language model, it's an intelligent assistant • that's not a line that goes up in to the right, that's an emergent capability • you're not running a hyperparameter sweep and making a graph, you're proposing a new scaling law
@aryaman2020 @ZhengxuanZenWu @Demandbase @stanfordnlp @awscloud do you guys want to come give a talk at Scale? we're in SF
@aryaman2020 @Demandbase @stanfordnlp @awscloud please work on the slides
Oh, look, I hear there's a DSPy meetup in San Francisco on Wednesday (May 1st) from 5:30 PM - 8:30 PM with @ArizePhoenix, @cohere, @weaviate_io. See you there, everyone!
We've aggregated all of the DSPy resources from the Weaviate team on one page! It is broken into two categories: 1. Hands on Learning and 2. Read and Listen 📚 DSPy round-up: weaviate.io/developers/wea… We're also very excited to host our in-person meetup with DSPy, @arizeai, and…