Sebastian Raschka @rasbt
Machine learning & AI researcher writing at https://t.co/A0tXWzG1p5. LLM research engineer @LightningAI. Previously stats professor at UW-Madison. sebastianraschka.com/books/ United States Joined October 2012-
Tweets15K
-
Followers265K
-
Following901
-
Likes19K
It’s here! Meet Llama 3, our latest generation of models that is setting a new standard for state-of-the art performance and efficiency for openly available LLMs. Key highlights • 8B and 70B parameter openly available pre-trained and fine-tuned models. • Trained on more…
Super nice and concise slide deck introducing instruction finetuning & alignment for LLMs! Also love that it starts with a clear definition of the terminology jargon. (We often use "finetuning" to mean instruction finetuning, which can be very confusing coming from classic ML.)
Super nice and concise slide deck introducing instruction finetuning & alignment for LLMs! Also love that it starts with a clear definition of the terminology jargon. (We often use "finetuning" to mean instruction finetuning, which can be very confusing coming from classic ML.) https://t.co/oUhosPMBwQ
Thanks @_akhaliq 🔥Surprisingly!! we figured out simple recipe to pre-train small language models with 1 GPU for less than half a day. Traditionally small base LMs w/ billion params (1B-2B) are trained on trillion tokens and it’s extremely compute intensive.
Thanks @_akhaliq 🔥Surprisingly!! we figured out simple recipe to pre-train small language models with 1 GPU for less than half a day. Traditionally small base LMs w/ billion params (1B-2B) are trained on trillion tokens and it’s extremely compute intensive.
Orca 2.5 7B based on Direct Nash Optimization, Mixtral 8x22B, Griffin, CodeGemma, @karpathy's llm.c, Gemini 1.5's audio, Gemini 1.5 Pro, new GPT-4 Turbo, Meta's & Google's new AI chips ... currently traveling, what else did I miss? 😅 Oh, and Llama 3 is also coming out next…
Freshly merged: you can now evaluate LLMs in LitGPT with just one command (thanks to the great LLM Eval Harness)! github.com/Lightning-AI/l…
Bojan Tunguz @tunguz
186K Followers 7K Following Machine Learning ex Nvidia. Kaggle Quadruple Grandmaster. Data Scientist. Physicist. Catholic. Husband. Father. Stanford Alum. e/xgb. XGBoost.eth. AMDG.Santiago @svpino
350K Followers 444 Following I tell stories about technology and teach hard-core Machine Learning at https://t.co/iZifcK7n47. YouTube: https://t.co/pROi08OZYJFrançois Chollet @fchollet
468K Followers 768 Following Deep learning @google. Creator of Keras. Author of 'Deep Learning with Python'. Opinions are my own.Yann LeCun @ylecun
708K Followers 716 Following Professor at NYU. Chief AI Scientist at Meta. Researcher in AI, Machine Learning, Robotics, etc. ACM Turing Award Laureate.Mark Tenenholtz @marktenenholtz
114K Followers 544 Following Head of AI @PredeloHQ. XGBoost peddler, transformer purveyor.Andrew Ng @AndrewYNg
1.0M Followers 908 Following Co-Founder of Coursera; Stanford CS adjunct faculty. Former head of Baidu AI Group/Google Brain. #ai #machinelearning, #deeplearning #MOOCselvis @omarsar0
188K Followers 480 Following Building with LLMs @dair_ai • Prev: Meta AI, Galactica LLM, PapersWithCode, Elastic, PhD • Creator of the Prompting Guide (~4M learners)AK @_akhaliq
308K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo follow on Hugging Face: https://t.co/q2Qoey80GxMatt Harrison @__mharrison__
158K Followers 890 Following Python 🐍 + Data Science 🚀 trainer @__metasnake__ 🦜 Speaker ✍ Author 👨🏫 Instructor (@Stanford) 📣 DM for SponsorshipJim Fan @DrJimFan
228K Followers 3K Following @NVIDIA Sr. Research Manager & Lead of Embodied AI (GEAR Lab). Creating foundation models for Humanoid Robots & Gaming. @Stanford Ph.D. @OpenAI's first intern.Akshay 🚀 @akshay_pachaar
133K Followers 419 Following Simplifying LLMs, MLOps, Python & Machine Learning for you! • AI Engineering @LightningAI • Lead DataScientist • BITS Pilani • 3 PatentsJean de Nyandwi @Jeande_d
38K Followers 768 Following Deep Learning, Vision 🤍 Language, Multimodal LLMs • AI Education • CMU Research blog: https://t.co/1BEFLZAqe7 ML Pack: https://t.co/7PkTyDvuriAlfredo Canziani @alfcnz
86K Followers 269 Following Musician, math lover, cook, dancer, 🏳️🌈, and an ass prof of Computer Science at New York Universityabhishek @abhi1thakur
81K Followers 661 Following 🤗 I build AutoTrain @huggingface 👨🏽💻 World's First 4x Grand Master @kaggle 🎥 YouTube 100k+: https://t.co/BHnem8fTu5 ⭐ GitHub StarChris Albon @chrisalbon
86K Followers 2K Following Director of Machine Learning at the Wikimedia Foundation. We host Wikipedia.Pau Labarta Bajo @paulabartabajo_
44K Followers 222 Following The Real-World ML guy | Learn to build real-world ML apps at https://t.co/xWr8Hm8zI5Dan | Machine Learnin.. @DanKornas
53K Followers 503 Following 🤖 ML Engineer 🔬 AI Educator 💻 I help you build AI skills through project-based learning ➡️ https://t.co/lC2UKMtRjjHarrison Kinsley @Sentdex
71K Followers 201 Following Neural networks from Scratch book: https://t.co/MWlYbXicwc YouTube: https://t.co/5osPue5EW9 @skunkworks_aiPyTorch @PyTorch
378K Followers 77 Following Tensors and neural networks in Python with strong hardware acceleration. PyTorch is an open source project at the Linux Foundation. #PyTorchFoundationArturo Villacañas @artuvillacanas
54 Followers 147 Following Interests: AI Safety & Security. Currently: @kasl_ai. Prev: @CISPA, @IMDEA_Software, @CCNCERT. 🏳️🌈Claudio @Claudio29359812
0 Followers 13 Followingₕₐₘₚₜₒₙ.. @Hamptonism
1K Followers 2K Following Founder @◉⃤ | paris│ founder of quant/acc. vampiric pursuit of quant finance — e\acclv @reboy_lee
0 Followers 43 Followingxpppopoo @xpppopoo79503
0 Followers 49 FollowingLindariellle @lindariellle
2K Followers 3K Following https://t.co/EuBqRNf0mG https://t.co/xnE6h6HmtY @stakeus https://t.co/ZvPC2kUbeQ "Hi! Hey! It's me! It's Linda! Come say HEY!"David Cantu @KennyMaert21378
318 Followers 2K FollowingEmmanuel Piard @epiard13
167 Followers 985 Following 👨🏾💻The ability to manufacture motivation is a scary trait. 👨🏾🎓Learning everyday. 💑 Proud husband📚🏀⚽️🏐🏓🐒 Views = my ownten ben @BasedTB
69 Followers 66 FollowingDavide Caponio @videc_
425 Followers 5K FollowingEC Martins @ECMartinz
288 Followers 3K FollowingJun Tiger @vuongthieuquan
0 Followers 11 FollowingChord ai @chordaiapp
589 Followers 1K Following iOS / Android app to get chords, voicing & beats for any song 🎸🎹📱 Developed by former PhD students in ML and music enthusiasts. Tweets mostly by @VivienSeguySWAPNIL MISHRA @swapnilm3
67 Followers 703 FollowingParth Bhardwaj @_Parth_Bhardwaj
8 Followers 141 FollowingHeriz Shrestha @stha_heriz
0 Followers 51 FollowingPatrizia Vegas @PATRIZIAVegas
492 Followers 101 Following Marines Daughter, Supporting our vets. Sarcastic Scottish-Viking With a Big German mate. Retired at 45 to go sailing. Working on turning my body into Steel.xike41 @xike41
1 Followers 20 FollowingNatural Object AI @NaturalObjectAI
2 Followers 62 Following We bring the greatest of all time to mentor youMake-Print @make_print95727
1 Followers 12 Following The digital platform for better faster additive manufacturing.akshay @RockzAks83507
0 Followers 81 FollowingRonnie ronald @Ronnierona83170
11 Followers 96 FollowingJoe Nasr @JoeNASR
864 Followers 1K Following Tales From The 'Crypto' ...with #VR. 01000011 01110010 01111001 01110000 01110100 01101111 01110010 01101001 01100011 01101000Ghazwan Hassan @gzwn369735
510 Followers 2K Following Welcome to @pangeanis,The marketing team,available 24 hours to serve you... مرحبا بكم فيpangeanis@ فريق التسويق متواجد 24 ساعة لخدمتكمJulio 🇻🇦 @ASanctoThoma
691 Followers 188 Following Catholic. Thomist. Into powerlifting and interested in machine learning.Mehmet Ali Yavuz @Mmehmetaliyavuz
1 Followers 27 FollowingMatt Fleisher @FFMattFleisher
169 Followers 1K FollowingAdrian Mills @adrian_mobysoft
111 Followers 357 Following Director of Marketing at Mobysoft since 2015 - really enjoy working with social landlords and seeing the results they deliver to society - this is a work acctRox Math🐝🔬💡�.. @Roxy_BioMath
297 Followers 2K Following mathematical Model in epidemic, dynamic systems, ODE, statistical models, machine learningfintech-portal @fintech_portal
5K Followers 2K Following Fintech and Crypto // DM for Cooperation // #Fintech #Crypto #Banking #Payment #Bitcoin #Finanzen #Blockchain #Investieren #TechnologieDhanush @Dhanush70417898
14 Followers 510 Followingsameer shahbaz @sameershahbaz
296 Followers 2K Following Life is now. #bitcoin#geopolitics#anti supremacist#futuristRebecca Moore @Rebecca18174873
469 Followers 906 Following Retired Educator-Administrator. Mom-Grandmother-Sister. Love my family, friends,and neighbors. Enjoy music, piano, singing. I enjoy walking,warmth,cool breezes.YasserC @yasser_ch
197 Followers 484 Following Tech professional . AI enthusiast LLM generative AI lamgchainGuillermo @MeMoO_7
135 Followers 1K Following Escribiré frases que me gustan, o compartiré estupideces, incluso compartiré un poco de tecnología e innovación.马豪杰 @JieMa9377
0 Followers 16 FollowingAmit @itsamit108
204 Followers 415 Following Software Developer 👨💻 • Trying to Be Better Every Day 🔆Chuangcius @chuangcius
15 Followers 19 FollowingNaveen K @naveen23k
4 Followers 148 FollowingBojan Tunguz @tunguz
186K Followers 7K Following Machine Learning ex Nvidia. Kaggle Quadruple Grandmaster. Data Scientist. Physicist. Catholic. Husband. Father. Stanford Alum. e/xgb. XGBoost.eth. AMDG.Santiago @svpino
350K Followers 444 Following I tell stories about technology and teach hard-core Machine Learning at https://t.co/iZifcK7n47. YouTube: https://t.co/pROi08OZYJFrançois Chollet @fchollet
468K Followers 768 Following Deep learning @google. Creator of Keras. Author of 'Deep Learning with Python'. Opinions are my own.Yann LeCun @ylecun
708K Followers 716 Following Professor at NYU. Chief AI Scientist at Meta. Researcher in AI, Machine Learning, Robotics, etc. ACM Turing Award Laureate.Mark Tenenholtz @marktenenholtz
114K Followers 544 Following Head of AI @PredeloHQ. XGBoost peddler, transformer purveyor.Andrew Ng @AndrewYNg
1.0M Followers 908 Following Co-Founder of Coursera; Stanford CS adjunct faculty. Former head of Baidu AI Group/Google Brain. #ai #machinelearning, #deeplearning #MOOCselvis @omarsar0
188K Followers 480 Following Building with LLMs @dair_ai • Prev: Meta AI, Galactica LLM, PapersWithCode, Elastic, PhD • Creator of the Prompting Guide (~4M learners)AK @_akhaliq
308K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo follow on Hugging Face: https://t.co/q2Qoey80GxMatt Harrison @__mharrison__
158K Followers 890 Following Python 🐍 + Data Science 🚀 trainer @__metasnake__ 🦜 Speaker ✍ Author 👨🏫 Instructor (@Stanford) 📣 DM for SponsorshipJim Fan @DrJimFan
228K Followers 3K Following @NVIDIA Sr. Research Manager & Lead of Embodied AI (GEAR Lab). Creating foundation models for Humanoid Robots & Gaming. @Stanford Ph.D. @OpenAI's first intern.Akshay 🚀 @akshay_pachaar
133K Followers 419 Following Simplifying LLMs, MLOps, Python & Machine Learning for you! • AI Engineering @LightningAI • Lead DataScientist • BITS Pilani • 3 PatentsJean de Nyandwi @Jeande_d
38K Followers 768 Following Deep Learning, Vision 🤍 Language, Multimodal LLMs • AI Education • CMU Research blog: https://t.co/1BEFLZAqe7 ML Pack: https://t.co/7PkTyDvuriAlfredo Canziani @alfcnz
86K Followers 269 Following Musician, math lover, cook, dancer, 🏳️🌈, and an ass prof of Computer Science at New York UniversityJeremy Howard @jeremyphoward
221K Followers 5K Following 🇦🇺 Co-founder: @AnswerDotAI & @FastDotAI ; Hon Professor: @UQSchoolITEE ; Digital Fellow: @Stanfordabhishek @abhi1thakur
81K Followers 661 Following 🤗 I build AutoTrain @huggingface 👨🏽💻 World's First 4x Grand Master @kaggle 🎥 YouTube 100k+: https://t.co/BHnem8fTu5 ⭐ GitHub StarChris Albon @chrisalbon
86K Followers 2K Following Director of Machine Learning at the Wikimedia Foundation. We host Wikipedia.Pau Labarta Bajo @paulabartabajo_
44K Followers 222 Following The Real-World ML guy | Learn to build real-world ML apps at https://t.co/xWr8Hm8zI5Harrison Kinsley @Sentdex
71K Followers 201 Following Neural networks from Scratch book: https://t.co/MWlYbXicwc YouTube: https://t.co/5osPue5EW9 @skunkworks_aiPyTorch @PyTorch
378K Followers 77 Following Tensors and neural networks in Python with strong hardware acceleration. PyTorch is an open source project at the Linux Foundation. #PyTorchFoundationAndrew Carr (e/🤸) @andrew_n_carr
15K Followers 3K Following science @getcartwheel AI writer @tldrnewsletter advisor @arcade_ai Past - Codegen @OpenAI, Brain @GoogleAI, world ranked Tetris playerBrian Roemmele @BrianRoemmele
266K Followers 31K Following we can only see what we think is possible...Ryan Lowe @ryan_t_lowe
5K Followers 357 Following what is the place from which we are creating? ❤️✨🤠❤️Damien C. Tanner @dctanner
5K Followers 209 Following Compounding curiosity: https://t.co/XcNa3OVLLW, https://t.co/xe1EARqzog, https://t.co/mJcN4MoFZV & https://t.co/4ibiJ4A5DB. Prev founded @pusher, @thoughtbotmain @main_horse
8K Followers 464 Following AGI Believer. Haven't applied @OpenAI. Likes are not always endorsement.Devi Parikh @deviparikh
23K Followers 151 Following Former Sr. Director, GenAI @Meta. Prof @GeorgiaTech. Generative artist https://t.co/z4n9IRQ3s5. Co-founded Caliper. @CarnegieMellon @RowanUniversity alum.Casper Hansen @casper_hansen_
2K Followers 174 Following NLP Scientist | AutoAWQ Creator | Open-Source ContributorNitya Narasimhan, PhD.. @nitya
10K Followers 1K Following Parent ❤ | Polyglot 👩🏽💻 | Innovator💡 | PhD 🎓 Dist Systems - Mobile/Web Dev - AI/ML Illustrations @SketchTheDocs Azure, AI, Advocacy Ma Phaleshu KadachanaSatya Medidi @MSatyasekhar
135 Followers 406 Followingantonio vergari 💥 .. @tetraduzione
4K Followers 1K Following human being | associate prof in #ML #AI @ancAtEd | PI of #APRIL https://t.co/7uTqRZtmEd | #probabilistic #models #tractable #generative #neuro #symbolic |Philipp Rudiger @PhilippJFR
2K Followers 2K Following Staff Software Engineer | Manager @anacondainc | Neuroinformatics PhD | Creator @Panel_Org, @HoloViews, https://t.co/x8Aj2aPMcA | Core developer @Bokeh and @DatashaderThomas Viehmann @ThomasViehmann
5K Followers 0 Following Thunder at https://t.co/p7EZaYIpg1 Author of Deep Learning with PyTorch @[email protected]George Hotz 🌑 @realGeorgeHotz
247K Followers 172 Following President @comma_ai. Founder @__tinygrad__Banghua Zhu @BanghuaZ
2K Followers 775 Following PhD @Berkeley_EECS, statistics, info theory, LLM, RL, Human-AI Interactions.Igor Babuschkin @ibab_ml
44K Followers 671 Following Maybe the real AGI was the friends we made along the way. @xAIRoss @rpoo
25K Followers 1K FollowingAndrew Gao @itsandrewgao
22K Followers 2K Following currently: @nomic_ai @stanford; prev @LangChainAI; Z Fellow 🇺🇸Meta4 @a_meta4
4K Followers 590 Following Crypto is a mind virus! / Live by authentic moral compass. / work on AI infrastructure / fan of first-principle reasoning, even when speculatingxAI @xai
998K Followers 34 FollowingSRK @sudalairajkumar
8K Followers 575 Following 📈 AI Innovations @TigerAnalytics 🦆 4x @kaggle Grandmaster 📊 AI / ML Advisor ⚡ AI Tinkerer in Web3 ⚡ 🖥️ Ex - @h2oai @FreshworksIncOfir Press @OfirPress
9K Followers 3K Following I build tough benchmarks for LMs and then I get the LMs to solve them. Postdoc @Princeton. PhD from @nlpnoah @UW. Ex-visiting researcher @MetaAI & @MosaicML.Soham De @sohamde_
2K Followers 971 Following Research Scientist at DeepMind. Previously PhD at the University of Maryland.Hamid R. Darabi @_hdarabi
207 Followers 105 Following Senior Manager, Data at TrueML | Ex-Amazon | Built models empowering 1% of video ads in U.S.Emily Witko @witkoochocinco
2K Followers 3K Following I'm a people person who supports our team @huggingface. I live in Sacramento by way of Brooklyn. Winter is better here.Marie Pellat @m_pellat
1K Followers 192 Following AI Scientist, @MistralAI - previously Google // Responsible AIML Contests @ml_contests
2K Followers 88 Following Home of competitive machine learning. Discover competitions across multiple platforms and conferences, form teams, find compute, and learn what winners do.Daniel Han @danielhanchen
7K Followers 923 Following Building @UnslothAI. Finetune LLMs 30x faster https://t.co/aRyAAgKOR7. Prev ML at NVIDIA. Hyperlearn used by NASA. I like maths, making code go fastLogan Kilpatrick @OfficialLoganK
92K Followers 2K Following Lead product for @Google AI Studio and working on the Gemini API, helping developers build with AI, my views!Rayed Bin Wahed @rayedbw
136 Followers 211 Following Machine Learning Engineer @therapservices. Reader, Footballer, Meditator. Opinions are my own.Bill Pollock -- nosta.. @billpollock
12K Followers 3K Following Founder, No Starch Press and Hacker Initiative. Views expressed are *entirely* my own. He/him/his Contact our editors at: [email protected]Edgar Bermudez, PhD �.. @viajesubmarino
708 Followers 1K Following Lead AI/ML Scientist at OraqAI. Adjunct Assistant Professor in CompNeuro @ccbn_neuro. Here and now.ifioravanti @ivanfioravanti
5K Followers 1K Following Co-founder and CTO of @CoreViewHQ GenAI/LLM addicted, Apple MLX, Ollama, Microsoft 365, Azure, Kubernetes, Investor in innovationDaniel D. Gutierrez @AMULETAnalytics
4K Followers 468 Following Data scientist. Editor-in-Chief at https://t.co/AIpjXyePCv. Having a blast teaching Data Science at UCLA Extension. Author of 5 "data" books. Academic at heart!Alicia Curth @AliciaCurth
3K Followers 496 Following PhD student Machine Learning in Cambridge, Statistician at ❤️ In search of statistical intuition for modern ML & simple explanations for complex things 👀Yixiao Ge @ge_yixiao
1K Followers 608 Following Principal Researcher @TencentGlobal, Tech Lead of #Multimodal at PCG ARC Lab and AI Lab CVC. Former PhD from MMLab @CUHKofficial.Ivomar Brito Soares @ivomarbrito
163 Followers 1K Following Software engineering and AI @Microsoft. Father of Juliette, the cat. He/him. The views represented here are my own.Saining Xie @sainingxie
14K Followers 1K Following researcher in #deeplearning #computervision | assistant professor at @NYU_Courant @nyuniversity | previous: research scientist @metaai (FAIR) @UCSanDiegoAlexia Jolicoeur-Mart.. @jm_alexia
10K Followers 1K Following AI Researcher at the Samsung SAIT AI Lab 🐱💻catid (e/acc) @MrCatid
3K Followers 629 Following Engineer at Juice Labs. Prior: Anduril, Oculus VR, Game Closure, MSEE@GATechJascha Sohl-Dickstein @jaschasd
19K Followers 621 Following Member of the technical staff @ Anthropic. Most (in)famous for inventing diffusion models. AI + physics + neuroscience + dynamics.SynthLabs @synth_labs
12K Followers 43 Following AI Aligned with Your Vision. We’re doing cutting edge research for transparent, auditable AI alignment.Here is a paper that finally answers a question: Which method is better for RLHF/supervised fine-tuning, DPO or PPO?
While I was eagerly awaiting the technical report/paper accompanying the Llama 3 release yesterday, I stumbled upon another very interesting research paper this week, which finally answers one of my pressing questions: "Is DPO Superior to PPO for LLM Alignment?" RLHF is one of…
62% HumanEval at 8B is just mind blowing. The future of LM-assisted programming is going to be amazing, and it's going to get here much sooner than we expected.
🧵 A nice LLAMA-3 summary by @natolambert. In short, performance is better, but largely due to the sheer increase in the pre-training scale. Training data is a secret. ↩️
@rasbt I'm learning about DPO and reading its paper. The authors also raise a question about whether DPO policies can generalize out of distribution, compared to learning from an explicit reward function. Iterative DPO is a clever tip! Thanks for the insights.
Is DPO superior to PPO for LLM Alignment ? A comprehensive Study.
While I was eagerly awaiting the technical report/paper accompanying the Llama 3 release yesterday, I stumbled upon another very interesting research paper this week, which finally answers one of my pressing questions: "Is DPO Superior to PPO for LLM Alignment?" RLHF is one of…
@rasbt And then there's this ORPO that combines SFT with preference alignment: arxiv.org/abs/2403.07691
It's great to see a direct comparison in the paper, shedding light on PPO vs. DPO. The hybrid approach used in Llama 3 sounds intriguing - blending techniques might be the way forward.
While I was eagerly awaiting the technical report/paper accompanying the Llama 3 release yesterday, I stumbled upon another very interesting research paper this week, which finally answers one of my pressing questions: "Is DPO Superior to PPO for LLM Alignment?" RLHF is one of…
It’s here! Meet Llama 3, our latest generation of models that is setting a new standard for state-of-the art performance and efficiency for openly available LLMs. Key highlights • 8B and 70B parameter openly available pre-trained and fine-tuned models. • Trained on more…
@rasbt @llama_index Amazing! Let me know how it goes🤗
So fast! Thanks @llama_index day 0 support!
The latest model from @MistralAI, the hugely powerful 8x22b, defines the state of the art in open models and is now available! As usual, we have day 0 support: check out @ravithejads's Mistral cookbook showing ➡️ RAG ➡️ Query routing ➡️ Tool use docs.llamaindex.ai/en/latest/exam…
"Weight watchers" should be a podcast (or a newsletter) covering latest model releases.
@alexalbert__ @MKBHD I think my votes on this go to @rasbt for technical content on the data science/ML side of language models and to @vboykis for the ML engineering side.
Watch my @Stanford CS 25 lecture next week, "aligning open language models," it'll be good, v excited
Slides are done! See you tomorrow at 4:30pm PST for the whole story. From alpaca, openassistant, qlora, zephyr, tulu, to today. Slides: docs.google.com/presentation/d… Course link (with zoom): web.stanford.edu/class/cs25/
Watch my @Stanford CS 25 lecture next week, "aligning open language models," it'll be good, v excited