Databricks Mosaic Research @DbrxMosaicAI
We remove the barriers to state-of-the-art generative AI model development and make data + AI available to all. databricks.com/research/mosaic San Francisco, CA Joined December 2020-
Tweets965
-
Followers29K
-
Following115
-
Likes666
Our team is incredibly proud to partner with @allen_ai and thrilled to see them cook! Achieving such a massive improvement in MMLU, while reducing the compute budget, is a fantastic win. And doing it fully open? Everyone wins. Congrats! Can't wait to see what's next 👀
Our team is incredibly proud to partner with @allen_ai and thrilled to see them cook! Achieving such a massive improvement in MMLU, while reducing the compute budget, is a fantastic win. And doing it fully open? Everyone wins. Congrats! Can't wait to see what's next 👀
Part of what makes #DBRX special is the performance we deliver when we serve it. In Daya Khudia's talk at @nvidia #GTC24, he shared some insights on how we do it. Watch the replay here: nvidia.com/en-us/on-deman…
Ready to use a programmatic approach to prompting #LLMs and building #RAG applications? The @stanfordnlp #dspy repo includes support for @databricks Model Serving and Vector Search! Details: databricks.com/blog/dspy-data…
📢 New blog post from our @databricks Mosaic AI researchers @mvpatel2000 and @vitaliychiley announcing the integration of MegaBlocks open source library into #LLM foundry, our open source #training stack! 🙌
📢 New blog post from our @databricks Mosaic AI researchers @mvpatel2000 and @vitaliychiley announcing the integration of MegaBlocks open source library into #LLM foundry, our open source #training stack! 🙌
📢TOMORROW! Join some of our amazing research team (@bandish @abhi_venigalla @davisblalock @ajaysaini725) online for a deep dive on #DBRX - hosted by @databricks DevOps guru @dennylee. Register now!
📢TOMORROW! Join some of our amazing research team (@bandish @abhi_venigalla @davisblalock @ajaysaini725) online for a deep dive on #DBRX - hosted by @databricks DevOps guru @dennylee. Register now!
DBRX is the top open-source model on the latest WildBench Leaderboard on HuggingFace! Thanks to our friends @allen_ai for this benchmark of LLMs with challenging tasks from real users in the wild. #DBRX huggingface.co/spaces/allenai…
Thank you @JuliaANeagu for recognizing the accomplishments of our @databricks Mosaic AI research and engineering teams in building our highly performant open source #DBRX model. 🙌🙌🙌
Thank you @JuliaANeagu for recognizing the accomplishments of our @databricks Mosaic AI research and engineering teams in building our highly performant open source #DBRX model. 🙌🙌🙌
Happy to announce that we're launching the DBRX-coin! Every new variant of the model creates a node on the chain, so train more, make more $$! Let's make crypto and AI actually correlate!
Curious about #DBRX and how it was trained? Join @abhi_venigalla and @ajaysaini725 to learn about the model and the @databricks platform that trained it! Hosted by our own Eric Peter, and the AI Alliance's @TimBonnemann and @ChiefScientist! lu.ma/kiidiyeb
#DBRX sets a new standard for efficient open source LLMs. While it has 132B total parameters, with its fine-grained MoE architecture, DBRX only uses 36B at any given time. Learn more about how we built #DBRX & benchmarked its performance. dbricks.co/3J13hed
Meet #DBRX: a general-purpose LLM that sets a new standard for efficient open source models. Use the DBRX model in your RAG apps or use the DBRX design to build your own custom LLMs and improve the quality of your GenAI applications. dbricks.co/43xaCMj
ICYMI: earlier this month, @NaveenGRao, @matei_zaharia, @jefrankle, and guest speakers from @Accenture and @ADP dropped some knowledge on enterprise #GenAI implementation methods.
ICYMI: earlier this month, @NaveenGRao, @matei_zaharia, @jefrankle, and guest speakers from @Accenture and @ADP dropped some knowledge on enterprise #GenAI implementation methods.
In this @mlopscommunity episode, MosaicML's @davisblalock and @bandish share war stories and lessons learned from pushing the limits of #LLM training and helping dozens of customers get LLMs into production. 🤝 👀 Watch the full episode: home.mlops.community/public/videos/… #mlops #LLMs
Training LLMs is tough work and lots can go wrong. Scale 📈is hard and things break 💔, often. Listen to @davisblalock and me break it down and get some insights💡into why developing custom GenAI on @databricks + @DbrxMosaicAI is the best solution for enterprises!
Training LLMs is tough work and lots can go wrong. Scale 📈is hard and things break 💔, often. Listen to @davisblalock and me break it down and get some insights💡into why developing custom GenAI on @databricks + @DbrxMosaicAI is the best solution for enterprises!
The @databricks Mosaic Research team is committed to building the best possible training stack for #LLMs and #genAI models. @mvpatel2000, @davisblalock, Saaketh Narayan, and Cheng Liang write about our latest benchmark results and training speedup methods here:…
Since becoming part of @databricks last July, the MosaicML team has continued its mission to optimize and improve #GenAI model training. Our rigorous science leads to real-world results. Visit our new research hub to discover what we've working on: databricks.com/research/mosaic
Jim Fan @DrJimFan
229K Followers 3K Following @NVIDIA Sr. Research Manager & Lead of Embodied AI (GEAR Lab). Creating foundation models for Humanoid Robots & Gaming. @Stanford Ph.D. @OpenAI's first intern.Soumith Chintala @soumithchintala
186K Followers 883 Following Cofounded and lead @PyTorch at Meta. Also dabble in robotics at NYU. AI is delicious when it is accessible and open-source.Horace He @cHHillee
23K Followers 449 Following Working at the intersection of ML and Systems @ PyTorch "My learning style is Horace twitter threads" - @typedfemaleEric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0pPercy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | PianistJeremy Howard @jeremyphoward
222K Followers 5K Following 🇦🇺 Co-founder: @AnswerDotAI & @FastDotAI ; Hon Professor: @UQSchoolITEE ; Digital Fellow: @StanfordJonathan Frankle @jefrankle
16K Followers 685 Following Chief Scientist, Neural Networks @Databricks via MosaicML. PhD @MIT_CSAIL. BS/MS @PrincetonCS. DC area native. Making AI efficient for everyone at @DbrxMosaicAIRosanne Liu @savvyRL
33K Followers 968 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDRDelip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈Naveen Rao @NaveenGRao
28K Followers 789 Following VP GenAI @Databricks. Former CEO/cofounder MosaicML & Nervana/IntelAI. Neuro + CS. I like to build stuff that will eventually learn how to build other stuff.rohan anil @_arohan_
12K Followers 2K Following Principal Engineer, @GoogleDeepMind Gemini. prev PaLM-2. Tinkering with optimization and distributed systems. opinions are my own.Tim Dettmers @Tim_Dettmers
29K Followers 821 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.Davis Blalock @davisblalock
12K Followers 165 Following Research scientist + first hire @MosaicML. @MIT PhD. I write + retweet technical machine learning content. If you write a thread about your paper, tag me for RTDan Roy @roydanroy
45K Followers 2K Following ML / AI researcher, emphasis on theory. Research Director and Canada CIFAR AI Chair, @VectorInst Professor, @UofT (Statistics/CS)Ethan Caballero is bu.. @ethanCaballero
8K Followers 2K Following ML PhD student @Mila_Quebec ; previously @GoogleDeepMindKyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).Sara Hooker @sarahookr
39K Followers 7K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.Omar Sanseviero @osanseviero
32K Followers 2K Following Chief Llama Officer @huggingface 🦙 Founder @AI_Learners. Xoogler (SWE @Google Assistant, 20% PM TF Graphics). 100% Hacker Llama🇵🇪🇲🇽Cameron R. Wolfe, Ph... @cwolferesearch
21K Followers 623 Following Director of AI @RebuyEngine • Writer @ Deep (Learning) Focus • PhD @optimalab1 • I make AI understandableShane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)RAJ DOSHI @rd9925
0 Followers 35 Following_sag70 @sagnik70
58 Followers 693 Following "War is wrong unless it is for truth and justice"-Budhha Proud Buddhist || AI SUPREMACIST. Sometimes I teach Excel.. Pronouns: Man/goMatt A. Porter @PhaedrusFlow
468 Followers 138 Following Follower of Jesus Christ | Service Disabled Vet DeepTech Founder/CEO: Qompass Cost-Conscious GenAI ServicesKugs @Kugs1776
427 Followers 2K Following notes to self - cynic sympathizer - at the least failing while daring greatly - 11B1PGeorge Will @georgemwill
1 Followers 14 Followingklaus @_kalemio
163 Followers 3K FollowingParas Rathod @parasrathod07
1 Followers 48 Followingnicholas @nicholaslee_7
6 Followers 136 Followingcallmedaddy @callmed64488563
5 Followers 50 Followingjeremyfelix @felixHcat
52 Followers 263 Following He's like some sort of strange robotic techno-kitten.Waleomoyeni @Waleomoyeni1
3 Followers 41 FollowingShahir @TslShahir
52 Followers 318 Following 🚀 Founder & Language Acquisition Enthusiast 🌟 Passionate about the transformative power of AI in education.Neeraj Agrawal @neeraj_abhi10
29 Followers 153 Following Engineer at Amazon Audible | Prev- Alexa AIBảo Nguyễn Long @Debugger_w_
7 Followers 32 FollowingDhruv Rajput @dhruuuvv__
2 Followers 62 FollowingSSW @SSWyougattamove
7 Followers 465 FollowingĐức Tài Nguyễn @marshmello250
2 Followers 19 FollowingRebekah @Rebekah_model
1 Followers 22 FollowingAl Buterol @alanbuterol
17 Followers 82 FollowingFSM @fsm_top
7 Followers 119 FollowingPosthumanisti @posthumanisti
9 Followers 106 Following Sivilisaatiomme tulevaisuus ei tarvitse biologisia ihmisiä. #tiede #tulevaisuus #posthumanismishivam @shivammk27
36 Followers 470 Following ॐ | ai @paytm. toying with models & reading research. love architecture (both real world and neural networks). very ambitious gpu poor humanGeorge Smith @georgeksmith
298 Followers 465 Following I really don't want to be addicted to Twitter, so I post randomly and rarely.Craig Freedman @rockstarketer1
6 Followers 95 FollowingJockbod @Jockbod
133 Followers 354 Followingavnish narayan @narayan_avnish
99 Followers 298 Following Working on Anyscale Endpoints @AnyscalecomputeMirH_x @MirH_x
44 Followers 386 FollowingCiosh @Ciosh111983
0 Followers 160 FollowingMike @mmaurialj
3 Followers 295 Following Academic background in healthcare admin & management; CS & infosec enthusiast.Sheikh Abdullahi BNSH.. @PrinceLam002
30 Followers 123 Following A Motivational Speaker and An Islamic Educationistworkrelated @workrelate60845
0 Followers 16 FollowingAditya Kumar Singh @Adityak204
7 Followers 236 Following Data Scientist@ABInBev | Deep Learning enthusiast | NLPAndrew Zhang @a9zhang
18 Followers 230 Followingnikhilsai katta @nikhilsaikatta
1 Followers 102 FollowingAnders @Hum61633
2 Followers 39 FollowingLennart Hennig @lennarthennig
233 Followers 587 Following Dedicated to creating spaces for individuals and organizations wanting to participate in the collective development of humanity. https://t.co/T2oLZe7JRPAditya Dixit @adityakiwi
268 Followers 3K Following 🥝 Kiwi in Tech | 🇺🇸 NYC ✈️ 🇮🇳 DEL ✈️ 🇳🇿 AKL🗽@nyuniversity🎓| MS CS 2023 🌉 @usfca🎓 | BS👨🏾💻CS +📈 Econ 2021Zhaoyang Chu @zhaoyang_c68411
8 Followers 365 Following CS Master@HUST. Interested in SE+ML, specifically focusing on building trustworthy and reliable AI-based software systems. Seeking PhD starting in 2025 Fall.Jonathas Enders @jonathas_enders
1 Followers 933 Following Co-Chairman @ Ehemaligenstiftung Hansenberg | https://t.co/3nfZ4NjTzM. ETH Biology | Studying https://t.co/efIPDazNXE. Biomedical Engineering @ ETH ZurichRondo corty @CortyRondo73261
91 Followers 115 FollowingJim Fan @DrJimFan
229K Followers 3K Following @NVIDIA Sr. Research Manager & Lead of Embodied AI (GEAR Lab). Creating foundation models for Humanoid Robots & Gaming. @Stanford Ph.D. @OpenAI's first intern.Soumith Chintala @soumithchintala
186K Followers 883 Following Cofounded and lead @PyTorch at Meta. Also dabble in robotics at NYU. AI is delicious when it is accessible and open-source.Jeremy Howard @jeremyphoward
222K Followers 5K Following 🇦🇺 Co-founder: @AnswerDotAI & @FastDotAI ; Hon Professor: @UQSchoolITEE ; Digital Fellow: @StanfordJonathan Frankle @jefrankle
16K Followers 685 Following Chief Scientist, Neural Networks @Databricks via MosaicML. PhD @MIT_CSAIL. BS/MS @PrincetonCS. DC area native. Making AI efficient for everyone at @DbrxMosaicAIRosanne Liu @savvyRL
33K Followers 968 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDRHugging Face @huggingface
344K Followers 189 Following The AI community building the future. https://t.co/VkRPD0VKaZ #BlackLivesMatter #stopasianhateNaveen Rao @NaveenGRao
28K Followers 789 Following VP GenAI @Databricks. Former CEO/cofounder MosaicML & Nervana/IntelAI. Neuro + CS. I like to build stuff that will eventually learn how to build other stuff.Davis Blalock @davisblalock
12K Followers 165 Following Research scientist + first hire @MosaicML. @MIT PhD. I write + retweet technical machine learning content. If you write a thread about your paper, tag me for RTCameron R. Wolfe, Ph... @cwolferesearch
21K Followers 623 Following Director of AI @RebuyEngine • Writer @ Deep (Learning) Focus • PhD @optimalab1 • I make AI understandableOriol Vinyals @OriolVinyalsML
167K Followers 82 Following VP of Research & Deep Learning Lead, Google DeepMind. Gemini co-lead. Past: AlphaStar, AlphaFold, AlphaCode, WaveNet, seq2seq, distillation, TF.Stella Biderman @BlancheMinerva
15K Followers 748 Following Open source LLMs and interpretability research at @BoozAllen and @AiEleuther. My employers disown my tweets. She/herMosiac Research @MosaicResearch
4 Followers 0 Followingshreya rajpal @ShreyaR
6K Followers 774 Following ML, systems, and everything in between. Building @guardrails_ai. Previously founding eng @predibase, @Apple SPG, @driveai_, @IllinoisCS, @iitdelhi.swyx @swyx
91K Followers 3K Following Anti-ego ideas for anti-ergodic life. Founder, @smolmodels ▹ Listen: @latentspacepod ▹ Read: @coding_career ▹ Join: @aiDotengineerZack Ankner @ZackAnkner
486 Followers 306 Following Junior @MIT. President of AI@MIT. Research Scientist Intern @MosaicML. A(CL)verage Embargo enjoyer.Linden Li @lindensli
1K Followers 535 Following CS @Stanford, @StanfordSVL. Research/Eng @MosaicML, previously @NVIDIA.Allen Institute for A.. @allen_ai
54K Followers 361 Following AI for the Common Good. › Join us: https://t.co/DqTs1G4bGO › Get our newsletter: https://t.co/tvb1VpySfLDaya @dskhudia
177 Followers 114 FollowingKristen Richardson @butwhyevernot
2K Followers 2K Following Book: The Season: A Social History of the Debutante (W.W. Norton: 2019). Inquiries via the Wylie Agency.Dan Biderman @dan_biderman
598 Followers 881 Following Final-year PhD student at @cu_neurotheory building ML systems for neuroscience. Also NLP research @DbrxMosaicAIDatabricks @databricks
70K Followers 1K Following Databricks is the data and AI company, helping data teams solve the world’s toughest problems.Hagay Lupesko @hagay_lupesko
244 Followers 88 Following VP of Software Engineering at MosaicML, making ML efficient and accessible for the masses. DM me to learn more!Matei Zaharia @matei_zaharia
39K Followers 1K Following CTO at @Databricks and CS prof at @UCBerkeley. Working on data+AI, including @ApacheSpark, @DeltaLakeOSS, @MLflow, https://t.co/94gROE5Xa0. https://t.co/nmRYAKG0LZShahin Farshchi @Farshchi
9K Followers 894 Following @lux_capital, invested @zoox, @planet, @relativityspace, @vardaspace, @epsilon3inc, @nervanasys, @mosaicml, @CovariantAI, @goformic, $AEVA, Dad/Bear/Bruin/PilotReplit ⠕ @Replit
122K Followers 1K Following Idea to software, fast. Build and deploy software collaboratively with the power of AI without spending a second on setup. Need help? @ReplitSupportGradient Flow @GradFlowTech
29 Followers 7 Following Official Account of Gradient Flow (https://t.co/xh4UsbDEgS) and The Data Exchange podcast (https://t.co/23gJEo92zo)Perplexity @perplexity_ai
132K Followers 29 Following Our mission is to serve the world’s curiosity. https://t.co/BBZ1kG0TVGStability AI @StabilityAI
190K Followers 31 Following We are building the foundation to activate humanity's potential.Sally Ward-Foxton @sallywf
2K Followers 1K Following Senior Reporter at @eetimes, reporting mainly on AI accelerators.Chase Lochmiller @ChaseLochmiller
3K Followers 2K Following CEO and Co-Founder of @CrusoeEnergy Former @polychain, @jumptrading, @Stanford, @MIT"nicole" @ninklefitz
1K Followers 517 Following master of decorum @alpacaml. prev: @MicrosoftResearch, @MosaicML, @Mila_QuebecBarry Dauber @barrydauber
697 Followers 468 Following VP of Mosaic AI GTM @DbrxMosaicAI / @Databricks, DC Native, Texas LonghornJesse Dodge @JesseDodge
3K Followers 2K Following Senior Research Scientist at AI2 @ai2_allennlp. Responsibly open work on the science of AI and AI for science. Environmental impact of AI. he/him 🏳️🌈Vitaliy Chiley @vitaliychiley
2K Followers 607 Following Head of NLP Pretraining @Databricks / @MosaicML | Former @CerebrasSystems | What do we want? FLOPS! When do we want it? TOKENS!OpenAI @OpenAI
3.5M Followers 0 Following OpenAI’s mission is to ensure that artificial general intelligence benefits all of humanity. We’re hiring: https://t.co/dJGr6LgzPAOracle Developers @OracleDevs
118K Followers 702 Following Oracle Developers is a community for developers by developersOracle @Oracle
819K Followers 825 Following Leading the cloud. We help people see data in new ways, discover insights, unlock endless possibilities.Manish Kapur @kapmani
372 Followers 236 Following Tech guy | Pensive thinker | Sports fan | Views expressed are my own.DavidLinthicum @DavidLinthicum
38K Followers 4K Following Cloud Computing visionary. CTO, CEO, blogger, speaker, best selling author. RT≠endorsements, all opinions are my own.Song Han @songhan_mit
6K Followers 144 Following Assoc. Prof. @MIT, Distinguished Scientist @NVIDIA, cofounder of DeePhi (now part of AMD) and OmniML (now part of NVIDIA). PhD @Stanford. Efficient AI computingNina da Hora - tweets.. @ninadhora
70K Followers 12K Following Master’s student in Ethics in AI @unicamp. 2024 GlobalFellow @ fordfoundation. Director @ institutodahora. Decolonize science 🌈🏳️⚧️Sahil Khose @SahilKhose
570 Followers 1K Following Incoming PhD @ Gatech @ICatGT | MSCS GaTech '24 🇺🇸| BTech MIT Manipal '22 🇮🇳Sharon Zhou @realSharonZhou
23K Followers 1 Following Building the future of LLMs | Cofounder & CEO, @LaminiAI | Prev: CS Faculty & PhD @Stanford. Product @Google. @Harvard | @MIT 35 under 35. Angel investor.Climate Change AI @ClimateChangeAI
12K Followers 362 Following Tackling climate change with machine learning. We facilitate cooperation and provide resources for those working in this area. RT is not endorsement.David Rolnick @david_rolnick
4K Followers 373 Following Assistant Professor in Computer Science, @mcgillu / @Mila_Quebec. Co-Founder and Chair, @ClimateChangeAI. MIT @techreview Innovator Under 35. he/him/hisPriya L. Donti @priyald17
5K Followers 819 Following Asst Prof @MITEECS & LIDS. Co-founder & Chair @ClimateChangeAI. MIT @techreview 35 Innovators Under 35. she/theyLynn Kaack @LynnHKaack
2K Followers 931 Following Assistant Prof @thehertieschool working on climate & energy policy and ML, co-founder and chair @ClimateChangeAI, previously @eth_epg & @CMU_EPPICML Conference @icmlconf
70K Followers 17 Following Int'l Conf on ML • July 21-27, 2024 (Vienna, Austria) • #icml2024 • Contact: https://t.co/6saHKWV01y • https://t.co/sFwmcQNWkEJohn Maeda @johnmaeda
390K Followers 347 Following VP AI & Design at Microsoft / How To Speak Machine (2019) https://t.co/eb6gj2wf1bLlama3-70B has settled at #5. With 405B still to come next... I remember when GPT-4 released in March 2023, it looked like it was nearly-impossible to get to the same performance. Since then, I've seen @Ahmad_Al_Dahle and the rest of the GenAI org in a chaotic rise to focus,…
Exciting update -- Llama-3 full result is out, now reaching top-5 on the Arena leaderboard🔥 We've got stable enough CIs with over 12K votes. No question now Llama-3 70B is the new king of open model. Its powerful 8B variant has also surpassed many larger-size models. What an…
There is no question that AI will eventually reach and surpass human intelligence in all domains. But it won't happen next year. And it won't happen with the kind of Auto-Regressive LLMs currently in fashion (although they may constitute a component of it).…
just realized how powerfully Google has owned the primary colors as a brand. I mindlessly scrolled past this chart and assumed it was a Google-related tweet even though Google never appears here. nice eval results for @MistralAI and @DbrxMosaicAI btw!
@jefrankle Super cool. Thanks so much for the commitment to open-source from both a model/weights and code perspective!
AI21 and Databricks show open source can radically slim down AI Two new large language models, Jamba and DBRX, dramatically reduce the compute and memory needed for predictions, while meeting or beating the performance of top models such as GPT-3.5 and Llama 2.…
I hit an error in my notebook and the @databricks Assistant politely told me what the cause was, how to work-a-around it, but it also told me that if I used another function, I would not have to use the work-a-round. Wow. (that compensated for all the times it was plain wrong!)
Open models FTW:
@DbrxMosaicAI DBRX outperforms @OpenAI GPT-4 on realistic, domain-specific benchmark datasets. For example, on a customer support summarization use-case👇👇👇 Still neck and neck but it shows that open models can be the no-brainer choice for actual enterprise applications.
#DBRX democratizes training + tuning of custom, high-performing LLMs so enterprises don't need to rely on a handful of closed models. Now, every organization can efficiently build production-quality GenAI applications while having control over their data. dbricks.co/3x8pxjK
Things are changing weekly but this seems to be the best open LLM for now.
It’s finally here 🎉🥳 In case you missed us, MosaicML/ Databricks is back at it, with a new best in class open weight LLM named DBRX. An MoE with 132B total parameters and 32B active 32k context length and trained for 12T tokens 🤯
We built a new model! 🧱 It's called DBRX 🧱 * mixture of experts * 16 choose 4 experts * 36B active, 132B total * trained on 12T tokens * built e2e in 2 months * using 3072xH100 * served up to 150 tok/s on @databricks * open weights :)
Congrats @abhi_venigalla and Mosaic Databricks team. You’re moving the AI industry forward, brick by brick 🧱 😘
We built a new model! 🧱 It's called DBRX 🧱 * mixture of experts * 16 choose 4 experts * 36B active, 132B total * trained on 12T tokens * built e2e in 2 months * using 3072xH100 * served up to 150 tok/s on @databricks * open weights :)
@code_star @jefrankle I can feel him vibrating through slack
What does it look like to knock a million dollars off the cost of training huge models? For us, it looked like this:
🚨New🌟blog✍️ on ⏩ maximizing🌙 FLOPS 🚀 Training large models requires maximizing flops/GPU, especially at scale. Excited to share a few of the cool tricks in thread👀. 1/N
@DbrxMosaicAI @databricks Love the new logo!
In this @mlopscommunity episode, MosaicML's @davisblalock and @bandish share war stories and lessons learned from pushing the limits of #LLM training and helping dozens of customers get LLMs into production. 🤝 👀 Watch the full episode: home.mlops.community/public/videos/… #mlops #LLMs
Training LLMs is tough work and lots can go wrong. Scale 📈is hard and things break 💔, often. Listen to @davisblalock and me break it down and get some insights💡into why developing custom GenAI on @databricks + @DbrxMosaicAI is the best solution for enterprises!
In this @mlopscommunity episode, MosaicML's @davisblalock and @bandish share war stories and lessons learned from pushing the limits of #LLM training and helping dozens of customers get LLMs into production. 🤝 👀 Watch the full episode: home.mlops.community/public/videos/… #mlops #LLMs
Wonderful to see @abhi_venigalla light up my feeds today. This man knows what he's talking about :)
Advancing AI: @databricks NLP Architect, Abhinav Venigalla, discusses the hardware and software advantages from AMD.
@databricks is the best platform to customize model behavior, and that includes grounding to feedback. We are constantly looking for ways to make Gen AI more reliable to make it usable in an enterprise context. 👇read below about some of the fundamental innovations in this area!
New blog post! @zeqiuwu1, @huyushi98, and @rajammanabrolu share a recent highlight from their work in #LLM finetuning research: Fine-Grained Reinforcement Learning from Human Feedback (RLHF) databricks.com/blog/fine-grai…
A recap of how to get better rewards for RLHF and a view into what I've been working on Scaling to production levels at Mosaic. We have so much more exciting work to show y'all vv soon
New blog post! @zeqiuwu1, @huyushi98, and @rajammanabrolu share a recent highlight from their work in #LLM finetuning research: Fine-Grained Reinforcement Learning from Human Feedback (RLHF) databricks.com/blog/fine-grai…