BigScience Research Workshop @BigscienceW
A research workshop on large language model gathering 1000+ researchers around the world Follow the training of the 176B multilingual model live @BigScienceLLM bigscience.huggingface.co 🌐 Joined April 2021-
Tweets356
-
Followers15K
-
Following1
-
Likes810
The top 15 most-liked organizations on @huggingface 1. @StabilityAI 20k likes 2. @AIatMeta 20k 3. @runwayml 11k 4. CompVis 10k 5. @thukeg 7k 6. @BigscienceW 7k 7. @TIIuae 7k 8. @Microsoft 6.5k 9. @GoogleAI 6k 10. @OpenAI 4k 11. @BigCodeProject 4k 12. @MosaicML 4k 13. @UKPLab 3k…
I respect the caution, but also need to stress that efforts that pursue transparency as an operational value in service of actual inclusion and accountability do exist - see for example the writing on this very topic by @BigscienceW, including its ethical charter. 1/3
I respect the caution, but also need to stress that efforts that pursue transparency as an operational value in service of actual inclusion and accountability do exist - see for example the writing on this very topic by @BigscienceW, including its ethical charter. 1/3
Never thought I'd see the day I'd have a publication in JMLR 🥹 So happy that the BLOOM carbon footprint paper has finally found a home at such an incredible venue! Thank you @shakir_za for being such a great editor, it warms my heart to see your name on this paper 💚
If you wanted to see the fun panel/Q&A we did with Londoners on AI, you can check out the recording here! My preso at the start is also on Open Science, representing @huggingface & @BigscienceW.
If you wanted to see the fun panel/Q&A we did with Londoners on AI, you can check out the recording here! My preso at the start is also on Open Science, representing @huggingface & @BigscienceW.
Introducing: 💫StarCoder StarCoder is a 15B LLM for code with 8k context and trained only on permissive data in 80+ programming languages. It can be prompted to reach 40% pass@1 on HumanEval and act as a Tech Assistant. Try it here: shorturl.at/cYZ06r Release thread🧵
Join us tomorrow, Wednesday 22nd (6:30 PM - 8:00PM CET) at the @mozillafestival Science Fair to learn more about our work in the open and responsible development of large language models (LLMs) for code. schedule.mozillafestival.org/session/TJRU3L… #Mozfest
As you already know, I am very proud of the collective work that enabled the development of @BigscienceW's ethical charter. Today I am even more proud to announce that it's part of @OECDinnovation's catalog to promote Trustworthy AI: such a milestone! oecd.ai/en/catalogue/t…
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset Documents the data creation and curation efforts of ROOTS corpus, a 1.6TB dataset used to train BLOOM Releases a large initial subset of the corpus data: huggingface.co/bigscience-data abs: arxiv.org/abs/2303.03915
Worried about benchmark data contamination? Studying LLM memorization or attribution? @BigscienceW BLOOM 🌸 now has exact & fuzzy search over full training data! with @olapiktus🏆 @christopher Paulo Villegas @HugoLaurencon @ggdupont @SashaMTL @YJernite arxiv.org/abs/2302.14035 /1
(Repost for corrected Arxiv) 🧐What’s the best way to quickly adapt large multilingual language models to new languages? We present our new paper from @BigscienceW 🌸: BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting. 📜 arxiv.org/abs/2212.09535 [1/9]
Petals, a system for easy decentralized inference and adaptation of 100B+ LLMs, is now online! 🌸Generate text with BLOOM-176B using Colab or a desktop GPU 🔌Fine-tune large models for your tasks 👥Help others by contributing your GPUs or host a new swarm colab.research.google.com/drive/1Ervk6HP…
The Bloom paper is out. Looks like it's doing worse than current GPT3 API in zero-shot generation tasks in English but better than other open-source LLMs & better than all in zs multi-lingual (which was the main goal). Proud of the work from the community! arxiv.org/abs/2211.05100
Big day today with two papers out! BLOOM carbon footprint at arxiv.org/abs/2211.02001, new models BLOOMZ and mt0 at huggingface.co/bigscience/blo…
The @BigscienceW carbon footprint paper is live!! 🎉 Check it out to see how we calculated BLOOM's carbon footprint, covering all steps from the manufacturing of equipment 💻 to deployment! 🚀 arxiv.org/abs/2211.02001
Crosslingual Generalization through Multitask Finetuning 🌸 Demo: huggingface.co/bigscience/blo… 📜 arxiv.org/abs/2211.01786 💻github.com/bigscience-wor… We present BLOOMZ & mT0, a family of models w/ up to 176B params that follow human instructions in >100 languages zero-shot. 1/7
print("Hello world! 🎉") Excited to announce the BigCode project led by @ServiceNowRSRCH and @huggingface! In the spirit of BigScience we aim to develop large language models for code in an open and responsible way. Join here: bigcode-project.org/docs/about/joi… A thread with our goals🧵
A few days left until the first BigScience follow-up project 🚀 Any guesses ?
.@BigscienceW has released BLOOM, one of the largest open source large language models to work across multiple languages fosslife.org/bloom-open-sou… #OpenSource #BLOOM #BigScience #LLM #AI #language #HuggingFace
Great #AIEthics initiative! Big Tech builds #AI with bad data Scientists so sought better data to reduce social, cultural, race, gender & sexual biases too often inherent in Big Tech's #MachineLearning washingtonpost.com/technology/202… @nitashatiku @washingtonpost w/ @YJernite👏 Cc @MiaD
To help the AI community reuse the @BigscienceW BLOOM RAIL License for distributing their own models, we adapted the terms to make the license applicable to any associated AI Model. Check out the BigScience OpenRAIL-M License here: licenses.ai/blog/2022/8/26… @Carlos_MFerr
Hugging Face @huggingface
344K Followers 189 Following The AI community building the future. https://t.co/VkRPD0VKaZ #BlackLivesMatter #stopasianhate
clem 🤗 @ClementDelangue
91K Followers 5K Following Co-founder & CEO @HuggingFace 🤗, the open and collaborative platform for AI builders
Delip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈
merve @mervenoyann
56K Followers 4K Following open-sourceress at @huggingface 🧙🏻♀️ proud mediterrenean 🍋 I do TL;DR on ML papers
Lucas Beyer (bl16) @giffmana
56K Followers 446 Following Researcher (Google DeepMind/Brain in Zürich, ex-RWTH Aachen), Gamer, Hacker, Belgian. Mostly gave up trying mastodon as [email protected]
Jeremy Howard @jeremyphoward
222K Followers 5K Following 🇦🇺 Co-founder: @AnswerDotAI & @FastDotAI ; Hon Professor: @UQSchoolITEE ; Digital Fellow: @Stanford
Omar Sanseviero @osanseviero
31K Followers 2K Following Chief Llama Officer @huggingface 🦙 Founder @AI_Learners. Xoogler (SWE @Google Assistant, 20% PM TF Graphics). 100% Hacker Llama🇵🇪🇲🇽
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
abhishek @abhi1thakur
81K Followers 662 Following 🤗 I build AutoTrain @huggingface 👨🏽💻 World's First 4x Grand Master @kaggle 🎥 YouTube 100k+: https://t.co/BHnem8fTu5 ⭐ GitHub Star
MMitchell @mmitchell_ai
80K Followers 1K Following Interdisciplinary researcher focused on shaping AI towards long-term positive goals. ML & Ethics. Same content in the Sky, Threads, & the Prehistoric Elephant
Thomas Wolf @Thom_Wolf
68K Followers 4K Following Co-founder and CSO @HuggingFace - open-source and open-science
Richard Socher @RichardSocher
101K Followers 971 Following CEO @youSearchEngine Investing at @aixventuresHQ Before: Stanford Adj Prof in AI/NLP, Chief Scientist at Salesforce, MetaMind
Sasha Luccioni, PhD �.. @SashaMTL
19K Followers 4K Following AI & Climate @HuggingFace, Board Member of @WiMLworkshop and @ClimateChangeAI. @techreview 35 Innovators under 35, @TEDTalks speaker. She/her/Dr/ 🦋
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Stella Biderman @BlancheMinerva
15K Followers 748 Following Open source LLMs and interpretability research at @BoozAllen and @AiEleuther. My employers disown my tweets. She/her
Eric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0p
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Sam Bowman @sleepinyourhat
35K Followers 3K Following AI alignment + LLMs at NYU & Anthropic. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Akari Asai @AkariAsai
11K Followers 650 Following Ph.D. student @uwcse & @uwnlp. NLP. IBM Ph.D. fellow (2022-2023). Meta student researcher (2023-) . ☕️ 🐕 🏃♀️🧗♀️🍳
chuldog @chuulldog
147 Followers 976 Following Engineer ⚙️ & Big Data EOI | Noob smc trader 📉 10-2022 | 08-2021⏳ | More work less talk 🥷🥷
Electronicsseeker @libertarian108
9 Followers 1K Following
Arash Kia @arashnkia
169 Followers 197 Following
momodz16 @MohArobe
4 Followers 133 Following
ytinyui @ytinyui
34 Followers 78 Following
Samantha Harris @Samanth4344566
0 Followers 37 Following
Ramneet Singh @Ramneet_Singhh
481 Followers 3K Following Visiting Researcher @gatech_scs | Senior Undergraduate in CS at IIT Delhi | Research in PL+ML | Learning Category Theory | Sports + Music + Food (in that order)
Jeff Lee @JeffLee88939390
17 Followers 119 Following
Erich Steinbüchel @steinbuchel
129 Followers 508 Following #InternetofThings, #IoT, #blockchain, #apis, #tennis
Bedeighn @bedeighn75458
28 Followers 261 Following In the dull and boring world, there is also occasional luck. No cross, no crown.
LivingstoneWu @livingstone_wu
2 Followers 28 Following
Abdulrahman Tabaza @embed_dim
4 Followers 799 Following enjoyer of various vector spaces, encoders and modalities
Itxaso Baskero Dorrea.. @IDorreak
12 Followers 365 Following
Xya_cerX_!3 @3Cerx
4 Followers 215 Following
ADNANE ABOUTALIB @AdnaneAboutalib
11 Followers 191 Following ML Engineer crafting cutting-edge AI systems poised to make a lasting impact on the world.
Kugs @Kugs1776
427 Followers 2K Following notes to self - cynic sympathizer - at the least failing while daring greatly - 11B1P
Marlon Barrios Solano @MarlonBarriosS2
148 Followers 487 Following Art+Technology+Embodiment+Cognition | WEBDEV | Generative AI | Synthetic Creativity and Cognition
Home Fire Games @HomeFireGames
193 Followers 115 Following I'm an indie game dev. Working on game dev tools and hopefully making things people enjoy
Blosso sales @BlossoSale15767
53 Followers 671 Following
rocata20 @data_nowhere
3 Followers 63 Following
Ankush Sharma @darxtrix
393 Followers 1K Following Engineering @Google | IIT BHU 16 | Loves talking cricket, technology and food.
Chamnan Muon (មួ�.. @chamnanmuon
1K Followers 3K Following 🎯 ICT & Digital Marketing Consultant 🏆 #LocalGuides Summit Alumni @GoogleMaps ✈️ Traveler 🇰🇭🇺🇲🇰🇷🇸🇬🇹🇭🇻🇳🇱🇦 ...🌍 📖 Lifelong Learner 📗 📊Lover📁
paulorcf @paulorcf
425 Followers 3K Following
Siba @SibaSiddiquee
48 Followers 279 Following Master's in Engineering Management + HCID 📚 Storyteller, language aficionado, and traveler 🌏
Muhammad Zubair Bhatt.. @M_Z_Bhatti
1 Followers 18 Following
Pankaj Niroula @npankaj365
265 Followers 2K Following Always Curious. Excited about Systems, Security and Sustainability. CS PhD Student @williamandmary
吴帅民 @wshuimn1
7 Followers 295 Following
paul @paul86469140
3 Followers 302 Following
Hadoop @Hadoop02277010
542 Followers 4K Following
YM Ro @ro_dwight
12 Followers 31 Following Professor at KAIST, directing AI research in IVL Lab and IVL lab.
Sarasuadi @Sarasuadi
177 Followers 1K Following Filósofa techie. Voyeur de tuits. Aquí solo favs y rt porque socializar me da amsiedá.
Clément Mandron @clement_mandron
211 Followers 695 Following opendata chez @datactivi_st auparavant en stratégies territoriales et urbaines à @ScPoEUrbaine
Tenandead @tenandead66464
15 Followers 271 Following In the dull and boring world, there is also occasional luck. No cross, no crown.
Seeppeighs @seeppeighs21063
29 Followers 291 Following In the dull and boring world, there is also occasional luck. No cross, no crown.
🌞 VALTER SFORZA E/.. @va1tersf0rza
108 Followers 1K Following #quantitative #trader #research $spy #spy #ai #machinelearning #data #statistics #arbitrage #nyc #manhattan 0dte sentiment analysis anomaly detection
Guan Xinyan @guanxy0406
17 Followers 73 Following
BigScience Large Mode.. @BigScienceLLM
9K Followers 1 Following Follow the training of "BLOOM 🌸", the @BigScienceW multilingual 176B parameter open-science open-access language model, a research tool for the AI community.
@ClementDelangue @clefourrier @AiEleuther @Bloomberg @BigscienceW and BigScience Data! - hf.co/bigscience - hf.co/bigscience-data
MODEL RELEASE: We are proud to release BLOOMChat-v2, a 32K sequence length, 176B multilingual language model trained on top of @BigscienceW ‘s BLOOM model. BLOOMChat-v2 is the successor to last year’s release of BLOOMChat-v1, and the largest open-source model that can be run…
Today, we’re launching Aya, a new open-source, massively multilingual LLM & dataset to help support under-represented languages. Aya outperforms existing open-source models and covers 101 different languages – more than double covered by previous models. cohere.com/research/aya
1. Data collection. Hoovering up data indiscriminately isn't a solution. Issues of licensing and content need to be considered from the get-go. Check out the ROOTS corpus work from @BigscienceW as one example of thoughtful data curation: arxiv.org/abs/2303.03915
The top 15 most-liked organizations on @huggingface 1. @StabilityAI 20k likes 2. @AIatMeta 20k 3. @runwayml 11k 4. CompVis 10k 5. @thukeg 7k 6. @BigscienceW 7k 7. @TIIuae 7k 8. @Microsoft 6.5k 9. @GoogleAI 6k 10. @OpenAI 4k 11. @BigCodeProject 4k 12. @MosaicML 4k 13. @UKPLab 3k…
I respect the caution, but also need to stress that efforts that pursue transparency as an operational value in service of actual inclusion and accountability do exist - see for example the writing on this very topic by @BigscienceW, including its ethical charter. 1/3
I did not sign this statement, tho I agree “open” AI is not the enemy of “safe” AI I can't endorse its premise that “openness” alone will “mitigate current+future harms from AI,” nor that it’s an antidote to concentrated power in the AI industry 1/ open.mozilla.org/letter/
This is your daily reminder that only three orgs have ever trained a LLM and released the model and full data: @AiEleuther @BigscienceW (non-OS license) @togethercompute. Small orgs like these make science possible in the face of industry power.
It's very nice that we have models with open weights and open training code. These are better than secret or proprietary weights by far. But we need to differentiate between open weights/training code and a truly *open source* model which absolutely should include training data
🌼 Revolutionizing AI with #Decentralization! 🌼 Discover Petals, the open-source initiative aiming to make machine learning affordable and accessible through peer-to-peer networks. Is it the future of #AI? You decide: saltmarch.com/insight/petals… @BigscienceW
Never thought I'd see the day I'd have a publication in JMLR 🥹 So happy that the BLOOM carbon footprint paper has finally found a home at such an incredible venue! Thank you @shakir_za for being such a great editor, it warms my heart to see your name on this paper 💚
Thanks so much to @jen_gineered for inviting me, and @lara_groves @carolinesinders @ireni_mirena for the great discussion! And @SciGalleryLon for the lovely space!
If you wanted to see the fun panel/Q&A we did with Londoners on AI, you can check out the recording here! My preso at the start is also on Open Science, representing @huggingface & @BigscienceW.
Couldn't make it along to last week's event with @mmitchell_ai? Head over to our blog to watch Margaret's full presentation plus the lively panel discussion that followed feat. @lara_groves @irini_mirena & @carolinesinders london.sciencegallery.com/blog/watch-aga… @londondataweek #AI4Me
... @MasakhaneNLP @BigscienceW @ml_collective @arabicml2 @AiEleuther @carperai @LelapaAI which have actively have championed open science efforts -- + programs like CURE at Google support independent researchers w compute. What other open science efforts should be recognized?
Multilingual text corpus or Petri dish?
For my part, I'll present some highlights of the state-of-the-art data governance efforts for LLMs from @BigscienceW - a huge body of work led by @YJernite @mmitchell_ai @mcmillan_majora @HugoLaurencon @olapiktus @SashaMTL and many, many others. /4
Today, I presented our work at @BigscienceW in JCRAI @KFUPM. PromptSource=>T0=>BLOOM=>BLOOMz. Here are the slides docs.google.com/presentation/d…
@071625348 @BigscienceW It was moved here! Where did you find the link (so we can update it!) huggingface.co/spaces/bigscie…
@BigscienceW is the data catalog still accessible ? http://23.251.145.180:8501/ does not respond. Thanks !
How can you near-deduplicate 1.4 TB of data in under 4 hours for $60? The secret ingredient of StarCoder's performance is data curation more than anything else. Besides manual inspection we did extensive deduplication. Great tutorial by @MouChenghao: hf.co/blog/dedup
We are excited to see what people are gonna build with StarCoder. Get started with code examples in this repo to fine-tune and run inference on StarCoder: github.com/bigcode-projec… You can find all models/datasets/demos at hf.co/bigcode
With @TolokaAI we recruited 1,399 crowd-workers across 35 countries to annotate a diverse dataset for PII in code. Our PII detection model surpasses regex-based tools, especially for secret keys. PII dataset and model are available via gated access. hf.co/bigcode/starpii
Trends for United States
You might like
