-
Tweets51
-
Followers4K
-
Following275
-
Likes619
IMHO there are many cool aspects to this work, but I’d like to call out page 16 - which contains the entire model code, and isn’t even full.
IMHO there are many cool aspects to this work, but I’d like to call out page 16 - which contains the entire model code, and isn’t even full.
If you are at #ICLR2019, make sure that you learn about our work, "Universal Transformers", w/ @sgouws, @OriolVinyalsML, @kyosu, and @lukaszkaiser (Thursday 11am-1pm, poster session at Great Hall BC, #62). You can also check out this blog post about UT: mostafadehghani.com/2019/05/05/uni…
Check out Universal Transformers, new research from the Google Brain team & @DeepMindAI that extends last year's Transformer (a neural network architecture based on a self-attention mechanism) to be computationally universal. goo.gl/j4jWnu
Ashish and Noam presenting our work on Transformers at NIPS right now! Come to our poster tonight! #NIPS2017
Check out the Transformer, a novel NN architecture based on a self-attention mechanism that is well-suited for NLU goo.gl/qxN6ej
AK @_akhaliq
310K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo follow on Hugging Face: https://t.co/q2Qoey80GxDelip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | PianistColin Raffel @colinraffel
30K Followers 655 Following nonbayesian parameterics, sweet lessons, and random birds. Friend of @srush_nlpKyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).David Pfau @pfau
22K Followers 1K Following Knowledge manifests itself in radiant dreams that shimmer like the wild sun Views are my own pfau at sigmoid dot social on 🦣 https://t.co/xqtVHHVI17 on 🦋Ofir Press @OfirPress
10K Followers 3K Following I build tough benchmarks for LMs and then I get the LMs to solve them. Postdoc @Princeton. PhD from @nlpnoah @UW. Ex-visiting researcher @MetaAI & @MosaicML.Sander Dieleman @sedielem
50K Followers 2K Following Research Scientist at Google DeepMind. I tweet about deep learning (research + software), music, generative models (personal account).Eric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0pMichael Bronstein @mmbronstein
43K Followers 4K Following #DeepMind Professor of #AI @UniofOxford / Fellow @ExeterCollegeOx / ML Lead @ProjectCETI / https://t.co/kZpGpDzYeVrishi @RishiBommasani
4K Followers 2K Following Stanford CS PhD @StanfordCRFM @StanfordNLP @StanfordAILab @StanfordHAI Advisers: @percyliang @jurafsky Previous: @CornellCIS @clairecardie #FoundationModelsJacob Andreas @jacobandreas
14K Followers 958 Following Teaching computers to read. Assoc. prof @MITEECS / @MIT_CSAIL (he/him). https://t.co/5kCnXHjtlY https://t.co/2A3qF5vdJwGraham Neubig @gneubig
31K Followers 588 Following Associate professor at CMU, studying natural language processing and machine learning.Abel Carrera @AbelKrrera
2 Followers 166 FollowingKarlmichl8 @Karlmichl8
144 Followers 3K FollowingNikita @nikitavoloboev
4K Followers 7K Following Make @LearnAnything_ Learn in public: https://t.co/GbFvuErkYn macOS course: https://t.co/JdbJWru6zG https://t.co/94R8ER7K2h https://t.co/ROkqhyhpEKjooeyzz @jooeyzz
127 Followers 3K Followingwalter de brouwer @walterdebrouwer
5K Followers 1K Following CompLinguistics| #Stanford | @TEDAI |W https://t.co/60JgztUeXL | @RecordingAcad|Chen Zhang @maginazc
7 Followers 90 FollowingDavid Meyer @DavidLMeyer1
63 Followers 481 FollowingDevanshu Tiwari @Devanshu0055
6 Followers 177 FollowingJuan Carlos Cuartas @jccuartar
15 Followers 28 FollowingMarya @MaryaUnw
46 Followers 146 Following 📍 Bay Area | XR | Gaming | Ex-Niantic/EA/Deloitte 🐻 MBA Candidate @ UC Berkeley Haas '25 🤘 McCombs BBA Finance '17 🤠 AustiniteDaniel Kirchleitner @danielkirchl
117 Followers 714 Following B2B SaaS & tech investor. @Cargo_one_, @sennderofficial, @Scoutbee, @Wandelbots, Alaiko, @joinBlink, @Varjodotcom. Fellow @KauffmanFellows.dino_dna @dino_dna_
498 Followers 4K FollowingGus @Gus63933654
311 Followers 3K FollowingPhonkNerdyBit @oncs01
20 Followers 332 Following Unleashing CS Inquiry bombs, chill composure, slinging sarcastic comments. Stay sharp, stay savvy. #NoFearInquiryAlexander Morosow @alex5m6
4 Followers 36 Following Head of Creative Engineering & Software Architect @refikanadol studio | @datalandmuseum | simplify omnidirectional motionMohamed Alfatih @MohamedAlfx
0 Followers 29 FollowingThomas Amberg @tamberg
2K Followers 5K Following Maker/👩💻engineer. Founder @yaler. Organiser @iotzh. Embracing the future. Becoming a teacher. Moving to https://t.co/8mQZUIrSGlJames Leu @skydetainer
127 Followers 3K Following When you can understand and explain the universe,you’re a smart man.Neil Lofland @DfenseNdepth
6 Followers 46 FollowingMike Sexton @MikeESexton
1K Followers 5K Following Cat dad. Cyber/AI @ThirdWayNatSec. NextGen @FP4America. 40 under 40 @MidEastPolicy. Polyglot-ish. He/هو/הוא. mikesexton(@)infosec(.)exchange.Orkan Telhan @orkan
547 Followers 288 Following Chief Information and Data Officer @ecovative Board President @biodesigneddavidluo17 @davidluo_ymu
45 Followers 132 Following A passionate enterprise digitalization transformer, product manager with over 20 years on ERP, BPM, ECommerce, focusing on UX backed by BI/AI+ Efficiency rulesPedro Martins @PedroHenMartins
75 Followers 587 Following Research Scientist at Unbabel | PhD in Machine learning and NLP | LiberalTenthLine49 @TenthLine49
5 Followers 2K FollowingThiago Alvarez @thiago_alvarez
106 Followers 598 Following Creator of Open Banking/Finance in Brazil. Guiabolso founder (sold to PicPay); Angel Investor; Hawaii/BrazilSzilágyi Pál @szilagyipal
292 Followers 971 Following Director (Competition Law Research Centre), Assistant Professor (PPCU) “I’m smart enough to know that I’m dumb.” – Richard FeynmanKunal e/acc---EA @kunal7732
66 Followers 620 Following e/acc | EA | Enthusiastic and Optimistic about life ,I post tweets about AI,Tesla,and some general stuff .gaizaþrūþ @glowinthedark38
19 Followers 713 FollowingJoel Mushagasha @JoelMushagasha
73 Followers 237 Following Co-founder of @EndurantHealth Computational Biologist, ML Engineer, Generative AI previously: @NIH, @CarnegieMellon Science is magic that worksDr. Sanjee Perera @SanjeePerera1
4K Followers 4K Following #CognitivePsychologist in Identity & Moral justice/ judgement development. Interdisciplinary academic forager. Formerly the Archbishops’ Adviser for MEAC.Ashutosh Mehra @ashutoshmehra
1K Followers 5K Following Senior Principal Scientist at Adobe. Working on Acrobat AI Assistant, LLMs, and document ML.Firas Fadhl @FadhlFiras
3 Followers 76 FollowingAndrew X Ye @Andrew_XYe
64 Followers 280 FollowingPuwanat Sangkhapreech.. @sangkhapreecha
4 Followers 2K Following City boy from Bangkok,Thailand. JHU Biochem Eng PhD Student. Interested in biotech, food, dancing, soccer, people, travelling.Andres @AndresMilioto
481 Followers 2K Following Robotics, Computer Vision, Machine Learning 👨🏻💻📷👨🏻🔧🤖Abey @HudAbey
46 Followers 136 FollowingPhilipp Recherche @PPAIRECH
11 Followers 43 FollowingAndrej Karpathy @karpathy
979K Followers 905 Following 🧑🍳. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥Yann LeCun @ylecun
712K Followers 719 Following Professor at NYU. Chief AI Scientist at Meta. Researcher in AI, Machine Learning, Robotics, etc. ACM Turing Award Laureate.François Chollet @fchollet
470K Followers 769 Following Deep learning @google. Creator of Keras. Author of 'Deep Learning with Python'. Opinions are my own.(((ل()(ل() 'yoav))).. @yoavgo
46K Followers 2K FollowingLucas Beyer (bl16) @giffmana
56K Followers 446 Following Researcher (Google DeepMind/Brain in Zürich, ex-RWTH Aachen), Gamer, Hacker, Belgian. Mostly gave up trying mastodon as [email protected]Delip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈George Hotz 🌑 @realGeorgeHotz
248K Followers 174 Following President @comma_ai. Founder @__tinygrad__Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGzPercy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | PianistChristopher Manning @chrmanning
127K Followers 116 Following Director, @StanfordAILab. Assoc. Director, @StanfordHAI. Founder, @stanfordnlp. Prof. CS & Linguistics, @Stanford. IP @aixventureshq. 🇦🇺 Do #NLProc & #AI. 👋Oriol Vinyals @OriolVinyalsML
167K Followers 82 Following VP of Research & Deep Learning Lead, Google DeepMind. Gemini co-lead. Past: AlphaStar, AlphaFold, AlphaCode, WaveNet, seq2seq, distillation, TF.Ilya Sutskever @ilyasut
370K Followers 2 Following towards a plurality of humanity loving AGIs @openaiColin Raffel @colinraffel
30K Followers 655 Following nonbayesian parameterics, sweet lessons, and random birds. Friend of @srush_nlpKyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).Richard Socher @RichardSocher
101K Followers 971 Following CEO @youSearchEngine Investing at @aixventuresHQ Before: Stanford Adj Prof in AI/NLP, Chief Scientist at Salesforce, MetaMindOfir Press @OfirPress
10K Followers 3K Following I build tough benchmarks for LMs and then I get the LMs to solve them. Postdoc @Princeton. PhD from @nlpnoah @UW. Ex-visiting researcher @MetaAI & @MosaicML.Tara Basu Trivedi @tbt94
3K Followers 3K Following using tech to improve human health • prev: product @pactpharma, swe @google @theteamatx, compbio @brownuniversity, ms/mba @harvardhbsTrevor McCourt @trevormccrt1
10K Followers 265 Following CTO @Extropic_AI. PhD (ABD) + MSc @MITEECS + @MitQuanta + @FieteGroup. former @GoogleAI Quantum, former @UWaterloo Mechanical Engineering. opinions my ownCaglar Gulcehre @caglarml
4K Followers 1K Following ML Researcher Prof @ EPFL, PI @ CLAIRE lab Ex: Staff Research Scientist @ Deepmind, MSR, IBM Research Follow me on Mastodon: https://t.co/LZ5sWt7AsjBen Mildenhall @BenMildenhall
5K Followers 991 Following making stuff 3D. formerly research scientist at Google, phd at Berkeley.BioNTech SE @BioNTech_Group
83K Followers 166 Following Our vision is to harness the power of the immune system to translate science into survival $BNTX https://t.co/eQJYOy56xwJakob Foerster @j_foerst
14K Followers 820 Following Assoc. Prof in ML @UniofOxford @StAnnesCollege @FLAIR_Ox, dad. Ex: {RS @MetaAI, (A)PM @Google, DivStrat @GS}, ex intern {@GoogleDeepmind, @GoogleBrain, @OpenAI}Gandeeva Therapeutics @Gandeeva_Tx
384 Followers 129 Following @Gandeeva_Tx is a structure-guided biotechnology company integrating the power of cryo-EM and machine learning to develop highly targeted, novel therapies.Elon Musk @elonmusk
181.6M Followers 585 FollowingJoshua March @joshuamarch
8K Followers 2K Following Co-founder & CEO of @eatscifi, making meat the world can depend on. Previously Co-founder & CEO of @Conversocial (acquired by Verint).Josephine Chen @josephinekchen
3K Followers 329 Following Partner @Sequoia (seed/A/B). Chef @joinRamenMafiaRenee Yao @ReneeYao1
2K Followers 1K Following Global Healthcare AI Startups Lead, NVIDIA | Ballroom Dancer, Food, Fashion Lover | San Jose | Tweets are my own, shamelessly promote what I loveMohit Bansal @mohitban47
9K Followers 651 Following Parker Distinguished Professor, UNC Chapel Hill (@unc). Director https://t.co/5qlPVgnrlN (@uncnlp). Prev: @Berkeley_AI, @TTIC_Connect @IITKanpur #NLP, #CV, #AI, #MLMatthew Johnson @SingularMattrix
12K Followers 3K Following Researcher at Google Brain. I work on JAX (https://t.co/UGa5tGfinF).Simon Kornblith @skornblith
3K Followers 999 Following researcher/engineer @AnthropicAI | former @GoogleDeepMind @mitbrainandcog @zotero | @[email protected]Tianle Cai @tianle_cai
5K Followers 4K Following ML PhD @Princeton. Life-long learner, hacker, and builder. Tech consultant & angel investor. Prev @togethercompute @GoogleDeepMind @MSFTResearch @citsecurities.S32 @S32_VC
1K Followers 127 Following S32 is a venture capital fund investing at the frontiers of technology.Inceptive — Learnin.. @InceptiveCom
76 Followers 2 FollowingVijay Pande @vijaypande
32K Followers 742 Following Founder, GP, & Managing Partner of a16z bio+health. Founder, Folding@home. Investor, Scientist, Engineer, Founder. AI, Bio, everything in between. https://t.co/X4mRNyio9SSue Hager @suehager94
251 Followers 621 Following Operating Partner, CMO @a16z Bio + Health Biotech exec, hack surfer, animal lover, hot yoga devoteeThermal @ThermalPR
724 Followers 4K Following The visionaries driving society’s greatest scientific and medical advances choose Thermal to communicate how they Impact Human Health™a16z Bio + Health @a16zBioHealth
3K Followers 108 Following Backing bold entrepreneurs who are engineering biology and reimagining healthcare.Ashish Vaswani @ashVaswani
19K Followers 2K FollowingTri Dao @tri_dao
19K Followers 365 Following Incoming Asst. Prof @PrincetonCS, Chief Scientist @togethercompute. Machine learning & systems.Databricks Mosaic Res.. @DbrxMosaicAI
30K Followers 115 Following We remove the barriers to state-of-the-art generative AI model development and make data + AI available to all.Sanja Fidler @FidlerSanja
14K Followers 483 Following Associate Professor @UofT, Vice President of AI Research @nvidia, founding member of @VectorInst. Computer vision, deep learning, 3D. Opinions are my own.Bernhard Schölkopf @bschoelkopf
14K Followers 60 FollowingFT Technology News @fttechnews
298K Followers 25 Following @financialtimes news and analysis about the tech industrySmart Biology @SmartBiology3D
61K Followers 362 Following 3D animated, interactive, biology courseware so students can truly understand. #3D #biology #education #animationKevin K. Yang 楊凱�.. @KevinKaichuang
16K Followers 5K Following Senior Researcher in BioML @MSFTResearch (@MSRNE). He/him/他. 🇹🇼Michael Levin @drmichaellevin
40K Followers 2K Following Scientist at Tufts University; my lab studies anatomical and behavioral decision-making at multiple scales of biological, artificial, and hybrid systems.Dipanjan Das @dipanjand
4K Followers 308 Following Senior Director of Research at @GoogleDeepmind. Working on improving the factuality of LLM generated content.ramsey homsany @rhomsany
2K Followers 2K Following building @Octantbio. ex-@Dropbox. ex-@Google. @representus & @rutgersSOE boards.Christian Wolf @chriswolfvision
7K Followers 1K Following Principal Scientist at @NaverLabsEurope, Lead of Spatial AI team. AI for Robotics, Computer Vision, Machine Learning. Austrian in France. IEEE-PAMI area editor.Vincent Sitzmann @vincesitzmann
13K Followers 296 Following Assistant Professor @ MIT, leading the Scene Representation Group (https://t.co/h5gvhLYrtw). Neural scene reps., neural rendering, inverse graphics.Richard H. Ebright @R_H_Ebright
70K Followers 282 Following Board of Governors Professor of Chemistry and Chemical Biology @RutgersU @BiosafetyNowSepp Hochreiter @HochreiterSepp
10K Followers 395 Following Pioneer of Deep Learning and known for vanishing gradient and the LSTM. I mostly tweet about random ArXiv papers which sparked my interest.Lilian Weng @lilianweng
95K Followers 148 Following Working on AI safety, past on robotics, applied research @OpenAI; Writing ML blogs to help myself & others to learn; Ideas my own.Sergey Levine @svlevine
80K Followers 122 Following Associate Professor at UC Berkeley Co-founder, Physical IntelligenceD. Sivakumar @dsivakumar
4K Followers 852 Following Co-Founder: https://t.co/CHdqCYS1My (Commerce Search with NLP Magic) Earlier: Member, @southpkcommons; NLP, ML, Algorithms at Google Research; CS Theory at IBM Almaden .Byron Reese @byronreese
4K Followers 3K Following Futurist, technologist, entrepreneur, bestselling author. I write about AI & deciphering our destiny. New Book: We Are Agora https://t.co/VBCy6tuZYJMarie Vidal @mv_mvidal
254 Followers 723 Following Innovation & industry manager @BIMSB_MDC @Virchow2_0 @LifeTimeIni previously @circrtrain @CORBEL_eu @singlecellomicsRunway @runwayml
185K Followers 300 Following An applied AI research company building for the next era of art, entertainment and human creativity. We're hiring: https://t.co/Aj11xyhxOgIf you ever actually looked at these benchmarks, the model predictions, and what the claimed "human performance" means, you would know. Hint: it's unrelated to intelligence. Looks like many people, especially more prominent ones, are commenting and opining blindly.
Interesting how in all these domains AI is asymptoting at roughly human performance - where's the AI zooming past us to superintelligence that Kurzweil etc. predicted/feared?
> be me > on vacation > kid asleep, wife away > but I'm not tired! > whip out colab > load my model > import new benchmark > try my model > tfw sota, sota by far > double-check for bugs or leaks > no bug found > no leak found idk man, probably a bug. Also, twitter is reddit now.
At this point I feel like we understand pretty well what's going on with LLMs: - Outputs are roughly equivalent to kernel smoothing over positional embeddings (arxiv.org/pdf/1908.11775…) - The learned computation model is *probably* bounded by RASP-L (arxiv.org/pdf/2310.16028…) -…
So much potential🥹 1: One moment, let me state the vision No more sliding windows, we on a mission! udio.com/songs/mfsfmg5V… 2: No 1x1, that's just a view @ylecun can't argue, our models slew udio.com/songs/eb2bmgzT…
Happy to share - blah blah blah. Gemma + Griffin = RecurrentGemma Competitive quality with Gemma-2B and much better throughput, especially for long sequences. Cracked model from cracked team! Check it out below 👇
Releasing RecurrentGemma - one of the strongest 2B-param open models designed for fast inference on long sequences and massive throughput! Both pre-trained and IT checkpoints available + code - try them out here! Code: github.com/google-deepmin… Weights: kaggle.com/models/google/…
@DannyMcAteer8 I was in Bhutan and, unsurprisingly, there was a lot of meditating. An excellent place for that, and really loved the place/people/culture there.
Returning from an experimental ~2 week detox from the internet. Main takeaway is that I didn't realize how unsettled the mind can get when over-stimulating on problems/information (like a stirred liquid), and ~2 weeks is enough to settle into a lot more zen state. I'm struck by…
@demishassabis @GoogleDeepMind Congrats, Sir Demis!
Slowly but surely freeing us from the discretization in tokenization! Now the code and checkpoints of GIVT are available, so you can play around with them:
We just released a big 🎁GIVT update! 📈 Larger models and improved image generation results across the board 💡 Improved GMM formulation and adapter module 💻 Code, model checkpoints, and a colab are now available at github.com/google-researc… More details below... 1/
Decoder-only models only work with discrete tokens, right? 🤔 Excited to present 🎁GIVT: Generative Infinite-Vocabulary Transformers, a simple way to generate arbitrary vector sequences with real-valued entries using transformer decoder-only models! arxiv.org/abs/2312.02116 1/
We just released a big 🎁GIVT update! 📈 Larger models and improved image generation results across the board 💡 Improved GMM formulation and adapter module 💻 Code, model checkpoints, and a colab are now available at github.com/google-researc… More details below... 1/
Decoder-only models only work with discrete tokens, right? 🤔 Excited to present 🎁GIVT: Generative Infinite-Vocabulary Transformers, a simple way to generate arbitrary vector sequences with real-valued entries using transformer decoder-only models! arxiv.org/abs/2312.02116 1/
It was so great to see almost everyone (we missed you @nikiparmar09!!) from the Transformer paper again. We still haven't all been in the same room at the same time, but we'll make it happen one day. @lukaszkaiser @kyosu @ashVaswani @ilblackdragon @YesThisIsLion
@ylecun @StevenLevy @DBahdanau @kchonyc Self-attention is NOT the innovation in Transformers. Others had used it before as is cited in their background section. The contribution of the realization that you no longer need recurrence, which, together with causal masking, enables training parallelization.
I’m really excited to be starting a new adventure with multiple amazing friends & colleagues. Our company is called Physical Intelligence (Pi or π, like the policy). A short thread 🧵
I am incredibly proud to be able to put this paper out finally! This paper shows that hybrid linear RNNs (Griffin) combined with local attention (or sliding window attention) can be incredibly efficient at language modeling.
We present Griffin: A hybrid model mixing a gated linear recurrence with local attention. This combination is extremely effective: it preserves all the efficient benefits of linear RNNs and the expressiveness of transformers. Scaled up to 14B! arxiv.org/abs/2402.19427
I agree, this year will be wild for foundation world models! Thanks a lot Jim 🙏 I need to emphasize that I was lucky to work with an incredibly talented team: Jake Bruce, @MichaelD1729, @ashrewards, @jparkerholder, @YugeTen, @edwardfhughes, Matthew Lai, @aditimavalankar,…
Tim is one of the most imaginative researchers I know, and Genie is one of his most imaginative works. Unlike Sora, Genie is actually a proper action-driven world model with inferred actions. 2024 will also be the Year of Foundation World Models! Congrats 👏👏
Rather than adding inductive biases, we focus on scale. We use a dataset of >200k hours of videos from 2D platformers and train an 11B world model. In an unsupervised way, Genie learns diverse latent actions that control characters in a consistent manner.
I am really excited to reveal what @GoogleDeepMind's Open Endedness Team has been up to 🚀. We introduce Genie 🧞, a foundation world model trained exclusively from Internet videos that can generate an endless variety of action-controllable 2D worlds given image prompts.
Love letter to @obsdmd to which I very happily switched to for my personal notes. My primary interest in Obsidian is not even for note taking specifically, it is that Obsidian is around the state of the art of a philosophy of software and what it could be. - Your notes are…