Bobby @bobby_he
Machine Learning postdoc @ETH. PhD from @UniofOxford and former research intern @DeepMind/@samsungresearch bobby-he.github.io Zürich Joined January 2012-
Tweets39
-
Followers519
-
Following225
-
Likes1K
Why in neural networks the learning rate can transfer from small to large models (both in width and depth)? It turns out that the sharpness dynamics can explain it. Check out our new work! arxiv.org/abs/2402.17457 w/ @alexmeterez (co-first), @orvieto_antonio and T. Hofmann
If you are looking for a PhD position in the intersection between Deep Learning and Optimization, it's not too late to apply to my group at @MPI_IS and @ELLISforEurope Institute Tübingen! Send a DM if you are interested :) institute-tue.ellis.eu/research-group…
Interested in large language models? Worried about impacts of climate change? Come join us @oxcsml @NatureRecovery @UniofOxford in pushing the frontiers in #LLMs and at the same time help #NatureRecovery and address the impacts of #ClimateChange! bit.ly/4750IBO
Really fun project led by the amazing @yuhui_ding 💫 simple and efficient long range graph learning!!!📈
Really fun project led by the amazing @yuhui_ding 💫 simple and efficient long range graph learning!!!📈
Hot off the presses: ResNet hyperparameter transfer across depth and width! Tl;dr transfer for LR+schedules, momentum, L2 reg., etc. for wide ResNets and ViTs, with and without Batch/LayerNorm w/ @lorenzo_noci @mufan_li @BorisHanin @CPehlevan arxiv.org/abs/2309.16620
How do you scale Transformers to infinite depth while ensuring numerical stability? In fact, LayerNorm is not enough. But *shaping* the attention mechanism works! arxiv.org/abs/2306.17759 w/ @ChuningLi @mufan_li @bobby_he @THofmann2017 @cjmaddison @roydanroy
Mysterious observation: re-initializing neural nets during training can improve generalization, despite *no* change to the model, data or compute. We asked: when do re-initialization methods work? Paper📄: arxiv.org/abs/2206.10011 Poster🖼️: bit.ly/3mdjJAF (1/6)
New TMLR paper (w/ Francisca Vasconcelos, @bobby_he, and @yeewhye) on uncertainty quantification for low-dose CT reconstruction with implicit neural representations:
Collapsed or whitened features in self-supervised learning?🤔Turns out you can improve generalisation (esp in low-labelled data settings) by bridging between the two. Check out our work @ #icml today! 📒icml.cc/virtual/2022/p…
Presenting our work "Feature Kernel Distillation" (w. Mete Ozay) at ICLR poster session 5 today. We study the relevance of NN feature learning for (ensemble) knowledge distillation😊 Paper: openreview.net/forum?id=tBIQE…
Presenting our work "Feature Kernel Distillation" (w. Mete Ozay) at ICLR poster session 5 today. We study the relevance of NN feature learning for (ensemble) knowledge distillation😊 Paper: openreview.net/forum?id=tBIQE… https://t.co/1wd2gKjH2x
Mitzi Rouselle @RouselleMi18334
58 Followers 5K Following
James Allingham @JamesAllingham
984 Followers 456 Following RS @GoogleDeepMind | Machine Learning PhD @CambridgeMLG | 🇿🇦
lookingforobject @lyk92943182
29 Followers 321 Following
Taufeeq Abbasi @taufeeq_me
323 Followers 1K Following Postdoctoral fellow at @ZJU_China, Former Ph.D. scholar at @sjtu1896, and MS scholar of (@hecpkofficial) at @HuazhongUST
Zada Pillips @PillipZa
71 Followers 5K Following
Reza Sheikhi @resheikhi
76 Followers 1K Following
Amiee Deets @amiee2590
77 Followers 5K Following
rohit babbar @rohitbabbar_rb
151 Followers 1K Following ML/AI academic @ University of Bath, UK & Aalto University, Finland. Views are personal
Eva Louise Marie Gabr.. @e681554349
9 Followers 3K Following
Patan Bojang @b25085
14 Followers 161 Following
Deborah Sulem @DeborahSulem
47 Followers 103 Following Postdoc in Statistics @bse_barcelona. PhD in Stats @UniofOxford
Daisy-may Mowen @MayMowen43098
82 Followers 5K Following
Luna Pigue @LunaPigue36295
57 Followers 5K Following
Wan Plett @w_plett
65 Followers 5K Following
Huy Tran @huytransformer
91 Followers 3K Following
Frank Arneecher @FrankArnee35216
85 Followers 5K Following
Ayça Takmaz @aycatakmaz
334 Followers 397 Following PhD student @ETH Zurich, Student Researcher @GoogleAI
某种意义上来说 @cc1221
431 Followers 418 Following 1. 万千傻子随他去。 2. 读书不一定会变聪明，但不读书就一定会变傻。3.人间不值得。 #产品设计 #阅读 #音乐 #freelancer #PM #UI #Jazz
Rosa Halbrook @HalbrooR
45 Followers 5K Following
Eva-rose Ratliff @EvaRatl
54 Followers 5K Following
Yorgos Felekis @yfelekis
323 Followers 757 Following PhD student in Machine Learning and Causality @uniofwarwick | Member of Warwick Machine Learning Group | 🧪🖥️ | https://t.co/NDLyaDbOXb
Sophia Garibaldi @garibaldisoph
3 Followers 217 Following I’m interested in AI and other cool things, like books. @thequietways
Michael Schaarschmidt @m_schaarschmidt
206 Followers 431 Following Research Scientist @IsomorphicLabs, via @DeepMind, interested in scalable training methods
Anya Sims @anyaasims
44 Followers 96 Following PhD student @oxcsml @UniofOxford supervised by @yeewhye. Prev interned @graphcoreai; placement @CambridgeMLG. Deep learning, offline RL, LLMs X RL, meta-RL.
Mikołaj Piórczyńsk.. @AjPiorczynski
4 Followers 297 Following
Tim Gleason @neuralNet314
278 Followers 628 Following Studying and building AI, especially LLMs and RL agents.
Karolina Stanczak @karstanczak
515 Followers 446 Following NLP & ML PhD candidate @uni_copenhagen @CopeNLU
Steve Shen @SteveSh00938072
1 Followers 2K Following
Carolyn @klingenberg_car
181 Followers 3K Following
Saleh Ashkboos @AshkboosSaleh
547 Followers 285 Following Intern at @Apple | PhD Student at @spcl_eth, focused on High-Performance Computing and Large Scale Deep Learning | Prev. intern at @Microsoft and @MSFTResearch
Adam Sigal @TheAdamSigal
86 Followers 258 Following ML Research Engineer @Samsung_RA Montreal. Formerly ML and robotics @mcgillu + @MILAMontreal, CompSci @UMontreal. All tweets are someone else's opinions.
Jungtaek Kim @jungtaek_kim
144 Followers 745 Following Postdoctoral Associate at the University of Pittsburgh. Ph.D. at POSTECH. Former intern at @VectorInst, @SigOpt, and @Samsung.
Mauro Camara Escudero @MauroCamaraE
302 Followers 2K Following Statistical Machine Learning, Dancing, Basketball & Cooking 🎈
Aadit Shukla @aadit_shukla
30 Followers 295 Following Learner| Data Analyst| Machine learning| CSE 24|| Student
Fabian Falck @fabianfalck
218 Followers 894 Following PhD student in ML @UniofOxford @oxcsml @OrielOxford Prev. @MSFTResearch @AmazonScience @imperialcollege @KITKarlsruhe (Probabilistic) Generative Models
Dimitri von Rütte @dvruette
709 Followers 171 Following Studies @ETH_en, Machine Learning @DeepJudgeAI
Tiago Pimentel @tpimentelms
1K Followers 248 Following Postdoc at @ETH_en. Formerly, PhD student at @Cambridge_Uni.
El Walid Aboulaakoul @ElWalid_
55 Followers 1K Following Student at ENSAB | AI Automation Enthusiast
Dominick Romano @dromanocpm
2K Followers 3K Following 🇺🇸 Engineer & Inventor Internationally Recognized Speaker 🇺🇳 UN/WHO/ITU AI for Radiology 🩻 💥 Top 50 AI CEO 2021 & 2023 🌎📡 #AI #ML #Startup #AIForGood
Josh Robinson @Josh_d_robinson
720 Followers 368 Following Postdoc at @Stanford. PhD from @MIT_CSAIL.
Rishabh Gupta @arishabh8
413 Followers 4K Following Climate Change + ML , Google ML Camp, UTokyo, IITK, Views are my own. RT ≠ endorsements
Vincent Lordier @vlordier
570 Followers 4K Following
Mohammed Abdallah @melsiddieg14
88 Followers 3K Following Postdoctoral Biomedical Reseracher, Intersted in integrative and scalable biomedical research
EdgeAI Geek @edgeaiguy
1K Followers 5K Following Crafting AI solutions for tiny devices. | Ex-Samsung |
Ayça Takmaz @aycatakmaz
334 Followers 397 Following PhD student @ETH Zurich, Student Researcher @GoogleAI
Michael Schaarschmidt @m_schaarschmidt
206 Followers 431 Following Research Scientist @IsomorphicLabs, via @DeepMind, interested in scalable training methods
Anya Sims @anyaasims
44 Followers 96 Following PhD student @oxcsml @UniofOxford supervised by @yeewhye. Prev interned @graphcoreai; placement @CambridgeMLG. Deep learning, offline RL, LLMs X RL, meta-RL.
Vaishnavh Nagarajan @_vaishnavh
2K Followers 530 Following Research scientist at Google || CS PhD at Carnegie Mellon. Interested in the theory of AI & Machine Learning. he/him 🏳️🌈
Saleh Ashkboos @AshkboosSaleh
547 Followers 285 Following Intern at @Apple | PhD Student at @spcl_eth, focused on High-Performance Computing and Large Scale Deep Learning | Prev. intern at @Microsoft and @MSFTResearch
Volodymyr Kyrylov @darkproger
2K Followers 2K Following AI student at USI/ETH. Donate https://t.co/GDSkWG2tak
Tim Dettmers @Tim_Dettmers
29K Followers 821 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.
Tiago Pimentel @tpimentelms
1K Followers 248 Following Postdoc at @ETH_en. Formerly, PhD student at @Cambridge_Uni.
Dimitri von Rütte @dvruette
709 Followers 171 Following Studies @ETH_en, Machine Learning @DeepJudgeAI
Mistral AI @MistralAI
90K Followers 0 Following Fast, open-source and secure language models. Join us https://t.co/INALdNGvCP
Alex Hägele @haeggee
382 Followers 475 Following PhD Student in Machine Learning @ICepfl. MSc/BSc from @ETH_en. Previously: Student Researcher @Apple MLR. @[email protected]
Jeff Dean (@🏡) @JeffDean
296K Followers 6K Following Chief Scientist, Google DeepMind and Google Research. Co-designer/implementor of things like @TensorFlow, MapReduce, Bigtable, Spanner, Gemini .. (he/him)
Shahine Bouabid @shbouabid
176 Followers 304 Following Statistical modelling for climate science @OxCSML @iMIRACLI_ITN 🇲🇦 @nechfate 📷 https://t.co/3P5sLpm3nE. (he/him)
Josh Robinson @Josh_d_robinson
720 Followers 368 Following Postdoc at @Stanford. PhD from @MIT_CSAIL.
Horace He @cHHillee
23K Followers 449 Following Working at the intersection of ML and Systems @ PyTorch "My learning style is Horace twitter threads" - @typedfemale
Basil Mustafa @_basilM
1K Followers 130 Following researching ML @ google brain ZRH | no strong opinions about AI, but very strong opinions about why herbal infusions are awful and should not be called teas
Momchil Konstantinov @MPKonstantinov
53 Followers 271 Following ML/NLP practitioner, former symplectic geometer
Daniel Worrall @danielewworrall
3K Followers 1K Following Research Scientist @ Google DeepMind ⚡ ML, Geometric DL, Learn2Simulate ⚡ LDN ⬅️ AMS ⬅️ LDN ⬅️ CAM ⚡ British-Indonesian ⚡ he/him/whomst 🏳️🌈 ⚡ Views my own
Javier Sanguino @fjsanguino
76 Followers 239 Following Machine Learning & AI as a graduate student in @ETH. Trying to understand how do we use AI to help humans
Michael Bronstein @mmbronstein
43K Followers 4K Following #DeepMind Professor of #AI @UniofOxford / Fellow @ExeterCollegeOx / ML Lead @ProjectCETI / https://t.co/kZpGpDzYeV
Maksym Andriushchenko.. @maksym_andr
3K Followers 931 Following phd student at @EPFL🇨🇭 // google & open phil phd ai fellow // past @adoberesearch @uni_tue // best way to support 🇺🇦 https://t.co/fxomgJ7NU9
Kevin K. Yang 楊凱�.. @KevinKaichuang
16K Followers 5K Following Senior Researcher in BioML @MSFTResearch (@MSRNE). He/him/他. 🇹🇼
Daisuke Okanohara / �.. @hillbig
30K Followers 619 Following Co-founder and CER of Preferred Networks (PFN). CEO of PFCC. CEO of PFE. Interested in AI, science, and business.
David McKean @DrDavidMcKean
1K Followers 2K Following MSK Radiologist. MSK&Spinal Fellowship Program Director. ISS, ESSR & BSSR member. 4kids, 2dogs, 1wife, 0time. #mskrad #MSKUS All views my own.
Giulia Mazzini @giuliammaz
189 Followers 624 Following Biomedicine PhD student at University of Zürich @uzh_en @uzh_vetsuisse. Interested in neuroscience and metabolism.
Shubhendu Trivedi @_onionesque
7K Followers 851 Following Cultivated Abandon. Twitter interests: Machine learning research, applied mathematics, mathematical miscellany, ML for Physics/Chemistry, books.
Lucas Beyer (bl16) @giffmana
56K Followers 447 Following Researcher (Google DeepMind/Brain in Zürich, ex-RWTH Aachen), Gamer, Hacker, Belgian. Mostly gave up trying mastodon as [email protected]
Amir Joudaki @AmirJoudaki
221 Followers 162 Following mathematical foundations of AI, AI for biomedicine @ETH_en website: https://t.co/hBVN7FWyCE
Nodens @NodensKoren
49 Followers 197 Following Machine Learning Researcher & Computational Astrophysicist. Aspiring Pianist. Studying gravitational waves and the universe using ML @ETH @CSatETH 😃
Enis Simsar @enisimsar
227 Followers 256 Following Phd at @ETH_en, Ex. ML Intern @Apple, MSc. Student at @TU_Muenchen
No Context Brits @NoContextBrits
1.8M Followers 0 Following A celebration of Great Britain. Contains sarcasm, irony and context. Want to know how to really be a Brit? Link below.
Grigory @gakhromov
16 Followers 65 Following
Sahib Singh @sahibcantsingh
73K Followers 537 Following standup comedian and writer. TOUR DATES/Tix👇
Jannis Bolik @BolikJannis
10 Followers 92 Following
Leo Klarner @leoklarner
261 Followers 147 Following PhD Student, Clarendon Scholar at University of Oxford. @OPIGlets & @oxcsml. Improving the robustness of deep learning algorithms in early-stage drug discovery.
Shreshth Malik @ShreshthMalik
201 Followers 612 Following Machine Learning PhD student @OATML_Oxford @aims_oxford
Gregor Bachmann @GregorBachmann1
233 Followers 274 Following I am a PhD student @ETH Zürich working on deep learning. MLP-pilled 💊. https://t.co/yWdDEV6Z15
Russ Cook @hardestgeezer
354K Followers 185 Following The first person ever to run the full length of Africa @huel @perfecttednrg @hoka CHARITY FUNDRAISER⬇️Thrilled to announce that (1) I've successfully defended my PhD thesis, "Improving Deep Learning with Probabilistic Approaches," which I'll share once corrections are done, and (2) Today I started my dream job as a research scientist @GoogleDeepMind! Excited for the future! 🥳
PhD students deserve better financial conditions.
While packing, I found this “vintage” OpenAI t-shirt from 2016 BC(-hatGPT). I feel old… #openai #neurips2016 #swag
Super excited to share that I successfully defended my PhD thesis "Understanding Generalization and Robustness in Modern Deep Learning" today 👨🎓 A huge thanks to the thesis examiners @SebastienBubeck, @zicokolter, and @KrzakalaF, jury president Rachid Guerraoui, and, of course,…
You are not prepared for how this guy pronounces Schweppes. 😭
why didn’t voldemort just run harry potter over with a car
I’m excited to announce that in July 2025 I will be joining @UWaterloo as an Assistant Professor in the Department of Statistics and Actuarial Science! Until then, I will continue at Princeton as a DataX Postdoc Fellow, working with Boris Hanin. I have many exciting projects…
Following our previous work, we are releasing RecurrentGemma - a fully open source 2B model based on our Griffin architecutre! Code + weights as everyone has wished for! Code on Github: github.com/google-deepmin… Weights on Kaggle: kaggle.com/models/google/…
Who did this? ‘Me and the lads are exactly the same’ 🤣 Genius ..
This was such a special experience. The MFO itself, the discussions there, the chance to talk about our work on online model selection, and ofc the mandatory group photo in front of the Boy's surface statue. Would recommend! 🫶
The Oberwolfach Research Institute for Mathematics (MFO) was also the location of the latest #ELLISProgram workshop on Interactive Learning and Interventional Representations. Check out this detailed recap by Giorgia Ramponi @gio_ramponi: ifi.uzh.ch/en/alpi/blog.h…
[1/7] Happy to release 🥕QuaRot, a post-training quantization scheme that enables 4-bit inference of LLMs by removing the outlier features. With @akmohtashami_a @max_croci @DAlistarh @thoefler @jameshensman and others Paper: arxiv.org/abs/2404.00456 Code: github.com/spcl/QuaRot
I successfully defended my PhD today (exciting!). I've struggled today balancing feelings of accomplishment+gratitude and anti-climatic disappointment. It's been more than worth it to hear my youngest daughter toddle around saying "Doc-tor Dad!" 🥰
transitioning from a Bay Area tech worker type of guy to a back problems type of guy
There is already a Switzerland of AI. It's called Switzerland
Brexit may yet turn out to have been a good idea, if it means the UK can be the Switzerland of AI.
📢 I am looking for a student researcher to work with me and my colleagues at Google DeepMind London on understanding & building new neural network architectures. Please reach out to me ([email protected]) and apply below before Mar 22 if interested! google.com/about/careers/…
Just got back from vacation, and super excited to finally release Griffin - a new hybrid LLM mixing RNN layers with Local Attention - scaled up to 14B params! arxiv.org/abs/2402.19427 My co-authors have already posted about our amazing results, so here's a 🧵on how we got there!
i passed my phd viva today !!! i give thanks to all of the (countless) beings that supported me and contributed to this <3 thank you @yeewhye @tom_rainforth @eric_nalisnick and so many others <3
We build neural codecs from a *single* image or video, achieving compression performance close to SOTA models trained on large datasets, while requiring ~100x fewer FLOPs for decoding ⚡ #CVPR2024 c3-neural-compression.github.io
From stochastic parrot 🦜 to Clever Hans 🐴? In our work with @_vaishnavh we carefully analyse the debate surrounding next-token prediction and identify a new failure of LLMs due to teacher-forcing 👨🏻🎓! Check out our work arxiv.org/abs/2403.06963 and the linked thread!
🗣️ “Next-token predictors can’t plan!” ⚔️ “False! Every distribution is expressible as product of next-token probabilities!” 🗣️ In work w/ @GregorBachmann1 , we carefully flesh out this emerging, fragmented debate & articulate a key new failure. 🔴 arxiv.org/abs/2403.06963
Trends for United States
You might like
