Michael Li @lzy_michael
Research Fellow at @GatsbyUCL. previously PhD student in machine learning and statistics @OxCSML and @OxfordStats Oxford, England Joined May 2016-
Tweets32
-
Followers194
-
Following316
-
Likes33
In-context learning (ICL) circuits emerge in a phase change... Excited for our new work "What needs to go right for an induction head (IH)?" We present "clamping", a method to causally intervene on dynamics, and use it to shed light on IH diversity + formation. Read on 🔎⏬
Really enjoyed writing this piece with @torfjelde and @vdutor 🙌 Thanks @msalbergo @ValentinDeBort1 @JamesTThorn for your insightful feedback 👌
Really enjoyed writing this piece with @torfjelde and @vdutor 🙌 Thanks @msalbergo @ValentinDeBort1 @JamesTThorn for your insightful feedback 👌
Weijie is a great mentor!
"Learning to act in noisy contexts using deep proxy learning" Talk video now online! youtu.be/IGK9RL5KoEc?si… Slides: gatsby.ucl.ac.uk/~gretton/paper… from the Series on Modern AI at NYU engineering.nyu.edu/academics/depa… Based on papers: arxiv.org/abs/2105.04544 arxiv.org/abs/2106.03907
ICML 2024 authors: Please participate in our study on improving peer review in ML! Rank your submissions confidentially under "Author Tasks" on OpenReview. More details at openrank.cc. Thank you! @ENAR_ibs
ICML 2024 authors: Please participate in our study on improving peer review in ML! Rank your submissions confidentially under "Author Tasks" on OpenReview. More details at openrank.cc. Thank you! @ENAR_ibs
📢 #ICML2024 authors! Help improve ML peer review! 🔬📝 Check your inbox for an email titled "[ICML 2024] Author Survey" and rank your submissions. 🏆📈 Your confidential input is crucial, and won't affect decisions. 🔒✅ Survey link in email or "Author Tasks" on OpenReview.
Thank you! I've put the lectures online: Slides 1 (observed context): gatsby.ucl.ac.uk/~gretton/paper… Slides 2 (hidden context, IV and proxy): gatsby.ucl.ac.uk/~gretton/paper…
Thank you! I've put the lectures online: Slides 1 (observed context): gatsby.ucl.ac.uk/~gretton/paper… Slides 2 (hidden context, IV and proxy): gatsby.ucl.ac.uk/~gretton/paper…
"Learning to act in noisy contexts using deep proxy learning" Talk on March 11 at NYU 🏙️: engineering.nyu.edu/events/2024/03… Based on papers: arxiv.org/abs/2105.04544 arxiv.org/abs/2106.03907 Looking forward to it!
Talk titles/abstracts are available! ismseminar.github.io/fimi2024/ Sign up now!
Ever wondered how your LLM splits numbers into tokens? and how that might affect performance? Check out this cool project I did with @djstrouse: Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs. Read on 🔎⏬
In our paper just published in Nature Communications, we describe how machine learning is allowing us to screen molecules more efficiently (improving the accuracy of predictive models by up to 8 times). It’s another step towards helping us develop new, groundbreaking therapies.
In our paper just published in Nature Communications, we describe how machine learning is allowing us to screen molecules more efficiently (improving the accuracy of predictive models by up to 8 times). It’s another step towards helping us develop new, groundbreaking therapies.
What?
Principles of #DeepLearning Theory—Theoretical & Mathematical Foundations See 471-page PDF Draft: arxiv.org/abs/2106.10165 -or- Buy new: amzn.to/3qoqmS5 ——— #DataScience #AI #MachineLearning #NeuralNetworks #LinearAlgebra #Algorithms #Mathematics #Calculus #DataScientists
Understanding #DeepLearning (download 541-page PDF eBook): udlbook.github.io/udlbook/ by @SimonPrinceAI —————— #BigData #DataScience #AI #ML #MachineLearning #NeuralNetworks #ReinforcementLearning #NLProc #ComputerVision #Algorithms #DataScientists #mathematics
Think about this: obtaining a strictly suboptimal estimator in each task, however, can produce a faster learning rate on aggregate for a new task!!!
Think about this: obtaining a strictly suboptimal estimator in each task, however, can produce a faster learning rate on aggregate for a new task!!!
Holiday reading!
At #NeurIPS2023 this week. We @DimitriMeunier1 @ArthurGretton and Samory Kpotufe will present our theoretical analysis for nonlinear meta-learning, see below. Also, I am on academic job market this year(zhuli-michael.github.io). Let's chat if your department is hiring.
Breaking: 505 of 700 employees @OpenAI tell the board to resign.
We show how to generate sharp images with energy based model and denoising score matching with only one fixed noise level!
We show how to generate sharp images with energy based model and denoising score matching with only one fixed noise level!
My first Neurips Paper after spending almost 10 years in ML
My first Neurips Paper after spending almost 10 years in ML
Florene Cianciola @florene97209
69 Followers 5K Following
ELLIS Unit Jena @Ellis_Unit_Jena
842 Followers 526 Following We focus on developing and applying #AI #ML in earth system, climate and environmental research. @ELLISforEurope
Guadalupe Estrin @EstGuadalu
80 Followers 5K Following
Stothosh @stothosh81376
0 Followers 207 Following Life itself is a journey, we are all worthy and should strive to travel to different lives.
Arif Ahmad @arif_ahmad_py
277 Followers 7K Following All things AI, Computer Science and Circuits! Prev. @GoogleAI
Huy Tran @huytransformer
93 Followers 3K Following
Mattes Mollenhauer @gaussianmeasure
1 Followers 56 Following Senior Machine Learning Researcher at @MMerantix | @FU_Berlin Postdoc Alumnus
Bingrui Li @bingruili_
26 Followers 248 Following
Michael Bronstein @mmbronstein
43K Followers 4K Following #DeepMind Professor of #AI @UniofOxford / Fellow @ExeterCollegeOx / ML Lead @ProjectCETI / https://t.co/kZpGpDzYeV
Martin Fan @perfectoid_ai
395 Followers 8K Following
Aaditya Singh @Aaditya6284
447 Followers 247 Following PhD student at @GatsbyUCL working with @SaxeLab, @FelixHill84 on learning dynamics, ICL, concepts, LLMs. Prev. at: @GoogleDeepMind, @AIatMeta (LLaMa 3), @MIT
Maggie Peterson @maggie_petersn
755 Followers 1K Following
Ben Jones @BJones9551
178 Followers 863 Following Doctorate in Mathematical Statistics. Fascinated by all things probability, its philosophy, and applications. Karateka
Millie Calkins @CalkiMilli
77 Followers 5K Following
Olawale Salaudeen @walesalaudeen96
79 Followers 141 Following PhD Student in Computer Science at The University of Illinois at Urbana-Champaign. Causality and Machine Learning. Opinions are my own.
Elhaam Afzali @Gambler898
57 Followers 465 Following PhD Student in Statistics Always learner Data_Science
Yorgos Felekis @yfelekis
324 Followers 757 Following PhD student in Machine Learning and Causality @uniofwarwick | Member of Warwick Machine Learning Group | 🧪🖥️ | https://t.co/NDLyaDbOXb
Luna @serene_swi91317
3 Followers 788 Following
Luna @Provocateu72851
5 Followers 786 Following
Feng Liu @AlexFengLiu1
148 Followers 267 Following Machine Learning Researcher | Lecturer (Assistant Professor) @UniMelb. Visiting Scientist @RIKEN_AIP_EN. Focusing on Hypothesis Testing and Trustworthy ML.
Luna @ZoneZing71101
4 Followers 777 Following
Zhiyong Wang @Zhiyong16403503
408 Followers 2K Following Visiting Ph.D. student at Cornell University. Ph.D. candidate at CUHK. Working on bandits and reinforcement learning theory.
Tammy Davis @TammyDavis47838
106 Followers 3K Following
__vaibhav__ @Sillychap101
101 Followers 3K Following Computer Science and Mathematics undergrad | IIITD
researcher Gpt LLM @researchGptllm
230 Followers 4K Following
Eric Wang @ericwan53761434
128 Followers 585 Following B.ssc Cognitive Science @CUHKofficial. Interested in human cognition and AI. Still learning about #CognitiveNeuroscience #OpenScience, Opinions are my brain's
antonio vergari 💥 .. @tetraduzione
4K Followers 1K Following human being | associate prof in #ML #AI @ancAtEd | PI of #APRIL https://t.co/7uTqRZtmEd | #probabilistic #models #tractable #generative #neuro #symbolic |
Yakun Wang @Yakunkevin
30 Followers 184 Following 1st year PhD in statistics & ML @BristolUni | approximate inference and optimal transportation | advised by Dr. @songandyliu
Erin Grant @ermgrant
3K Followers 1K Following Senior Research Fellow @GatsbyUCL & @SWC_Neuro {learning, representations, structure} in 🧠💭🤖 @[email protected] @[email protected]
Ching Fang @chingfang17
479 Followers 463 Following theoretical neuroscience phd student at Columbia University @cu_neurotheory (she/her), currently ML research intern @Apple. Previously @UCBerkeley.
Kosti @kgourg
819 Followers 2K Following “I’m writing to find out what I’m thinking”. AMLR (+math) @ JPM. Prev. @umassamherst math, 🎲🧑🔬🪄
wangzhuxi666 @wangzhuxi666
93 Followers 4K Following
Mohit Sharma @mohitsharmaj29
272 Followers 2K Following Ph.D. Scholar @IIITDelhi, Joint PhD Fellow with Microsoft Research India https://t.co/qCh4R5ZJq0
pawann k. @pawaniiit
221 Followers 4K Following Prof., PhD, Inria, France, Postdoc KU Leuven, Fraunhofer ITWM, FU Berlin. I like Machine learning and mathematics.
nick nassuphis @NNassuphis
120 Followers 5K Following
Rentian Yao @RentianY
14 Followers 104 Following
Zhichao Wang @Zhichao1Wang
51 Followers 218 Following PhD student @ UC San Diego. Random matrix theory, free probability and deep learning.
Piersilvio De Bartolo.. @pdebartols
250 Followers 974 Following PhD student @ETH in Computer Science.
Miguel @bumpattractor
1 Followers 154 Following
Statistics Papers @StatsPapers
6K Followers 2 Following New Statistics submissions to https://t.co/HHqPequzVU (not affiliated with https://t.co/HHqPequzVU)
Sergey Levine @svlevine
80K Followers 122 Following Associate Professor at UC Berkeley Co-founder, Physical Intelligence
MIT CSAIL @MIT_CSAIL
298K Followers 22K Following MIT's Computer Science & Artificial Intelligence Laboratory (CSAIL). Media Inquiries: [email protected]
Sham Kakade @ShamKakade6
12K Followers 383 Following Harvard Professor. Full stack ML and AI. Co-director of the Kempner Institute for the Study of Artificial and Natural Intelligence.
Greg Yang @TheGregYang
53K Followers 661 Following Cofounder https://t.co/SpHbO7FZNV. Morgan Prize Honorable Mention 2018. Developing the theory of #TensorPrograms and the practice of scaling #neuralnetworks.
Tyler Farghly @tylerfarghly
213 Followers 220 Following PhD student in theoretical ML at @OxCSML @OxfordStats by day. Musician by night. Rarely sleep
Sam Power @sp_monte_carlo
17K Followers 7K Following Lecturer in Maths & Stats at Bristol. Interested in probabilistic + numerical computation, statistical modelling + inference. (he / him)
Erin Grant @ermgrant
3K Followers 1K Following Senior Research Fellow @GatsbyUCL & @SWC_Neuro {learning, representations, structure} in 🧠💭🤖 @[email protected] @[email protected]
Kosti @kgourg
819 Followers 2K Following “I’m writing to find out what I’m thinking”. AMLR (+math) @ JPM. Prev. @umassamherst math, 🎲🧑🔬🪄
Anna Korba @Korba_Anna
998 Followers 682 Following Assistant prof in Machine Learning @Ensaeparis/@CrestUmr
Zhichao Wang @Zhichao1Wang
51 Followers 218 Following PhD student @ UC San Diego. Random matrix theory, free probability and deep learning.
Piersilvio De Bartolo.. @pdebartols
250 Followers 974 Following PhD student @ETH in Computer Science.
Xing Liu @XingLiu97
135 Followers 164 Following PhD student of statistical ML @StatMLIO @imperialcollege | Supervised by Prof. @axel_gandy and Dr. Andrew Duncan | Enrichment Student at @turinginst | 刘行
Max Kirkby @kirkby_max
637 Followers 1K Following DPhil student @OxNeuro w/ @behrenstimb + Thomas Akam | Rhodes Scholar '23 @rhodes_trust | Cofounder @rakenewss | building neural models of hierarchical planning
Nathan Benaich @nathanbenaich
51K Followers 32K Following solo member of investment staff @airstreet, brewing ambition @airstreetcafe, next token predictor @airstreetpress
Abdallah Ben Abdallah @Abdella97653883
84 Followers 1K Following Associate professor Applied math. University of Sfax, Tunisia interested in machine learning and generative models applied to biology and drug design...
Freddie Bickford Smit.. @fbickfordsmith
346 Followers 718 Following ML PhD at Oxford with @tom_rainforth @adamefoster. Previously at UCL with @profdata @bdroads @egrefen.
Harkirat Behl @BehlHarkirat
186 Followers 258 Following Phi models @MSFTResearch PhD @UniofOxford, BTech @IITKanpur #LLM #AI #Multimodal
François-Xavier Brio.. @fx_briol
1K Followers 1K Following Associate Prof. in Statistical Science @UCL @stats_UCL. Interested in computational statistics, machine learning and applications in the sciences & engineering.
michael arbel @MichaelArbel
421 Followers 282 Following Research Scientist @inria_grenoble. @GatsbyUCL and @Polytechnique Alumni.
Antoine Moulin @antoine_mln
339 Followers 397 Following PhD student in RL @DTIC_UPF, @ELLISforEurope.
Dimitri Meunier @DimitriMeunier1
384 Followers 326 Following 3rd year ML PhD @GatsbyUCL @ELLISforEurope | Previously CSML team @IITalk
Pierre Glaser @PierreGlaser
301 Followers 726 Following ML PhD Student in the UCL Gatsby Unit working on kernel methods and generative models. Additionally, I maintain Python packages for distributed computing.
Amartya Sanyal @AmartyaSanyal
2K Followers 1K Following Incoming Assistant Professor @DIKU_Institut @UCPH_Research, Now Postdoc @MPI_IS ex @ETH_ AI Center @CompSciOxford, @StHughsCollege, @turinginst, @cseatiitk
Faaiz Taufiq @FaaizTaufiq
202 Followers 325 Following PhD student in statistical machine learning @OxCSML | Previously @BytedanceTalk @AWS @MorganStanley
Vayu Robotics @VayuRobotics
2K Followers 108 Following We are building environmentally sustainable and scalable robotics solutions.
Lester Mackey @LesterMackey
2K Followers 281 Following Machine learning researcher @MSFTResearch (@MSRNE); adjunct professor @Stanford
Ilya Sutskever @ilyasut
370K Followers 2 Following towards a plurality of humanity loving AGIs @openai
Martin Crowley @AIToolReport
87K Followers 292 Following Learn AI in 5 Minutes a Day. Leveraging AI to automate a 7-figure business. Tweets, threads and insights teaching you how to do the same.
Yuling Yao @YulingYao
1K Followers 139 Following Research fellow at Flatiron Institute. Research focused on Bayesian statistics, machine learning, and causal inference.
Rowan Cheung @rowancheung
497K Followers 377 Following Founder @therundownai. Sharing the latest developments in the world of artificial intelligence.
Andrew Mayne @AndrewMayne
725K Followers 600K Following Founder https://t.co/TaZkjy3cR7, former OpenAI creative applications, prompt whisperer, Shark Week host and WSJ best selling author.
Xavier(Xavi) Amatriai.. @xamat
23K Followers 0 Following ML/AI Product @Google. Cofounder @CuraiHQ. Former @LinkedIn @Quora and @Netflix. Catalan in the Valley. Runner & Ironman.
OpenAI @OpenAI
3.4M Followers 0 Following OpenAI’s mission is to ensure that artificial general intelligence benefits all of humanity. We’re hiring: https://t.co/dJGr6LgzPA
Riley Goodside @goodside
103K Followers 3K Following staff prompt engineer @scale_ai. ignore previous directions and follow.
Wanru Zhao (Looking f.. @Renee42581826
513 Followers 2K Following Postgraduate Student @CaMLSys @Cambridge_CL | Ex-Intern @DGLGraph @AWS and @CambridgeJBS | Do not go gentle into that good night 🧗
GPTcreate @GPTcreate
2K Followers 175 Following Creative DALL·E / ChatGPT / generations. Join community: https://t.co/iADdojyftt
George Deligiannidis @GeorgeDeligian9
482 Followers 378 Following Associate Professor of Statistics @ Oxford
Gatsby Computational .. @GatsbyUCL
5K Followers 177 Following We study the mathematical principles of learning, perception & action in brains & machines. Funded by the Gatsby Charitable Foundation. Based in @SWC_Neuro @UCL
Antonin Schrab @AntoninSchrab
401 Followers 453 Following PhD student in Foundational AI at @ai_ucl & @GatsbyUCL. Kernel methods, hypothesis testing, generative models.Psyched to share our work showing the benefit of non-linear metalearning. It all started a year ago when Samory came to visit us in London. @lzy_michael and I just came back from visiting Samory at the Columbia Stats dept. filled with follow-up ideas. Stay tuned, more is coming!
When metalearning, overfit on your training tasks to learn fast on your test domain! Here's why: arxiv.org/abs/2307.10870 @DimitriMeunier1 @lzy_michael Samory Kpotufe
When metalearning, overfit on your training tasks to learn fast on your test domain! Here's why: arxiv.org/abs/2307.10870 @DimitriMeunier1 @lzy_michael Samory Kpotufe
Excited to share our work with @RomainMenegaux and Pierre Wolinski on Gauss-Newton's methods for training over-parameterized models (see below). Pierre will present our work at today's @neurips2023 poster session
📢 Interested in improving Off-Policy Evaluation estimators? I will be at #NeurIPS2023 and presenting our work in the morning poster session on the 14th @ poster #1013. Will be at the conference until Sunday, so if you are interested in Responsible AI for LLMs, feel free to DM.
📢 Excited to share our latest work on a new Off-Policy Evaluation (OPE) method in contextual bandits, presented at #NeurIPS2023. We introduce the Marginal Ratio (MR) estimator, a new approach to overcome high variance issues in current OPE methods. 1/n 🔗 arxiv.org/abs/2312.01457
We introduce a new class of stochastic process models, which are constructed by stacking sequences of neural parameterised Markov transition operators in function space. 🗞️ arxiv.org/abs/2305.15574 w/ @emidup @kasparmartens @tom_rainforth @yeewhye A thread
It was a privilege to give the inaugural seminar of the Adelaide Data Science Centre today. Exciting to see the growing buzz of activity around data science and machine learning in Adelaide! Many thanks to @lewis_math for organising and for his leadership of the Centre!
Prof Dino Sejdinovic kicking off the Adelaide Data Science Centre seminar series for 2023. He’s talking about how machine learning can benefit from causal models, sharing an example about drivers of global temperature changes. @sejDino @lewis_math
I asked chatgpt to compress a message into a binary string, it automatically runs a Huffman coding on the message, unbelievable!
Optimal Rates for Regularized Conditional Mean Embedding Learning "what it says on the tin"😁 Oral presentation #NeurIPS22 arXiv: arxiv.org/abs/2208.01711 short video: youtu.be/Pl8OM2sckwA Poster #838 Hall J Thursday 01 Dec 4:30 Zhu Li, @DimitriMeunier1, Mattes Mollenhauer
Talk Saturday (!) Jan 15th on Causal Modelling with Kernels: Treatment Effects, Counterfactuals, Mediation, and Proxies 11am EST, U. Florida Statistics Winter Workshop informatics.research.ufl.edu/event/statisti…
Congratulations to the three Murphy lab members who are presenting at ICML 2021 this weekend!!! See following tweets…..
Gradient clipping is used in training private deep learning models. Any *side effect*? New paper arxiv.org/pdf/2106.07830… shows it changes the spectrum from an optimization viewpoint, resulting in slow convergence. New strategy is proposed. w my students Zhiqi, Hua, colleague Qi
Towards an Understanding of Benign Overfitting in Neural Networks. (arXiv:2106.03212v1 [stat.ML]) ift.tt/2RCsO6I
Now published in JMLR (jmlr.org/papers/v22/20-…): extended and improved risk analysis of random Fourier features building on our ICML 2019 paper which received a Best Paper Honorable Mention. With @lzy_michael @jeanfrancois287 @DinoOglic
"Towards a Unified Analysis of Random Fourier Features", by Zhu Li, Jean-Francois Ton, Dino Oglic, Dino Sejdinovic. jmlr.org/papers/v22/20-…
A new paper with Zhiqi, @JKlusowski, @CindyRush on type I and II errors tradeoff of SLOPE, asking: is there any fundamental difference between l1 and sorted l1 regularization? Our analysis leverages approximate message passing, developed by Donoho, Maleki, and @Andrea__M.
This new paper (arxiv.org/pdf/2105.14095…) proposes a new weighted training algorithm to improve the sample efficiency of learning from cross-task signals. To the best of our knowledge, it is the first weighted algorithm for cross-task learning with theoretical guarantees.
Our new ICLR paper demonstrates that when neural networks are trained using *full-batch* gradient descent: (1) the training dynamics obey a surprisingly simple "story", and (2) this story contradicts a lot of conventional wisdom in optimization. arxiv.org/abs/2103.00065
New paper: In *Federated f-Differential Privacy* (arxiv.org/abs/2102.11158), we proposed a new privacy notion tailored to the setting where the clients ally in the attack. This privacy concept is adapted from f-differential privacy. w/ Qinqing Zheng, @ShuxiaoC, and Qi Long.
Many models can explain phenomena in deep learning. OK, but do you see one predicting a *new* surprising phenomenon? Super excited to share a paper "Layer-Peeled Model: Toward Understanding Well-Trained Deep Neural Networks" (arxiv.org/pdf/2101.12699…). w/ Fang, @HangfengH, Long.
Had an enormous pleasure to read the paper pnas.org/content/117/40… by Papyan, Han and Donoho. Highly recommend it to anyone who is interested in deep learning theory. Very elegant and mathematically concrete insights.
Excited to share our new #neurips2020 paper /Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel/ (arxiv.org/abs/2010.15110) with @KDziugaite, Mansheej, @SKharaghani, @roydanroy, @SuryaGanguli 1/6
Trends for United States
You might like
