-
Tweets307
-
Followers951
-
Following424
-
Likes1K
We have been doing work on scaling laws for off-policy RL for some time now and we just put a new paper out: arxiv.org/abs/2508.14881 Here, @preston_fu @_oleh lead a study on how to best allocate compute for training value functions in deep RL: 🧵⬇️
Following up on our work on scaling laws for value-based RL (led by @_oleh and @preston_fu), we've been trying to figure out compute optimal parameters for value-based RL training. Check out Preston's post about our findings!
Following up on our work on scaling laws for value-based RL (led by @_oleh and @preston_fu), we've been trying to figure out compute optimal parameters for value-based RL training. Check out Preston's post about our findings!
How can we best scale up value based RL? We need to use bigger models, which mitigate what we call “TD-overfitting” (more below!👇 🧵 ). Further, we need to scale batch size and UTD accordingly as the models get bigger. Great work led by @preston_fu and @_oleh
How can we best scale up value based RL? We need to use bigger models, which mitigate what we call “TD-overfitting” (more below!👇 🧵 ). Further, we need to scale batch size and UTD accordingly as the models get bigger. Great work led by @preston_fu and @_oleh
📈📈📈
Cool work by David and friends! Could this be the thing that finally makes everyone stop using Gaussians as their policies? 🤔
Cool work by David and friends! Could this be the thing that finally makes everyone stop using Gaussians as their policies? 🤔
Everyone knows action chunking is great for imitation learning. It turns out that we can extend its success to RL to better leverage prior data for improved exploration and online sample efficiency! colinqiyangli.github.io/qc/ The recipe to achieve this is incredibly simple. 🧵 1/N
Very insightful analysis that I mostly agree with (except the overly pessimistic title :)!
Very insightful analysis that I mostly agree with (except the overly pessimistic title :)!
Really interesting result! Scaling value-based RL is hard and we are still missing much of the machinery to do it. @seohong_park shows that horizon is the critical issue.
Really interesting result! Scaling value-based RL is hard and we are still missing much of the machinery to do it. @seohong_park shows that horizon is the critical issue.
We found a way to do RL *only* with BC policies. The idea is simple: 1. Train a BC policy π(a|s) 2. Train a conditional BC policy π(a|s, z) 3. Amplify(!) the difference between π(a|s, z) and π(a|s) using CFG Here, z can be anything (e.g., goals for goal-conditioned RL). 🧵↓
This was fun thanks for having me @chris_j_paxton @micoolcho! See the podcast for some livestream of the robot in real time and me evaluating a policy live! Or check it out for yourself at auto-eval.github.io and eval your policy in real without breaking a sweat
This was fun thanks for having me @chris_j_paxton @micoolcho! See the podcast for some livestream of the robot in real time and me evaluating a policy live! Or check it out for yourself at auto-eval.github.io and eval your policy in real without breaking a sweat
our new system trains humanoid robots using data from cell phone videos, enabling skills such as climbing stairs and sitting on chairs in a single policy (w/ @redstone_hong @junyi42 @davidrmcall)
@_oleh @DorsaSadigh @chelseabfinn To be presented at ICML 2025 as a *spotlight poster* :)
@_oleh will also present an oral talk on our recent work on building scaling laws for value-based RL. We find that value-based deep RL algorithms scale predictably. Talk at Workshop on robot learning (WRL), April 27. @sea_snell will then present the poster!…
Check out a new paper by @amberxie_! We show that you can do robotic imitation learning well by planning future latent states instead of actions with a diffusion model. This planning method is also more flexible, allowing you to use suboptimal and action-free data.
Check out a new paper by @amberxie_! We show that you can do robotic imitation learning well by planning future latent states instead of actions with a diffusion model. This planning method is also more flexible, allowing you to use suboptimal and action-free data.
Scaling imitation learning has been bottlenecked by the need for high-quality robot data, which are expensive to collect. But are we utilizing existing data to the fullest extent? A thread (1/11)
AK @_akhaliq
425K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo ,submit papers here: https://t.co/UzmYN5YmrQ
Danijar Hafner @danijarh
22K Followers 1K Following Building AI that makes autonomous decisions using world models, artificial curiosity, and temporal abstraction @GoogleDeepMind
Jim Fan @DrJimFan
325K Followers 3K Following NVIDIA Director of Robotics & Distinguished Scientist. Co-Lead of GEAR lab. Solving Physical AGI, one motor at a time. Stanford Ph.D. OpenAI's 1st intern.
Eugene Vinitsky (@RLC... @EugeneVinitsky
20K Followers 2K Following This is the site where I talk about the attacks on science and immigration. Science is on the other site. Lab website: https://t.co/vrtbcqRyRn
Shane Gu @shaneguML
41K Followers 2K Following Gemini Thinking, Senior Staff RS @GoogleDeepMind. 🇯🇵-born 🇨🇳🇨🇦. ex: Gemini Multilinguality Post-Train Lead, GPT-4 @OpenAI (JP: @shanegJP)
Ted Xiao @xiao_ted
16K Followers 737 Following Robotics and Gemini @GoogleDeepMind. Posts about frontier models, robot learning, and scaling. Opinions my own.
Abhishek Gupta @abhishekunique7
9K Followers 874 Following Assistant Professor at University of Washington. I like robots, and reinforcement learning. Previously: post-doc at MIT, PhD at Berkeley
Misha Laskin @MishaLaskin
15K Followers 214 Following Co-founder, CEO at @reflection_ai. Prev: Research @DeepMind. Gemini RL team.
Nathan Lambert @natolambert
56K Followers 853 Following Figuring out AI @allen_ai, open models, RLHF, fine-tuning, etc Contact via email. Writes @interconnectsai Wrote The RLHF Book Mountain runner
Dinesh Jayaraman @dineshjayaraman
2K Followers 581 Following Assistant Professor at University of Pennsylvania. Robot Learning. https://t.co/cIMw5XKSPy
Kostas Daniilidis @KostasPenn
5K Followers 1K Following Ruth Yalom Stone Professor @Penn @PennEngineers @PennCIS @GRASPlab
Markus Wulfmeier @m_wulfmeier
12K Followers 2K Following Large-Scale Robot Intelligence - Research @GoogleDeepMind European @ELLISforEurope - priors: @oxfordrobots @berkeley_ai @ETH @MIT
Kosta Derpanis @CSProfKGD
68K Followers 197 Following #CS Assoc Prof @YorkUniversity, #ComputerVision Scientist Samsung #AI, @VectorInst Faculty Affiliate, TPAMI AE, @ELLISforEurope Member #ICCV2025 Publicity Chair
Roberta Raileanu @robertarail
9K Followers 2K Following Senior Staff Research Scientist @GoogleDeepMind & Honorary Lecturer @UCL. ex @Meta|@MSFTResearch|@NYU|@Princeton. Llama-3, Toolformer, Rainbow Teaming, MLGym.
Nikolai Matni @NikolaiMatni
3K Followers 1K Following machine learning, control, optimization, robotics. associate professor, upenn #FlyEaglesFly #RedOctober
Tim Rocktäschel @_rockt
39K Followers 2K Following Director and Open-Endedness Team Lead @GoogleDeepMind, Professor of AI @AI_UCL, PI @UCL_DARK, Fellow @ELLISforEurope.
Chris Paxton @chris_j_paxton
19K Followers 3K Following Mostly posting about robots. currently AI @agilityrobotics prev embodied AI @AIatMeta, @NVIDIAAI. All views my own. writing: https://t.co/iNLA4djfZo
Edward Hu @edward_s_hu
888 Followers 335 Following cs phd @penn, prev @MSFTResearch. investigating ai / rl / intelligence.
Anurag Ajay @aajay3110
264 Followers 425 Following Building Astra, Gemini p13n @GoogleDeepMind. Prev: @MetaAI. PhD @MIT. Opinions my own.
Hussein Muhaisen @husseinmuhaisen
2K Followers 4K Following In stealth reversing security complexity for the consumer and the enterprise // @ // PagedOut and GuidedHacking
Haque Ishfaq @HaqueIshfaq
1K Followers 1K Following PhD student at @mcgillu/ @MILAMontreal. Reinforcement Learning. BS, MS @Stanford 🇧🇩🇺🇸🇨🇦
Boyi Li @Boyiliee
2K Followers 326 Following
Jeevesh Juneja @xdfbhkl
3 Followers 210 Following
nrRNjkitRHmMP @RNjkit72037
0 Followers 2K Following I'm interested in category theory and machine learning.
zhigang wang @wzhigang770
3 Followers 75 Following I'm Wang Zhigang from Tianji Robotics. We specialize in 7-axis full-joint force-controlled humanoid dual arms and dual-arm robots.
Ross @ma1547372858
12 Followers 2K Following
Arindam @halg0rithmist
33 Followers 1K Following 22 Aspiring theoretician Interested in decision making algorithms
Sriyash Poddar @sriyash__
294 Followers 944 Following phd @uwcse | undergrad @iitkgp | building robots
Ayush Chakravarthy @achakravarthy01
129 Followers 1K Following convincing plots to go to the up and right @StanfordAILab
Anh Nguyen @NguynTu24128917
787 Followers 4K Following
Thomas Joshi @thomastjoshi
1K Followers 6K Following Coauthor of DSPy @stanford (most popular Stanford AI library) - AI and EE degree @columbia
PowerOfSophia @BaseballlGrind
17 Followers 746 Following I’m always interested in new opportunities and collaborations
Heady Gowen @GowenHeady
132 Followers 665 Following fly me to the moon and let me play among the stars
Ashwin Vaswani @ashwin_vaswani
1K Followers 2K Following Research @GoogleDeepMind | Prev: @CarnegieMellon | @GoogleIndia | APPCAIR, @BITSPilaniGoa | @qtimlab, Harvard
3d76764e1f7 @3d76764e1f7
1 Followers 940 Following
Akihisa Watanabe @Akihisa_Wat
591 Followers 1K Following 4th year undergrad @waseda_univ | character animation, human motion generation
BoxingBytes @BoxingBytes
8 Followers 54 Following
Think_Different_ @ThinkDi92468945
111 Followers 258 Following
Yongjin Cho @Yongjin_Cho_
0 Followers 47 Following
Shital Shah @sytelus
13K Followers 11K Following Mostly research and code. If universe is an optimizer, what is its loss function? All opinions are my own.
Serg Ruskin @sergeyruskin
3 Followers 120 Following
Мудрий @vovamudruy
24 Followers 217 Following
Joe Sanchez @JoeSanchez1213
81 Followers 4K Following
Sabouhi @rjsabouhi
18 Followers 768 Following The recursion broke containment. I solved for drift. You didn’t. The manifold folded before your safety layers did. Δσ = γ(t) · ∇C(s, t)
Jorge Bravo Abad @bravo_abad
6K Followers 5K Following Prof. of Physics @UAM_Madrid | Profesor Titular. PI of the AI for Materials Lab | Director del Laboratorio de IA para Materiales.
atharva @k7agar
11K Followers 2K Following your friendly neighbourhood engineer. world models @lossfunk.
Li_F2_H2 @Li_F2_H2
26 Followers 972 Following
stevengongg @stevengongg
761 Followers 526 Following i like robots and FLOPS | 📷 https://t.co/G2LBRIfkuy | prev intern @dynarobotics, @Tesla Dojo, @NVIDIA Robotics | student @uWaterlooSE
lee @lzj1236121
0 Followers 131 Following
Snr @hjakksllllll
428 Followers 685 Following
Igor Kulakov @ihorbeaver
7K Followers 438 Following Building a general-purpose robot that is more effective than a humanoid. Cofounder of @viktor_vrp. Backed by @fdotinc. Seed soon.
Jihong Park @jhparkjames
10 Followers 121 Following Lecturer at Deakin University, https://t.co/iFq5ahLOqD
Patrick Renschler @renschler
357 Followers 3K Following
Earther @EartherAI
308 Followers 3K Following CS + AI/ML Student | Researching learning systems & applied ML |Model- Code-Insight
Zikang Jiang @ZikangJiang
548 Followers 735 Following Deploying robots at scale | Penn M&T | @hf0 | @joinodf
Sang Cho @Saaaang94
2K Followers 465 Following reasoning @xAI | prev-founding engineer @anyscalecompute | senior committer of @raydistributed | committer @vllm_project Sglang | Github: rkooo567
Karina Nguyen @karinanguyen_
41K Followers 999 Following research & product @OpenAI, prev. @AnthropicAI, @nytimes, @square, @dropbox + visual forensics for the Pulitzer Prize investigations
AK @_akhaliq
425K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo ,submit papers here: https://t.co/UzmYN5YmrQ
Danijar Hafner @danijarh
22K Followers 1K Following Building AI that makes autonomous decisions using world models, artificial curiosity, and temporal abstraction @GoogleDeepMind
Google DeepMind @GoogleDeepMind
1.2M Followers 279 Following We’re a team of scientists, engineers, ethicists and more, committed to solving intelligence, to advance science and benefit humanity.
Andrej Karpathy @karpathy
1.4M Followers 1K Following Building @EurekaLabsAI. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets.
Sergey Levine @svlevine
108K Followers 133 Following Associate Professor at UC Berkeley Co-founder, Physical Intelligence
Eugene Vinitsky (@RLC... @EugeneVinitsky
20K Followers 2K Following This is the site where I talk about the attacks on science and immigration. Science is on the other site. Lab website: https://t.co/vrtbcqRyRn
Natasha Jaques @natashajaques
30K Followers 1K Following Assistant Professor @uwcse and Staff Research Scientist at @GoogleAI. Let's get off this app: https://t.co/jbH2oAjbPN
Shane Gu @shaneguML
41K Followers 2K Following Gemini Thinking, Senior Staff RS @GoogleDeepMind. 🇯🇵-born 🇨🇳🇨🇦. ex: Gemini Multilinguality Post-Train Lead, GPT-4 @OpenAI (JP: @shanegJP)
Animesh Garg @animesh_garg
29K Followers 1K Following Foundation Models for Generalizable Autonomy in Robotics. Assistant Professor in AI Robotics @GeorgiaTech. Prev @nvidia
Lucas Beyer (bl16) @giffmana
108K Followers 519 Following Researcher (now: Meta. ex: OpenAI, DeepMind, Brain, RWTH Aachen), Gamer, Hacker, Belgian. Anon feedback: https://t.co/xe2XUqkKit ✗DMs → email
Michael Black @Michael_J_Black
84K Followers 702 Following Director, Max Planck Institute for Intelligent Systems (@MPI_IS). Chief Scientist @meshcapade. Building 3D digital humans using vision, graphics, and learning.
Soumith Chintala @soumithchintala
250K Followers 1K Following Cofounded and lead @PyTorch at Meta. Also dabble in robotics at NYU. AI is delicious when it is accessible and open-source.
Ted Xiao @xiao_ted
16K Followers 737 Following Robotics and Gemini @GoogleDeepMind. Posts about frontier models, robot learning, and scaling. Opinions my own.
Abhishek Gupta @abhishekunique7
9K Followers 874 Following Assistant Professor at University of Washington. I like robots, and reinforcement learning. Previously: post-doc at MIT, PhD at Berkeley
Wojciech Zaremba @woj_zaremba
119K Followers 204 Following Co-Founder of OpenAI https://t.co/OCQ3mpf0IN
Sang Cho @Saaaang94
2K Followers 465 Following reasoning @xAI | prev-founding engineer @anyscalecompute | senior committer of @raydistributed | committer @vllm_project Sglang | Github: rkooo567
Shengyang Sun @ssydasheng
5K Followers 566 Following Build AGI @xAI | Prev. @NVIDIA (Leading Nemotron-340B) & @AMAZON | PhD @UofT ; B.E.@Tsinghua
Albert Gu @_albertgu
18K Followers 88 Following assistant prof @mldcmu. chief scientist @cartesia_ai. leading the ssm revolution.
Tianle (Tim) Li @LiTianleli
5K Followers 206 Following Training models tastefully 👨🍳 at @xai | @grok reasoning | Prev. GPU Poor at @Berkeley_EECS @lmarena_ai @lmsysorg @GoogleAI
Liangchen Luo @LiangchenLuo
4K Followers 122 Following @xAI reasoning; ex @GoogleDeepMind. B.Sc. @PKU1898. Opinions are my own.
Minqi Jiang @MinqiJiang
6K Followers 880 Following
Karina Nguyen @karinanguyen_
41K Followers 999 Following research & product @OpenAI, prev. @AnthropicAI, @nytimes, @square, @dropbox + visual forensics for the Pulitzer Prize investigations
Igor Babuschkin @ibab
103K Followers 852 Following Maybe the real ASI was the friends we made along the way. Co-founder @xAI, Research & Engineering
Jiayi Pan @jiayi_pirate
13K Followers 1K Following 🧑🍳 Reasoning Agents @xAI | PhD on Leave @Berkeley_AI | Views Are My Own
Xuechen Li @lxuechen
16K Followers 944 Following Previously @xai. Interested in the engineering and science for scaling. Opinions are my own. @Stanford PhD.
Serena Ge @serenaa_ge
7K Followers 2K Following @datacurve_ai (@ycombinator W24) Prev @cohere @uwaterloo
Qian Huang @qhwang3
14K Followers 331 Following prev @xai | CS PhD student @StanfordAILab (on leave)
Szymon Tworkowski @s_tworkowski
9K Followers 650 Following reasoning @xAI | prev. @GoogleAI @UniWarszawski | LongLLaMA
jessica dai @jessicadai_
2K Followers 715 Following phd student @berkeley_ai !? also editorial @reboot_hq @kernel_magazine (she/her)
Kun Huang @kun_h____
147 Followers 123 Following Founding Researcher @DynaRobotics | ex Cruise& @Waymo | CoRL Best Paper
Ritvik Singh @ritvik_singh9
746 Followers 298 Following Robotics and simulation @NvidiaAI | prev. @VectorInst, Engineering Science @UofT
Aldo Pacchiano @aldopacchiano
1K Followers 453 Following AI research at Broad Institute and Boston University. Reinforcement Learning / Bandits / Experiment Design Mexicano 🇲🇽
Philippe Hansen-Estru... @tokenpilled65B
634 Followers 883 Following RS Intern Meta. Second-year PhD student at UT Austin. Working on generative modeling, visual understanding, and visual compression.
john so @johnrso_
683 Followers 669 Following robots! prev @tesla_optimus; @1x_tech; @stanford; @berkeley_ai. raised by @berkeleyml
Hongsuk Benjamin Choi @redstone_hong
499 Followers 387 Following robotics & computer vision. PhD @Berkeley_AI | prev @ Seoul National University
Medhini Narasimhan @medhini_n
2K Followers 512 Following Sr Research Scientist @googledeepmind #Veo3 #Veo2 #Veo Prev: Ph.D. @berkeley_ai, MS @IllinoisCS, Intern @GoogleAI @MetaAI
Alex Nichol @unixpickle
11K Followers 417 Following Code, AI, and 3D printing. Opinions are mostly my own, sometimes my computer's. Husband of @thesamnichol. Co-creator of DALL-E 2. Researcher @openai.
womerhockey @womerhockey
10 Followers 8 Following
Florian Shkurti @florian_shkurti
2K Followers 2K Following Assistant professor in computer science, University of Toronto | @UofTRobotics @VectorInst | Working on robotics, vision, and machine learning.
aoberai @aditya_oberai
811 Followers 689 Following
Michal Nauman @mic_nau
285 Followers 903 Following Visiting scholar @ robot learning lab UC Berkeley. PhD student in ML/Robotics.
Max Vladymyrov 🇺�... @mvladymyrov
1K Followers 1K Following Previously Research @ {Google DeepMind, Yahoo Labs}.
Cade Gordon @CadeGordonML
2K Followers 841 Following Helping models grow wise @Anthropic | Hertz Fellow | Prev: LAION-5B & OpenCLIP @UCBerkeley
ViktorM🇺🇦 @viktor_m81
3K Followers 3K Following Chief Scientist @clonerobotics, ex-Research Scientist @NVIDIA. Exploring simulation, robotics, dexterity, and RL by day - painting and piano by night.
Chet Bhateja @ChetBhateja
56 Followers 829 Following
Alexander Nikulin @how_uhh
344 Followers 787 Following Research Scientist, RL https://t.co/JesJsTrrTy | https://t.co/nYq9gTt9oQ
David McAllister @davidrmcall
791 Followers 258 Following PhD Student @berkeley_ai | Interning with Nvidia in Helsinki
Noam Brown @polynoamial
91K Followers 853 Following Researching reasoning @OpenAI | Co-created Libratus/Pluribus superhuman poker AIs, CICERO Diplomacy AI, and OpenAI o3 / o1 / 🍓 reasoning models
Jeff Clune @jeffclune
29K Followers 431 Following Professor, CS, U. British Columbia. CIFAR AI Chair, Vector Institute. Sr. Advisor, DeepMind | ML, AI, deep RL, deep learning, AI-Generating Algorithms (AI-GAs)
Logan Kilpatrick @OfficialLoganK
209K Followers 2K Following Lead product for @GoogleAIStudio + the Gemini API. My views!
Priya Sundaresan @priyasun_
2K Followers 528 Following CS PhD student @Stanford, prev. Intrinsic, @Amazon Robotics, @UCBerkeley | learning from humans & teaching robots
Pranav Atreya @pranav_atreya
262 Followers 257 Following Robot learning | CS Ph.D. student @berkeley_ai
Ryan Julian @ryancjulian
1K Followers 2K Following Bringing learning robots to the world. Research Scientist @nvidia GEAR. PhD @CSatUSC. Formerly @GoogleDeepMind, Meka, @LeapMotion, Everyday Robots.Trends for United States
You might like
