-
Tweets314
-
Followers994
-
Following428
-
Likes1K
Oleg Rybkin retweeted
With the right design decisions, value-based RL admits predictable scaling. value-scaling.github.io We wrote a blog post on our two papers challenging conventional wisdom that off-policy RL methods are fundamentally unpredictable.
4
57
445
37K
437
Download Image
Oleg Rybkin retweeted
@preston_fu @_oleh and I wrote a blog post on scaling laws and value function based RL, summarizing our two papers in this direction and discussing open questions! value-scaling.github.io Check it out! Feedback & comments are very welcome!
Oleg Rybkin retweeted
We have been doing work on scaling laws for off-policy RL for some time now and we just put a new paper out: arxiv.org/abs/2508.14881 Here, @preston_fu @_oleh lead a study on how to best allocate compute for training value functions in deep RL: 🧵⬇️
Oleg Rybkin retweeted
Following up on our work on scaling laws for value-based RL (led by @_oleh and @preston_fu), we've been trying to figure out compute optimal parameters for value-based RL training. Check out Preston's post about our findings!
Following up on our work on scaling laws for value-based RL (led by @_oleh and @preston_fu), we've been trying to figure out compute optimal parameters for value-based RL training. Check out Preston's post about our findings!
Oleg Rybkin retweeted
How can we best scale up value based RL? We need to use bigger models, which mitigate what we call “TD-overfitting” (more below!👇 🧵 ). Further, we need to scale batch size and UTD accordingly as the models get bigger. Great work led by @preston_fu and @_oleh
How can we best scale up value based RL? We need to use bigger models, which mitigate what we call “TD-overfitting” (more below!👇 🧵 ). Further, we need to scale batch size and UTD accordingly as the models get bigger. Great work led by @preston_fu and @_oleh
📈📈📈
Cool work by David and friends! Could this be the thing that finally makes everyone stop using Gaussians as their policies? 🤔
Cool work by David and friends! Could this be the thing that finally makes everyone stop using Gaussians as their policies? 🤔
Oleg Rybkin retweeted
Everyone knows action chunking is great for imitation learning. It turns out that we can extend its success to RL to better leverage prior data for improved exploration and online sample efficiency! colinqiyangli.github.io/qc/ The recipe to achieve this is incredibly simple. 🧵 1/N
Very insightful analysis that I mostly agree with (except the overly pessimistic title :)!
Very insightful analysis that I mostly agree with (except the overly pessimistic title :)!
Really interesting result! Scaling value-based RL is hard and we are still missing much of the machinery to do it. @seohong_park shows that horizon is the critical issue.
Really interesting result! Scaling value-based RL is hard and we are still missing much of the machinery to do it. @seohong_park shows that horizon is the critical issue.
Oleg Rybkin retweeted
We found a way to do RL *only* with BC policies. The idea is simple: 1. Train a BC policy π(a|s) 2. Train a conditional BC policy π(a|s, z) 3. Amplify(!) the difference between π(a|s, z) and π(a|s) using CFG Here, z can be anything (e.g., goals for goal-conditioned RL). 🧵↓
Oleg Rybkin retweeted
This was fun thanks for having me @chris_j_paxton @micoolcho! See the podcast for some livestream of the robot in real time and me evaluating a policy live! Or check it out for yourself at auto-eval.github.io and eval your policy in real without breaking a sweat
This was fun thanks for having me @chris_j_paxton @micoolcho! See the podcast for some livestream of the robot in real time and me evaluating a policy live! Or check it out for yourself at auto-eval.github.io and eval your policy in real without breaking a sweat
AK @_akhaliq
428K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo ,submit papers here: https://t.co/UzmYN5YmrQ
Danijar Hafner @danijarh
22K Followers 1K Following Building AI that makes autonomous decisions using world models, artificial curiosity, and temporal abstraction @GoogleDeepMind
Jim Fan @DrJimFan
327K Followers 3K Following NVIDIA Director of Robotics & Distinguished Scientist. Co-Lead of GEAR lab. Solving Physical AGI, one motor at a time. Stanford Ph.D. OpenAI's 1st intern.
Eugene Vinitsky (@RLC... @EugeneVinitsky
21K Followers 2K Following This is the site where I talk about the attacks on science and immigration. Science is on the other site. Lab website: https://t.co/vrtbcqRyRn
Shane Gu @shaneguML
42K Followers 2K Following Gemini Thinking, Senior Staff RS @GoogleDeepMind. 🇯🇵-born 🇨🇳🇨🇦. ex: Gemini Multilinguality Post-Train Lead, GPT-4 @OpenAI (JP: @shanegJP)
Ted Xiao @ CoRL 2025 @xiao_ted
16K Followers 739 Following Robotics and Gemini @GoogleDeepMind. Posts about frontier models, robot learning, and scaling. Opinions my own.
Abhishek Gupta @abhishekunique7
9K Followers 880 Following Assistant Professor at University of Washington. I like robots, and reinforcement learning. Previously: post-doc at MIT, PhD at Berkeley
Misha Laskin @MishaLaskin
15K Followers 215 Following Co-founder, CEO at @reflection_ai. Prev: Research @DeepMind. Gemini RL team.
Nathan Lambert @natolambert
57K Followers 857 Following Figuring out AI @allen_ai, open models, RLHF, fine-tuning, etc Contact via email. Writes @interconnectsai Wrote The RLHF Book Mountain runner
Dinesh Jayaraman @dineshjayaraman
2K Followers 581 Following Assistant Professor at University of Pennsylvania. Robot Learning. https://t.co/cIMw5XKSPy
Kostas Daniilidis @KostasPenn
5K Followers 1K Following Ruth Yalom Stone Professor @Penn @PennEngineers @PennCIS @GRASPlab
Markus Wulfmeier @m_wulfmeier
12K Followers 2K Following Large-Scale Robot Intelligence - Research @GoogleDeepMind European @ELLISforEurope - priors: @oxfordrobots @berkeley_ai @ETH @MIT
Kosta Derpanis @CSProfKGD
69K Followers 197 Following #CS Assoc Prof @YorkUniversity, #ComputerVision Scientist Samsung #AI, @VectorInst Faculty Affiliate, TPAMI AE, @ELLISforEurope Member #ICCV2025 Publicity Chair
Roberta Raileanu @robertarail
9K Followers 2K Following Senior Staff Research Scientist @GoogleDeepMind & Honorary Lecturer @UCL. ex @Meta|@MSFTResearch|@NYU|@Princeton. Llama-3, Toolformer, Rainbow Teaming, MLGym.
Nikolai Matni @NikolaiMatni
3K Followers 1K Following machine learning, control, optimization, robotics. associate professor, upenn #FlyEaglesFly #RedOctober
Tim Rocktäschel @_rockt
40K Followers 2K Following Director and Open-Endedness Team Lead @GoogleDeepMind, Professor of AI @AI_UCL, PI @UCL_DARK, Fellow @ELLISforEurope.
Chris Paxton @chris_j_paxton
20K Followers 3K Following Mostly posting about robots. currently AI @agilityrobotics prev embodied AI @AIatMeta, @NVIDIAAI. All views my own. writing: https://t.co/iNLA4djfZo
Edward Hu @edward_s_hu
913 Followers 338 Following cs phd @penn, prev @MSFTResearch. investigating ai / rl / intelligence.
Jagdeep Bhatia @JagdeepBhatia8
281 Followers 436 Following ML and Robotics PhD Student @berkeley_ai prev ug @MIT
Jeong Jun Kim @jng_jun
1 Followers 73 Following
Guneet Mummaneni @guneetm19
0 Followers 33 Following
Davis Elizabeth @DavisEliza9331
34 Followers 26 Following 138. My coffee is cold, and so is my soul. ☕❄️
Hanna Zee @HannaZeeX
38 Followers 277 Following Full-time crypto builder exploring tokenomics since 2016., Let's navigate this space together 🚀
AnnaliseBrockman27110 @RntxHammond696
5 Followers 15 Following On a journey as a Humble learner who enjoys photography.
vasudev anubrolu @vasudevanubrolu
90 Followers 2K Following ML enthusiast, Engineer. Sr. SE @koredotai. ex @deloitte @vmware. @bitspilaniindia 2015-20.
PJC @FakerPJC
80 Followers 244 Following Visual artist creating NFT magic that sticks. — Stay sharp. Stay decentralized 🧠
Patrick Birch @Patrick_BirchM
13 Followers 122 Following
Arthur Yau @imitation_alpha
2 Followers 89 Following ML SWE @Google; Work on Representation Learning w/ GNN and Information Retrieval; Interested in Reinforcement Learning
Koi Hunter @KoiHunterX
23 Followers 271 Following Crypto content creator 🎬 | Turning complex topics into simple stories. Narratives drive markets 🚀.
Logan Erickson @LoganErick99033
17 Followers 105 Following
Williams Linda @WilliamsLi11413
38 Followers 31 Following 87. When you’re trying to relax, but your brain says, “Let’s overthink everything.” 🧠🤯
Mandy Lim @Giobbncstark
682 Followers 4K Following If I play my best, I can win anywhere in the world against anybody.
Alex Pan @aypan_17
540 Followers 275 Following CS PhD @UCBerkeley working on LLM safety and interpretability
Eric Zelikman @ericzelikman
21K Followers 2K Following building for humans // was lgtm-ing @xAI, phd-ing @stanford
Jason_wjs @wujered
2 Followers 68 Following
Ajitesh Shukla @ajitesh_shukla7
1K Followers 6K Following Student,Love to solve hardest math problem. LLM's, Mathematical Research(Geometric Topology,Differential Geometry),Quantum Computing.Lord Krishna is God Of Math
Dionysis Manousakas @neuralvertigo
311 Followers 4K Following Applied scientist @AWS AI/ML. Ex @Meta|@Cambridge_Uni|@ucl|@ecentua 🇬🇷. No cats.
Yağmur Yıldız @Yagmur84512
42 Followers 203 Following Finding patterns and opportunities in DeFi. Tracking emerging tokens 🌱.
Guthrie Williamson @guthriejw
1K Followers 7K Following co-owner of: PORT LOCKROY, BOIS D’ARGENT, ZAAKI, GEAR UP, LAWS OF INDICES, BRUTAL. Same brand used by my father on: Great Klaire, Eight Carat, Cotehele House
Ben Lu @bnzylu
2 Followers 58 Following
chloe baxter @ChloeBaxter86
42 Followers 333 Following Your early radar for airdrops and ecosystem plays., DYOR before the next wave 🌊
Bhavya Agrawalla @AgrawallaBhavya
97 Followers 380 Following Research Interests - Statistics, Deep Reinforcement Learning. PhD student @CMU CS. Prev - Math and CS undergrad at MIT (2021-24) and IISc Bangalore (2020-21).
三水寿 @userdemo_
28 Followers 241 Following
Robert Scoble @Scobleizer
543K Followers 24K Following The best from ML/AI community | Ex-Microsoft, Rackspace, Fast Company | Wrote eight books about the future | Silicon Valley robots, holodecks, BCIs, & startups.
mitchell @mockups
1K Followers 2K Following Founder @spoil, sold @responsesapp. Sharing my ideas & mockups.
Rohan Bhise @rohanbhise836
0 Followers 497 Following
Nir Aviv @nir_aviv
122 Followers 5K Following
Anurag Ajay @aajay3110
316 Followers 470 Following Building Astra, Gemini p13n @GoogleDeepMind. Prev: @MetaAI. PhD @MIT. Opinions my own.
Hussein Muhaisen @husseinmuhaisen
2K Followers 4K Following Making LLMs better at security in {stealth} // @ // PagedOut and GuidedHacking
Haque Ishfaq @HaqueIshfaq
1K Followers 1K Following PhD student at @mcgillu/ @MILAMontreal. Reinforcement Learning. BS, MS @Stanford 🇧🇩🇺🇸🇨🇦
Boyi Li @Boyiliee
2K Followers 320 Following
AK @_akhaliq
428K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo ,submit papers here: https://t.co/UzmYN5YmrQ
Danijar Hafner @danijarh
22K Followers 1K Following Building AI that makes autonomous decisions using world models, artificial curiosity, and temporal abstraction @GoogleDeepMind
Google DeepMind @GoogleDeepMind
1.2M Followers 279 Following We’re a team of scientists, engineers, ethicists and more, committed to solving intelligence, to advance science and benefit humanity.
Andrej Karpathy @karpathy
1.4M Followers 1K Following Building @EurekaLabsAI. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets.
Sergey Levine @svlevine
110K Followers 133 Following Associate Professor at UC Berkeley Co-founder, Physical Intelligence
Eugene Vinitsky (@RLC... @EugeneVinitsky
21K Followers 2K Following This is the site where I talk about the attacks on science and immigration. Science is on the other site. Lab website: https://t.co/vrtbcqRyRn
Natasha Jaques @natashajaques
31K Followers 1K Following Assistant Professor @uwcse and Staff Research Scientist at @GoogleAI. Let's get off this app: https://t.co/jbH2oAjbPN
Aran Komatsuzaki @arankomatsuzaki
146K Followers 305 Following Looking for a cofounder. Sharing AI research. Early work on AI (GPT-J, LAION, scaling, MoE). Ex ML PhD (GT) & Google.
Shane Gu @shaneguML
42K Followers 2K Following Gemini Thinking, Senior Staff RS @GoogleDeepMind. 🇯🇵-born 🇨🇳🇨🇦. ex: Gemini Multilinguality Post-Train Lead, GPT-4 @OpenAI (JP: @shanegJP)
Animesh Garg @CORL202... @animesh_garg
29K Followers 1K Following Foundation Models for Generalizable Autonomy in Robotics. Reinforcement Learning. Assistant Professor in AI Robotics @GeorgiaTech. Prev @nvidia
Lucas Beyer (bl16) @giffmana
110K Followers 523 Following Researcher (now: Meta. ex: OpenAI, DeepMind, Brain, RWTH Aachen), Gamer, Hacker, Belgian. Anon feedback: https://t.co/xe2XUqkKit ✗DMs → email
Michael Black @Michael_J_Black
85K Followers 706 Following Director, Max Planck Institute for Intelligent Systems (@MPI_IS). Chief Scientist @meshcapade. Building 3D digital humans using vision, graphics, and learning.
Soumith Chintala @soumithchintala
252K Followers 1K Following Cofounded and lead @PyTorch at Meta. Also dabble in robotics at NYU. AI is delicious when it is accessible and open-source.
Ted Xiao @ CoRL 2025 @xiao_ted
16K Followers 739 Following Robotics and Gemini @GoogleDeepMind. Posts about frontier models, robot learning, and scaling. Opinions my own.
Abhishek Gupta @abhishekunique7
9K Followers 880 Following Assistant Professor at University of Washington. I like robots, and reinforcement learning. Previously: post-doc at MIT, PhD at Berkeley
Wojciech Zaremba @woj_zaremba
121K Followers 204 Following Co-Founder of OpenAI https://t.co/OCQ3mpf0IN
Nat McAleese @__nmca__
15K Followers 358 Following Research @AnthropicAI. Previously @OpenAI, @DeepMind. Views my own.
Anurag Ajay @aajay3110
316 Followers 470 Following Building Astra, Gemini p13n @GoogleDeepMind. Prev: @MetaAI. PhD @MIT. Opinions my own.
Alex Pan @aypan_17
540 Followers 275 Following CS PhD @UCBerkeley working on LLM safety and interpretability
SangBin Cho @Saaaang94
3K Followers 488 Following reasoning @xAI | prev-founding engineer @anyscalecompute | senior committer of @raydistributed | committer @vllm_project Sglang | Github: rkooo567
Shengyang Sun @ssydasheng
5K Followers 574 Following Build AGI @xAI | Prev. @NVIDIA (Leading Nemotron-340B) & @AMAZON | PhD @UofT ; B.E.@Tsinghua
Albert Gu @_albertgu
18K Followers 88 Following assistant prof @mldcmu. chief scientist @cartesia_ai. leading the ssm revolution.
Tianle (Tim) Li @LiTianleli
6K Followers 223 Following 🧑🌾 MTS, Training models 24/7 @xai | @grok reasoning and post-training | Prev. GPU Poor at @lmarena_ai @lmsysorg @GoogleAI @Berkeley_EECS
Liangchen Luo @LiangchenLuo
5K Followers 126 Following @xAI reasoning; ex @GoogleDeepMind. B.Sc. @PKU1898. Opinions are my own.
Minqi Jiang @MinqiJiang
6K Followers 880 Following
Karina Nguyen @karinanguyen_
41K Followers 998 Following research & product @OpenAI, prev. @AnthropicAI, @nytimes, @square, @dropbox, visual forensics for Pulitzer Prize investigations
Igor Babuschkin @ibab
103K Followers 855 Following Maybe the real ASI was the friends we made along the way. Co-founder @xAI, Research & Engineering
Jiayi Pan @jiayi_pirate
13K Followers 2K Following 🧑🍳 Reasoning Agents @xAI | PhD on Leave @Berkeley_AI | Views Are My Own
Xuechen Li @lxuechen
16K Followers 944 Following Previously @xai. Interested in the engineering and science for scaling. Opinions are my own. @Stanford PhD.
Serena Ge @serenaa_ge
7K Followers 2K Following @datacurve_ai (@ycombinator W24) Prev @cohere @uwaterloo
Yuchen He @YuchenHe07
2K Followers 640 Following learning @xai | prev @openai@meta@apple@uiuc@utaustin
Qian Huang @qhwang3
14K Followers 330 Following prev @xai | CS PhD student @StanfordAILab (on leave)
Szymon Tworkowski @s_tworkowski
10K Followers 661 Following reasoning @xAI | prev. @GoogleAI @UniWarszawski | LongLLaMA
jessica dai @jessicadai_
2K Followers 713 Following phd student @berkeley_ai !? also editorial @reboot_hq @kernel_magazine (she/her)
Kun Huang @kun_h____
231 Followers 149 Following Founding Researcher @DynaRobotics | ex Cruise& @Waymo | CoRL Best Paper
Ritvik Singh @ritvik_singh9
913 Followers 309 Following PhD student @berkeley_ai. prev. @NvidiaAI, @UofT
Aldo Pacchiano @aldopacchiano
1K Followers 455 Following AI research at Broad Institute and Boston University. Reinforcement Learning / Bandits / Experiment Design Mexicano 🇲🇽
Philippe Hansen-Estru... @tokenpilled65B
641 Followers 888 Following RS Intern Meta. Second-year PhD student at UT Austin. Working on generative modeling, visual understanding, and visual compression.
john so @johnrso_
699 Followers 679 Following robots! prev @tesla_optimus; @1x_tech; @stanford; @berkeley_ai. raised by @berkeleyml
Hongsuk Benjamin Choi @redstone_hong
546 Followers 412 Following robotics & computer vision. PhD @Berkeley_AI | prev @ Seoul National University | Intern at Amazon FAR
Medhini Narasimhan @medhini_n
2K Followers 514 Following Sr Research Scientist @googledeepmind #Veo3 #Veo2 #Veo Prev: Ph.D. @berkeley_ai, MS @IllinoisCS, Intern @GoogleAI @MetaAI
Alex Nichol @unixpickle
11K Followers 422 Following Code, AI, and 3D printing. Opinions are mostly my own, sometimes my computer's. Husband of @thesamnichol. Co-creator of DALL-E 2. Researcher @openai.
Eric Zelikman @ericzelikman
21K Followers 2K Following building for humans // was lgtm-ing @xAI, phd-ing @stanford
womerhockey @womerhockey
10 Followers 8 Following
Florian Shkurti @florian_shkurti
2K Followers 2K Following Assistant professor in computer science, University of Toronto | @UofTRobotics @VectorInst | Working on robotics, vision, and machine learning.
aoberai @aditya_oberai
812 Followers 694 Following
Michal Nauman @mic_nau
304 Followers 910 Following Visiting scholar @ robot learning lab UC Berkeley. PhD student in ML/Robotics.
Max Vladymyrov @mvladymyrov
1K Followers 1K Following MTS at Anthropic. Previously: Research @ {Google DeepMind, Yahoo Labs} 🇺🇦
Cade Gordon @CadeGordonML
2K Followers 845 Following Helping models grow wise @Anthropic | Hertz Fellow | Prev: LAION-5B & OpenCLIP @UCBerkeley
ViktorM🇺🇦 @viktor_m81
3K Followers 3K Following Chief Scientist @clonerobotics, ex-Research Scientist @NVIDIA. Exploring simulation, robotics, dexterity, and RL by day - painting and piano by night.
Chet Bhateja @ChetBhateja
57 Followers 853 Following
Alexander Nikulin @how_uhh
348 Followers 793 Following Research Scientist, RL https://t.co/JesJsTrrTy
David McAllister @davidrmcall
810 Followers 262 Following PhD Student @berkeley_ai | Interning with Nvidia in Helsinki
Noam Brown @polynoamial
92K Followers 856 Following Researching reasoning @OpenAI | Co-created Libratus/Pluribus superhuman poker AIs, CICERO Diplomacy AI, and OpenAI o3 / o1 / 🍓 reasoning models
Jeff Clune @jeffclune
29K Followers 431 Following Professor, CS, U. British Columbia. CIFAR AI Chair, Vector Institute. Sr. Advisor, DeepMind | ML, AI, deep RL, deep learning, AI-Generating Algorithms (AI-GAs)Trends for United States
170 B posts
49,8 B posts
86 B posts
6.948 posts
41,5 B posts
13,7 B posts
4.732 posts
39,4 B posts
11,6 B posts
56,3 B posts
7.253 posts
6.801 posts
20,9 B posts
47,4 B posts
3.552 posts
11,1 B posts
6.712 posts
3.698 posts
7.735 posts
8.122 posts
You might like
Abhishek Gupta
@abhishekunique7
9K Followers
Ofir Nachum
@ofirnachum
5K Followers
Ben Eysenbach
@ben_eysenbach
5K Followers
Rishabh Agarwal
@agarwl_
17K Followers
Vikash Kumar
@Vikashplus
5K Followers
Karl Pertsch
@KarlPertsch
4K Followers
Stephen James
@stepjamUK
6K Followers
Igor Mordatch
@IMordatch
5K Followers
Dhruv Shah @ CoRL �...
@shahdhruv_
5K Followers
Kostas Daniilidis
@KostasPenn
5K Followers
Aravind Rajeswaran
@aravindr93
3K Followers
Dinesh Jayaraman
@dineshjayaraman
2K Followers
Murtaza Dalal
@mihdalal
8K Followers
Lerrel Pinto
@LerrelPinto
8K Followers
Jesse Zhang
@Jesse_Y_Zhang
1K Followers