Inherent biases and imbalances in robot data can make training steerable VLA policies challenging. We introduce CAST, a method to augment datasets with counterfactuals to induce better language following
cast-vla.github.io ← paper, code, data, and more available here! 🧵
Excited to share Flow Matching Policy Gradients: expressive RL policies trained from rewards using flow matching. It’s an easy, drop-in replacement for Gaussian PPO on control tasks.
Imitation learning has seen great success, but IL policies still struggle with OOD observations
We designed a 3D backbone, Adapt3R, that can combine with your favorite IL algorithm to enable zero-shot generalization to unseen embodiments and camera viewpoints!
In LLM land, a slow model is annoying. In robotics, a slow model can be disastrous! Visible pauses at best, dangerously jerky motions at worst. But large VLAs are slow by nature. What can we do about this? An in-depth 🧵:
Was super fun to demo Gemini Robotics @ Google I/O! This was a big effort with the @GoogleDeepMind team including @ColinearDevin, @SudeepDasari, and many others. Here's a fun uncut video of me playing with the demo :)
Imitation learning has a data scarcity problem.
Introducing EgoDex from Apple, the largest and most diverse dataset of dexterous human manipulation to date — 829 hours of egocentric video + paired 3D hand poses across 194 tasks.
Now on arxiv: arxiv.org/abs/2505.11709 (1/4)
Excited to introduce PyRoki ("Python Robot Kinematics"): easier IK, trajectory optimization, motion retargeting... with an open-source toolkit on both CPU and GPU
our new system trains humanoid robots using data from cell phone videos, enabling skills such as climbing stairs and sitting on chairs in a single policy
(w/ @redstone_hong@junyi42@davidrmcall)
432 Followers 1K FollowingHumanoid robots are reshaping our world.
Hundreds of companies are pioneering this revolution — we're here to tell their stories!
19K Followers 3K FollowingFrom SLAM to Spatial AI; Professor of Robot Vision, Imperial College London; Director of the Dyson Robotics Lab; Co-Founder of Slamcore. FREng, FRS.
543K Followers 23K FollowingThe best from ML/AI community | Ex-Microsoft, Rackspace, Fast Company | Wrote eight books about the future | Silicon Valley robots, holodecks, BCIs, & startups.
368 Followers 444 FollowingRobotics PhD student @GeorgiaTech advised by Animesh Garg | M.S. in MAE @Princeton | B.S. in ASE @UTAustin | Robotics, robot learning
220 Followers 228 Followingsecond year PhD student at Georgia Tech @ICatGT working on 3D scene representations and foundation models | MS from @berkeley_ai
15K Followers 7K FollowingI build tough benchmarks for LMs and then I get the LMs to solve them. SWE-bench & SWE-agent. Postdoc @Princeton. PhD @nlpnoah @UW.
916 Followers 583 FollowingAI Research Scientist at Meta Reality Labs (in Zurich) | PhD at UC Berkeley | MIT EECS BS '20 & MEng '21 | CV for AR/VR & robotics | https://t.co/YhPzCHLcqi
323 Followers 4K Following“Honesty and Integrity shall never be brought in question, or his motives doubted or impugned” Allan Pinkerton General Principles & Rules 1867
101 Followers 2K FollowingJava/Python developer. I'm Studying iOS development now. I'm pushing myself to express in English/Japanese instead of 中文. I wanna explore more vast world.
1K Followers 2K FollowingResearch Scientist @ToyotaResearch | PhD in AI and DL @GeorgiaTech | Researching Large Behavioral Models | 3D Vision | Robotics
20K Followers 3K FollowingMostly posting about robots.
currently AI @agilityrobotics
prev embodied AI @AIatMeta, @NVIDIAAI. All views my own.
writing: https://t.co/iNLA4djfZo
220 Followers 228 Followingsecond year PhD student at Georgia Tech @ICatGT working on 3D scene representations and foundation models | MS from @berkeley_ai
916 Followers 583 FollowingAI Research Scientist at Meta Reality Labs (in Zurich) | PhD at UC Berkeley | MIT EECS BS '20 & MEng '21 | CV for AR/VR & robotics | https://t.co/YhPzCHLcqi
1K Followers 2K FollowingResearch Scientist @ToyotaResearch | PhD in AI and DL @GeorgiaTech | Researching Large Behavioral Models | 3D Vision | Robotics