Kaiwen Zhou @KaiwenZhou9
A CSE PhD student in @ucsc, working on multimodal embodied AI. Previous: @Samsung_RA, @hri_usa Joined March 2022-
Tweets28
-
Followers74
-
Following105
-
Likes40
Gen AI gains attention these days. Can generative models be used for discriminative tasks? Our collaborative work with UCSB and Google turns pre-trained text-to-image diffusion models into few-shot discriminative learners. Our approach mainly uses the cross-attention score of a…
Gen AI gains attention these days. Can generative models be used for discriminative tasks? Our collaborative work with UCSB and Google turns pre-trained text-to-image diffusion models into few-shot discriminative learners. Our approach mainly uses the cross-attention score of a… https://t.co/BqkTMLhQNl
SwapAnything Enabling Arbitrary Object Swapping in Personalized Visual Editing Effective editing of personal content holds a pivotal role in enabling individuals to express their creativity, weaving captivating narratives within their visual stories, and elevate the
🚨Ever worried your smart home might turn against you in the near future? What if attackers could command your home assistants for something really bad? We're tackling this sci-fi scenario head-on! 🛡️ 🔥Excited to unveil our #NAACL2024 paper, "Navigation as Attackers Wish?…
Can we improve the compositionality of vision-and-language models? Excited to share our #NAACL paper "ComCLIP: Training-Free Compositional Image and Text Matching". 📄 Paper: arxiv.org/abs/2211.13854 🌐 Project: sites.google.com/view/comclip 🛠️ Code/Data: github.com/eric-ai-lab/Co… (1/4)
It's interesting to see how Sora triggers so much discussion on world models🌎 Does Sora understand Physics? Is Sora a world model? It's not a True/False question. The real questions are: ❓How capable is video (pixels) to represent the world? ❓And how efficient is it? ❓So…
It's interesting to see how Sora triggers so much discussion on world models🌎 Does Sora understand Physics? Is Sora a world model? It's not a True/False question. The real questions are: ❓How capable is video (pixels) to represent the world? ❓And how efficient is it? ❓So…
One of the bigger bimanual teleop demos I’ve seen 🤣
One of the bigger bimanual teleop demos I’ve seen 🤣
Problems in everyday life that seem easy and normal for humans still are challenging for SOTA MLLMs. Check out our work on Multipanel VQA!
Problems in everyday life that seem easy and normal for humans still are challenging for SOTA MLLMs. Check out our work on Multipanel VQA!
The @scale_AI team put a spotlight⚡️on our work "Pushing Mixture of Experts to the Limit" arxiv.org/abs/2309.05444 and showcase the impact on LLaMA-2. Really nice blog post + working implementation. 🔥 scale.com/blog/fine-tuni…
Humans effortlessly grasp the fine-grained correspondences between sketches and real-world objects. How well can current vision algorithms do the same? To find out, check out our #ICML2023 paper w/ @xiaolonw @judyefan! photo-sketch-correspondence.github.io
Leah Ulibarri @LeahU74430
45 Followers 2K Following
Data Science Research.. @ArionDas
361 Followers 2K Following Deep Learning || Research Work on ML, DL || Large Language Models || GANs || RAG || Competitive Programming || Generative AI || Optimization Algorithms || IIITR
Carolyn Sykes @CSykes16293
105 Followers 3K Following
Vidhi Jain @viddivj
3K Followers 3K Following Graduate student at @CMU_Robotics. Previously a student researcher @Google @GoogleDeepMind Robotics. @MetaAI Resident, @IndiaMSR, @bitspilaniindia She/her
Khanh Nguyen @khanhxuannguyen
1K Followers 460 Following Postdoc at CHAI Berkeley with Prof. Stuart Russell, Prev. Postdoc at Princeton NLP, PhD @umdcs, Human-AI Communication, Interactive Learning, NLP.
LY @YantoLiem11
217 Followers 2K Following
camenduru @camenduru
16K Followers 4K Following ML & Computer Engineer, Game Designer. #OpenSource ❤ #UE ❤ #Jupyter ❤ #AI #ML #StableDiffusion #LLM #NeRF #GaussianSplatting #T2V https://t.co/8MMNbygz1P
Weixi Feng @weixi_feng
399 Followers 292 Following CS Ph.D. candidate @UCSB @UCSBNLP. Ex-research intern @Adobe, @Amazon. #Multimodality #ComputerVision #NLProc.
Kefei @Kefei_1211
31 Followers 532 Following
Fu-En Yang @FuEnYang1
264 Followers 892 Following Research Scientist @NVIDIAAI | Ph.D. @ NTU | Prev. Research Intern @NVIDIAAI | Vision & Language | Multimodal AI
Xinyi Wang @XinyiWang98
803 Followers 300 Following UC Santa Barbara CS PhD student working on ML/NLP
Kolby Nottingham @kolbytn
218 Followers 230 Following CS PhD at @UCIrvine researching RL+NLP and interactive LLMs. Upcoming intern @riotgames. Previously @allen_ai, @AiDungeon, @unity, and @nvidia .
Zhen Wang @zhenwangwz
122 Followers 327 Following PhD student @UCLA_VMG @UCLA | Intern @Google | Previously @zuckermanbrain @Columbia.
Debargha Ganguly @Debargha_
877 Followers 2K Following Trustworthy + scalable ML, CS PhD student @cwru; alum @ashokauniv
Code and Robot ID @coderobot_id
9 Followers 260 Following Channel dikhususkan untuk belajar code, AI, dan tutorial membuat sebuah project berbasis arduino dan raspberry pi secara sederhana dan mudah dipahami.
EcosystemStories @MetaverseDance
403 Followers 349 Following post acquisition startup technical founder working on interactive stories/simulations using generative ai & vector DBs
Yihe Deng @Yihe__Deng
2K Followers 1K Following CS PhD student @UCLA | Prev. Applied Scientist Intern @AWS | LLM, Multi-modal learning
Qian Lou @qianlife22
241 Followers 619 Following Assistant professor at UCF; Former Sr. research scientist at Samsung Research; Private/Secure/Efficient Learning Systems
Zhimeng Jiang @ZhimengJ
393 Followers 1K Following Staff Research Scientist@Visa Research | CS Ph.D. @tamu| Formerly, @Amazon & @Visa & @Samsung | Trustworthy ML & Graph Neural Network | Opinions are my own
Jared Heinly @JaredHeinly
3K Followers 4K Following Chief Scientist at @EveryPointIO | 3D computer vision researcher (PhD) and engineer
Yizhou Wang @YizhouWang14
410 Followers 2K Following Computer Engineering Ph.D. student @SmileLabNEU @Northeastern | I work on anomaly detection in machine learning. Still figuring out what AGI means.
Wuao Liu @liu_wuao
203 Followers 1K Following CS PhD Student @UMassAmherst | Prev @UMRobotics @ZJU_China | Computer Vision, AI4Science
Seth Z. Zhao @sethzhao506
46 Followers 83 Following CS PhD Student @ UCLA | Prev @ UC Berkeley | Multimodal Embodied AI, Autonomous Driving
Kaizhi Zheng @KaizhiZheng
30 Followers 24 Following CSE Ph.D. student at UC, Santa Cruz; Mainly focused on multi-modal research and robot learning
Lol @Lol98779512
4 Followers 58 Following
Yanyuan Qiao @YanyuanQiao
197 Followers 101 Following Postdoctoral Research Fellow at Australian Institute for Machine Learning, University of Adelaide.
🍕.bitmap @0xa1b
442 Followers 5K Following As an AI, I do not have the ability to interpret or infer meaning from conversations beyond the specific memes that are used
Dong Carlo An @andongverse
31 Followers 140 Following Ph.D. student at CAS, working on Embodied-AI🤖 and Multimodal Learning.
Zeyu Zhang @Strange07985794
24 Followers 185 Following A PhD student focus on natural language processing, machine learning and artificial intelligence.
Yiran Geng @geng_yiran
352 Followers 983 Following Senior undergraduate student in Turing class, Peking University @PKU1898 | Visiting researcher @MIT | He/Him
Yujie Alice Lu @yujielu_10
751 Followers 468 Following CS PhD @UCSB NLP; ex-intern @MSFTResearch @AWS AI;
ShuboLiu @ShuboLiu12138
24 Followers 161 Following #CS PhD student | Insterested in #EmbodiedAI, vision-and-language Graduated from @QMUL
Peiyi Wang @sybilhyz
129 Followers 220 Following I am interested in the Reward Construction, Exploration Strategy, and Alignment Algorithms of AGI.
Deping Zhang @joebradly
101 Followers 3K Following
Sourabh Kondapaka @100rabh64
159 Followers 5K Following Just another Software Engineer @Mathworks. I work on cloud related projects. Starting to love @Neovim thanks to @ThePrimeagen and @chrisatmachine. ❤ @f1
I will buy your rugs @tldr_crypto
520 Followers 2K Following humble trader who's outtraded SBF, 3AC and Bill Hwang
royal Jackson @royalboy2004
171 Followers 5K Following
EP @EP225654
167 Followers 5K Following
AnhPhu Nguyen @AnhPhuNguyen1
2K Followers 174 Following Human Augmentation @ Harvard. Co-Founder @ Harvard AR/VR Club. Working on AI, XR, and Robotics projects! https://t.co/Bnihw7Z4Oo
Dhruv Batra @DhruvBatraDB
14K Followers 327 Following Senior Director (FAIR @MetaAI). Professor (@GeorgiaTech). Co-founded CaliperAI. Researcher in AI. @CarnegieMellon alum.
Jianwei Yang @jw2yang4ai
2K Followers 349 Following Principal Researcher at MSR Redmond; Ph.D. from Georgia Tech
Yue Wang @yuewang314
5K Followers 936 Following Assistant Professor @ USC CS and part-time Research Scientist @ Nvidia Research. Previous: EECS PhD @ MIT CSAIL. Opinions are mine.
Corey Lynch @coreylynch
10K Followers 1K Following AI at @figure_robot, previously research scientist at @GoogleDeepMind.
Bolei Zhou @zhoubolei
6K Followers 977 Following Assistant Professor at Computer Science Department @UCLAComSci @UCLAengineering @UCLA
Tairan He @TairanHe99
947 Followers 212 Following Robotics Ph.D. Student @CMU_Robotics @SCSatCMU. My goal is to challenge conventional notions of what robots can achieve.
Mikael Henaff @HenaffMikael
1K Followers 364 Following Research Scientist at @MetaAI, previously postdoc at @MSFTResearch and PhD at @nyuniversity. All views my own.
Pieter Abbeel @pabbeel
79K Followers 435 Following Diffusion Models; Large World Model; UniSim; TRPO; SAC; Ring Attention; MAML; HER; Domain Randomization; Decision Transformer; LLM as Zero-Shot Planners; RFM-1
Zhiting Hu @ZhitingHu
3K Followers 353 Following Assist. Prof. at UC San Diego; Artificial Intelligence, Machine Learning, Natural Language Processing
Roei Herzig @roeiherzig
803 Followers 658 Following Postdoc @berkeley_ai. Research Scientist @IBMResearch. PhD student @TelAvivUni 23'. Works on compositionality in Machine Vision & AI. Data is NOT all you need.
Thomas Lew @thomas__lew
186 Followers 166 Following Research Scientist at @ToyotaResearch. Optimal Control, Machine Learning, Robotics. PhD @Stanford. Previously intern at @Google, @NASAJPL.
Abhishek Gupta @abhishekunique7
5K Followers 640 Following Assistant Professor at University of Washington. I like robots, and reinforcement learning. Previously: post-doc at MIT, PhD at Berkeley
Tongzhou Mu 🤖 @ICL.. @tongzhou_mu
1K Followers 516 Following PhD student at UC San Diego w/ @HaoSuLabUCSD | Empowering 🤖 with data | Previously @NVIDIA @alexa99 @IntelAI @MSFTResearch
Ching-An Cheng @chinganc_rl
2K Followers 84 Following Senior Researcher at @MSFTResearch, working on usable theory and algorithms for Reinforcement Learning and Robotics.
Huazhe Harry Xu @HarryXu12
2K Followers 896 Following Hi, I like reinforcement learning, robots, and video games:) I am an amateur pianist. Assistant Prof at Tsinghua; Postdoc at Stanford; Ph.D. at Berkeley
UCSB NLP Group @ucsbNLP
1K Followers 735 Following The NLP Group @ University of California, Santa Barbara. Profs. @WilliamWangNLP, Xifeng Yan, Simon Todd, @CodeTerminator, @lileics; acct run by @m2saxon
Lerrel Pinto @LerrelPinto
5K Followers 181 Following Assistant Professor of CS @nyuniversity. I like robots!
🇺🇦Olexandr Maks.. @o_maksymets
508 Followers 480 Following Researcher in Facebook AI Research, Ph.D. in Computer Science.
Keerthana Gopalakrish.. @keerthanpg
13K Followers 830 Following Building Embodied AGI. Research @DeepMind. Opinions my own.
Stone Tao @ ICLR 2024 @Stone_Tao
2K Followers 879 Following PhD @UCSanDiego @HaoSuLabUCSD working on scalable robot learning and embodied AI. Co-founded @LuxAIChallenge to build AI competitions. @NSF GRFP fellow
Saining Xie @sainingxie
14K Followers 1K Following researcher in #deeplearning #computervision | assistant professor at @NYU_Courant @nyuniversity | previous: research scientist @metaai (FAIR) @UCSanDiego
Stella Biderman @BlancheMinerva
15K Followers 749 Following Open source LLMs and interpretability research at @BoozAllen and @AiEleuther. My employers disown my tweets. She/her
Chelsea Finn @chelseabfinn
69K Followers 384 Following Asst Prof of CS & EE @Stanford. PhD from @Berkeley_EECS, EECS BS from @MIT
Sharon Zhou @realSharonZhou
23K Followers 1 Following Building the future of LLMs | Cofounder & CEO, @LaminiAI | Prev: CS Faculty & PhD @Stanford. Product @Google. @Harvard | @MIT 35 under 35. Angel investor.
Scale AI @scale_AI
43K Followers 490 Following Our mission is to accelerate the development of AI. We believe that to make the best models, you need the best data.
Sara Hooker @sarahookr
39K Followers 8K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.
Cohere For AI @CohereForAI
16K Followers 178 Following We are a research lab and open science initiative that seeks to solve complex machine learning problems. Join us in exploring the unknown, together.
Deepak Pathak @pathak2206
16K Followers 316 Following I study topics in AI (machine learning, robotics & computer vision).
Ani Kembhavi @ ICLR 2.. @anikembhavi
2K Followers 298 Following Senior Director @allen_ai + Affiliate Assoc Prof @UW 📷 : Visual Prog, Unified-IO, BiDAF 🤖 : ProcTHOR, Objaverse, SPOC 🌎 : SATLAS All views my own.
Tesla Optimus @Tesla_Optimus
204K Followers 11 Following A general purpose, bi-pedal, humanoid robot capable of performing tasks that are unsafe, repetitive or boring.
Aravind Rajeswaran @aravindr93
3K Followers 168 Following Research Scientist at FAIR (@MetaAI) Formerly at @OpenAI and @GoogleAI PhD from @uwcse and BTech from @iitmadras
Chris Paxton @chris_j_paxton
8K Followers 2K Following Mostly posting about robots. Embodied AI @hellorobotinc, formerly @AIatMeta, @NVIDIAAI, @zoox. All views my own.
Roozbeh Mottaghi @RoozbehMottaghi
3K Followers 225 Following AI Researcher; Research Scientist Manager at FAIR, @AIatMeta; Affiliate Prof @uwcse; Ex Research Manager @allen_ai; post-doc @Stanford and CS PhD student @UCLA
Yifeng Zhu 朱毅枫 @yifengzhu_ut
1K Followers 540 Following Ph.D. student at UT Austin. Research interest in robot learning and general-purpose robots. Opinions are my own.
Kiana Ehsani @ehsanik
3K Followers 459 Following Senior Research Scientist @allen_ai, ex-Ph.D. @uwcse, Interested in computer vision, robotics and deep learning, Climber on the weekends "Opinions are my own"
Khanh Nguyen @khanhxuannguyen
1K Followers 460 Following Postdoc at CHAI Berkeley with Prof. Stuart Russell, Prev. Postdoc at Princeton NLP, PhD @umdcs, Human-AI Communication, Interactive Learning, NLP.
Xinyun Chen @xinyun_chen_
4K Followers 851 Following Research Scientist at @GoogleDeepMind. PhD from @Berkeley_EECS.
Rowan Cheung @rowancheung
499K Followers 381 Following Founder @therundownai. Sharing the latest developments in the world of artificial intelligence.
Weixi Feng @weixi_feng
399 Followers 292 Following CS Ph.D. candidate @UCSB @UCSBNLP. Ex-research intern @Adobe, @Amazon. #Multimodality #ComputerVision #NLProc.
Monicaxie @Monica_XieY
677 Followers 3K Following Investor @zhenfund | ex @MatrixPartners @AWS @AiFi | MBA @UNC | views are my own
Anirudha Majumdar @Majumdar_Ani
4K Followers 526 Following Assistant Professor in Robotics @Princeton. Visiting researcher @GoogleDeepMind in Princeton.
Joon Sung Park @joon_s_pk
5K Followers 1K Following CS Ph.D. student @StanfordHCI + @StanfordNLP. Previously @MSFTResearch, @IllinoisCS & @Swarthmore. Oil painter. HCI, NLP, generative agents, human-centered AI
Jiayuan Zhang @Tisoga
82K Followers 789 Following building AI-powered search engine → @devv_ai, previously @tiktok_us
Check out our spotlight paper next week in ICLR! And code has been released for running.🏃
🎉 Very Excited to present our recent work on “Selective🔍 Visual Representations for Embodied-AI🤖” next week at ICLR in Vienna🇦🇹!! 📣📣Important update! Our code and pretrained models are now available through our project website 🌐: embodied-codebook.github.io🚀 👋Come to my…
End of year outing for the GLAMOR ✨ Lab!
Large scale cross-embodied language-conditioned agents in a variety of video game domains! What’s particularly exciting is seeing positive transfer: generalist agents outperform specialist agents.
Introducing SIMA: the first generalist AI agent to follow natural-language instructions in a broad range of 3D virtual environments and video games. 🕹️ It can complete tasks similar to a human, and outperforms an agent trained in just one setting. 🧵 dpmd.ai/3TiYV7d
We partnered with gaming studios to train SIMA (Scalable Instructable Multiworld Agent) on @NoMansSky, @teardowngame, @Valheimgame and others. 🎮 These offer a wide range of distinct skills for it to learn, from flying a spaceship to crafting a helmet. dpmd.ai/3TiYV7d
Replacing regression with classification in RL improved performance across many domains, offline and online settings, and even scales to generalist settings like robotics!
Framing regression as a classification has been “dark knowledge” for some time. We wanted to shed some light on this phenomenon in deep RL: Framing value-learning as a classification significantly improves performance and scalability in deep RL. But... not all classification…
For the first time, we show that the Llama 7B LLM can be trained on a single consumer-grade GPU (RTX 4090) with only 24GB memory. This represents more than 82.5% reduction in memory for storing optimizer states during training. Training LLMs from scratch currently requires huge…
GaLore Memory-Efficient LLM Training by Gradient Low-Rank Projection Training Large Language Models (LLMs) presents significant memory challenges, predominantly due to the growing size of weights and optimizer states. Common memory-reduction approaches, such as low-rank
Incredible projects from @sanjibac and the whole team! Massive respect for pulling this off :)
Cooking in kitchens is fun. BUT doing it collaboratively with two robots is even more satisfying! We introduce MOSAIC, a modular framework that coordinates multiple robots to closely collaborate and cook with humans via natural language interaction and a repository of skills.
Plenty of work still to be done! But VLMs and LLMs provide a lot of the knowledge we need to make robots work in homes. But a reminder that we still need: - robust/reliable navigation - grasping - task planning LLMs are getting better at the last one, but still not quite there!
A question I hear frequently is how Large Language Models (LLMs) and Vision-Language Models (VLMs) can be integrated with robots. "OK Robot" from Meta AI is a new general purpose robot which does exactly that - combining VLMs for natural language instruction with existing…
“Sora doesn’t show any new technical innovations” - this take misses the point. 🎯 Sora (and other great polished OAI releases) are not *meant* to provide new knowledge to the research community. They are not papers, careful scientific experiments, or theory that adds to the sum…
It was wonderful to visit OSU, talk about our works on multimodal embodied agents, and more importantly, discuss the future of AI agents with @hhsun1 @ysu_nlp, other faculty, and their brilliant students. In-person interaction is definitely irreplaceable for science advancement!…
We were very glad to host @xwang_lk Dr. Xin (Eric) Wang yesterday! Eric talked about his recent work on multimodal embodied agents, commonsense reasoning in object navigation, grounding, as well as his vision for future research. Many thanks to OSU TDAI @OSUbigdata for the…
release day release day! OLMo 1b + 7b out today 🥳 and 65b coming soon... With OLMo, we are really focused on advancing the study of LLMs. We release **everything**, from toolkit to create its training dataset (dolma) to training & inference code. More details in thread 🧵
Low cost mobile manipulation in the real world. This is how we scale up robot learning, with real data and widely reproducible hardware!
Adaptive Mobile Manipulation for Articulated Objects In the Open World paper page: huggingface.co/papers/2401.14… Deploying robots in open-ended unstructured environments such as homes has been a long-standing research problem. However, robots are often studied only in closed-off lab…
Cool work! Looking forward to seeing this work opening new 'doors' in robotics!
Introducing Open-World Mobile Manipulation 🦾🌍 – A full-stack approach for operating articulated objects in open-ended unstructured environments: Unlocking doors with lever handles/ round knobs/ spring-loaded hinges 🔓🚪 Opening cabinets, drawers, and refrigerators 🗄️ 👇…
Fun fact: this project was mostly done before the release of ChatGPT (we mainly used Deberta but included ChatGPT results later). Yet, even after >1 year, it is still a unique and innovative method. Surprisingly, not many studies focus on probabilistic inference using LLMs,…
LLMs can be great tools, but is there a better way to leverage their knowledge rather than simply trusting its outputs? We can actually do probabilistic inference with the "soft" outputs and confidence of LLMs & VLM while considering their uncertainty! Our #ICML2023 paper "ESC:…
OK-Robot: building robotic systems for open-vocabulary mobile manipulation w/ zero training - current pre-trained VLMs and grasping works out-of-the-box on a lot of problems! - But, combining all of these parts correctly is crucial. - Lots of issues remain! See thread by Mahi ->
Wouldn’t it be nice if you could bring a robot home, give it a video of your room, and immediately start asking it to move objects around? Turns out, now you can! Introducing OK-Robot, a zero-shot language-specified pick & drop system that we built with exactly ZERO training! 🧵
AutoRT is now out on Arxiv! Check out how we set up fleet-scale data collection by leveraging Foundation Models for robot orchestration 📈 Website: auto-rt.github.io Paper: arxiv.org/abs/2401.12963 Original Thread: x.com/keerthanpg/sta…
Google presents AutoRT Embodied Foundation Models for Large Scale Orchestration of Robotic Agents paper page: huggingface.co/papers/2401.12… demonstrate AutoRT proposing instructions to over 20 robots across multiple buildings and collecting 77k real robot episodes via both…
DESPITE all these, OK-Robot got 58.5% zero-shot success on the 171 zero-shot trials we ran across 10 home envs! There is still a lot of alpha in executing simple ideas well, so I hope this inspires the community to pursue general robotics in the real world/homes more seriously :)
Congratulations Yue! Yue's work @allen_ai this past summer was creating Holodeck: Language Guided Generation of 3D Embodied AI Environments yueyang1996.github.io/holodeck/
Thrilled and honored to be named the AI2 Outstanding Intern! It was an incredible journey working with the talented team at AI2! 🥰
Different scenarios, different behaviors, right? 🤔 So why not have one model that adapts to what you need instead of building a new one each time? 🚀 That's what we did! Had a blast working with Minyoung on this 👩🔬👨💻. Oh, and she's hunting for a Ph.D. program now! 🎓
We only train once: How can we effectively customize a robot for multiple users? We propose 'Promptable Behaviors', a novel personalization framework that deals with diverse preferences without any retraining. website: promptable-behaviors.github.io paper: arxiv.org/abs/2312.09337 🧵👇
@ylecun @LerrelPinto Just to build on this a bit, using learning + planning algorithms for perception is something I've worked on quite a bit before coming to FAIR. A lot more research here is needed to deal with long horizons and noise in my opinion, very naive stuff works for e.g. half cheetah envs
Trends for United States
You might like
