dheeru dua @ddua17
Joined April 2014-
Tweets32
-
Followers149
-
Following111
-
Likes254
I've been asked what's the biggest thing in 2024 other than LLMs. It's Robotics. Period. We are ~3 years away from the ChatGPT moment for physical AI agents. We've been cursed by the Moravec's paradox for too long, which is the counter-intuitive phenomenon that "tasks that humans…
What do you know, when you make comparable experiments, ICL and fine-tuning (FT) aren't that different! We find that in- and out-of-domain few-shot learning works well with FT, if not better than ICL, at scale. Check out our recent work, led by @mariusmosbach the great.
What do you know, when you make comparable experiments, ICL and fine-tuning (FT) aren't that different! We find that in- and out-of-domain few-shot learning works well with FT, if not better than ICL, at scale. Check out our recent work, led by @mariusmosbach the great.
F in RLHF is overall preference, which conveys limited info🙁 We introduce Fine-Grained RLHF🚀and train LMs with explicit feedback like "sentence 1 is not factual", "sentence 2 is toxic" More effective & enables LM customization arxiv.org/abs/2306.01693 finegrainedrlhf.github.io
F in RLHF is overall preference, which conveys limited info🙁 We introduce Fine-Grained RLHF🚀and train LMs with explicit feedback like "sentence 1 is not factual", "sentence 2 is toxic" More effective & enables LM customization arxiv.org/abs/2306.01693 finegrainedrlhf.github.io https://t.co/kYHRrn3tlL
Do we need Attention? (v0 github.com/srush/do-we-ne…): Slides for a survey talk summarizing recent Linear RNN models with a focus on NLP. Tries to cover a lot of different S4-related models (as well as RWKV/MEGA) in a digestible way.
Under-rated is how hard it is to create datasets that stand the test of time. And DROP from my labmate @ddua17 has done just that, as the only academic benchmark GPT-4 doesn't get SOTA on. 4(!!) years since release, no method has hit "human performance" (~96 F1).
Under-rated is how hard it is to create datasets that stand the test of time. And DROP from my labmate @ddua17 has done just that, as the only academic benchmark GPT-4 doesn't get SOTA on. 4(!!) years since release, no method has hit "human performance" (~96 F1). https://t.co/W1dQ0Z4qlO
🚨 Negation misunderstanding in IR systems can lead to dire outcomes, as seen in the quoted Google Search example from 2021. But are SOTA IR models any better now? 🔍 Spoiler: nearly all IR models perform worse than random! #IR #NLProc
This was a truly amazing year for #NLProc, and I tried my best to summarize it as well as I could. Thank for you the invitation, @samcharrington! Here's an annotated bibliography of the stuff I mentioned, warning: long 🧵
This was a truly amazing year for #NLProc, and I tried my best to summarize it as well as I could. Thank for you the invitation, @samcharrington! Here's an annotated bibliography of the stuff I mentioned, warning: long 🧵
Large language models generate impressively fluent content across a variety of tasks, but do so without citing their sources even when such content is about the external world. 1/
A lot of machine learning research has detached itself from solving real problems, and created their own "benchmark-islands". How does this happen? And why are researchers not escaping this pattern? A thread 🧵
#NLPaperAlert: QA Dataset Explosion!🔥 A survey of 200+ QA/RC datasets proposing a taxonomy of formats & reasoning skills. Also in the bag: modalities, conversational QA, domains & beyond-English data. Honored to work on this with @nlpmattg & @IAugenstein arxiv.org/abs/2107.12708
Sharing some takeaways from Learning Neural Network Subspaces, a fun project we learned a lot from and hopefully you can too. (1/n) tldr: We train lines, curves, and simplexes of neural networks from scratch arxiv: arxiv.org/abs/2102.10472 code: github.com/apple/learning… #ICML2021
The difference between information and knowledge:
Equations that changed the world. 🧠
Some people say that one shouldn't care about publication and the quality matters. However, the job market punishes those who don’t have publications in top ML venues. I empathize with students and newcomers to ML whose good papers are not getting accepted. #ICLR2021 1/
Today we continue the 2020 AI Rewind series, joined by @sameer_ Singh, who helped us break down NLP in 2020 into 4 categories, Massive Language Modeling, Fundamental Problems with Language Models, Practical Vulnerabilities with LMs, and Evaluation. twimlai.com/trends-in-natu…
This week we went through the second part of my lecture on latent variable 👻 energy 🔋 based models. 🤓 We've warmed up a little the temperature 🌡, moving from the freezing 🥶 zero-temperature free energy Fₒₒ(y) (you see below spinning) to a warmer 🥰 Fᵦ(y).
This week we went through the second part of my lecture on latent variable 👻 energy 🔋 based models. 🤓 We've warmed up a little the temperature 🌡, moving from the freezing 🥶 zero-temperature free energy Fₒₒ(y) (you see below spinning) to a warmer 🥰 Fᵦ(y). https://t.co/wAzbDdNRjB
Evaluating NLP Models via Contrast Sets New work that is a collaboration between 26 people at 10 institutions (!) arxiv.org/abs/2004.02709 Trying to tag everyone at the top of the thread, here it goes:
1/5 Self-Distillation loop (feeding predictions as new target values & retraining) improves test accuracy. But why? We show it induces a regularization that progressively limits # of basis functions used to represent the solution. bit.ly/2HnOACo w/@farajtabar P.Bartlett
#nlphighlights 105: Question Generation with @raosudha89. We discussed an overview of the settings in which you would want to generate questions, and focused on Sudha's work of generating clarification questions. Thanks for joining us, Sudha! soundcloud.com/nlp-highlights…
Michael Xieyang Liu @lxieyang
1K Followers 2K Following Research Scientist @GoogleAI People + AI Research. HCI + AI + Programming Support + Sensemaking. ex @SCSatCMU @UMich @MSFTResearch @GoogleAI. He/him
Preethi Seshadri @Preethi__S_
163 Followers 900 Following Student Researcher interested in ethical and societal issues around language and multimodal models + data and model evaluation 💜💛🏀
Vivek Gupta @keviv9
2K Followers 5K Following PostDoc @cogcomp UPenn | Ph.D. CS @UUtah | @iitkanpur. @Bloomberg & @MSFTResearch Fellow | ex-@MetaAI, @IBM, @Verisk, @samsungresearch, @Synopsys #nlp #ml
Yu Fei @Walter_Fei
77 Followers 223 Following PhD student @UCIrvine working on NLP/ML. Previously: ms @ETH, undergrad @PKU1898
Mimansa Jaiswal @MimansaJ
1K Followers 3K Following MoTS @normativeai. Ex @UMichCSE, 2x @MetaAI, @allen_ai | Speech & NLP | Robustness, Data & Annotations, Evaluation & Interpretability in LLMs
~/.asparagusbo/ @haskelloween
644 Followers 2K Following
Bob Aman @sporkmonger
903 Followers 2K Following He/him. I want to make today more just than yesterday. Automated security decision-making under uncertainty. May I pet your dog? @[email protected]
Jessica De Freitas @jeskdef
216 Followers 378 Following PhD Student Electronic health records and Machine learning 👩🏽💻
Prasanna Sattigeri @prasatti
458 Followers 2K Following Principal Research Scientist @IBMResearch and @MITIBMLab.
taesiri @taesiri
529 Followers 4K Following PhD Student at UofA, Working on Large Multimodal Models.
Guillaume Pitel @PitZeGlide
913 Followers 3K Following Co-founder/CTO @BabbarTech #SEO https://t.co/LykgUokVzH
Prasann Singhal @prasann_singhal
141 Followers 681 Following 3rd-year undergrad #NLProc Researcher at UT Austin, advised by @gregd_nlp
Hung-Ting Chen @ EMNL.. @hungting_chen
146 Followers 241 Following PhD student in @UTCompSci, working on NLP.
Albert Xu @albertxu__
460 Followers 508 Following PhD Student @USC, @nlp_usc. Prev. @UCBerkeley, @BerkeleyNLP
Ameya Godbole @ameya_godbole1
151 Followers 250 Following PhD student @nlp_usc working on generalization and reasoning, prev @UMassAmherst, @iitg (he/him)
Geetanjali Rakshit @geet2708
26 Followers 325 Following
Shivanshu Gupta @shivanshug11
165 Followers 93 Following PhD Candidate at UC Irvine | Previously @asapp @amazon @linkedin @msftresearch @iitdelhi | #NLP & #ML Research
Dr. Smarty Pants @DrSmartyPants44
138 Followers 5K Following Recovering astrophysicist, now a deep-learner
DrSmartyPants49 @DrSmartyPants49
81 Followers 5K Following
DrSmartyPants46 @DrSmartyPants46
89 Followers 5K Following Adventurer. Explorer. Seeker of ancient mysteries and hidden treasures. Join me on thrilling expeditions as we uncover the secrets of the world.
Mr. Money Bags @drsmartypants42
138 Followers 5K Following Harnessing the twitter finance community to make money ;)
Nishant Raj @Nishant95R
60 Followers 669 Following Applied Science @ Microsoft | UMass Amherst | IIT Roorkee
Yasumasa Onoe @yasumasa_onoe
339 Followers 281 Following Software Engineer @GoogleAI working on vision and language research
JiaxinZhang @KnightZhang625
121 Followers 447 Following CS PhD student in University of Strathclyde, Glasgow, #NLP #NumericalReasoning. Master degree from University of Sheffield. Make the world better place.
Yasaman Razeghi @yasaman_razeghi
541 Followers 406 Following PhD student in UC Irvine, researching on NLP/ML Student Researcher at Google-Deepmind
Muhammad Anwar @Muhamma81966391
277 Followers 3K Following Graphs Graphon. DL Researcher at @Proteinea
Ashish Awasthi @ashishawasthi
429 Followers 1K Following technologist, explorer, developer. keen on science and technology. working on #machinelearning
Bashar Alhafni @balhafni
454 Followers 1K Following CS PhD student @nyuniversity. Previously: @Grammarly, @Dataminr, @USC_ISI
Rakesh Chada @RakC3
127 Followers 479 Following Applied Scientist at Amazon Alexa. Opinions my own :)
Vikas Yadav @Vikas_NLP_UA
204 Followers 554 Following Staff Research Scientist at SN, PhD from @uarizona @LabCLU, research interests- GenAI, LLM alignment, RLHF, QA, IR, compression, robustness and explainability.
CLS @ChengleiSi
2K Followers 3K Following vibing @stanfordnlp | real AGI is the friends we made along the way
Dataf3l @dataf3l
64 Followers 918 Following
Kourosh Meshgi クー.. @KouroshMeshgi
688 Followers 2K Following Research Scientist @RIKEN_AIP_EN, ML/CV/NLP and Robotics
Robin @robinwritescode
423 Followers 4K Following Leading Product for CoreML Group @yelp Prev: ML Researcher @McMasterU, ML EM @etsy | Investing and Building at the Frontier @frontier_fund
Zé R. Fernandes @ezerfernandes
503 Followers 2K Following Jesus the Messiah, Son of God, have mercy on me, a broken man. How abundant is your goodness, Oh God! which you have stored up for those who fear you!
Björn Holzhauer @BjornHolzhauer
194 Followers 440 Following Biostatistician in drug development - quite Bayesian. I also do deep learning & chess. Our geese let me share their garden. @[email protected]
Krishna Srinivasan @krishna2
475 Followers 2K Following I code stuff. I build models. I work in NLP/DL at Google Research.
Avijit Thawani (Avi) @thawani_avijit
838 Followers 1K Following Graduating PhD @USC_ISI. LLMs/GenAI. Fintech Founding MLE. Filmmaker 100k+ views. Lived in UK, Singapore, India, US. ex: Microsoft Research, Amazon Alexa, AI2.
Julian Harris @julianharris
2K Followers 4K Following 30 years making internet software: ex-Googler & ex-founder. AI engineer since early 2023 obsessing about how AI and robotics can help climate change somehow.
Peyman Milanfar @docmilanfar
67K Followers 264 Following Distinguished Scientist at Google Research. Computational Imaging, Machine Learning, and Vision. Tweets = personal opinions. May change or disappear over time.
AK @_akhaliq
310K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo follow on Hugging Face: https://t.co/q2Qoey80Gx
Armen Aghajanyan @ArmenAgha
6K Followers 263 Following Research Scientist @ Meta AI (FAIR) https://t.co/8XF2vtiIVy Opinions are my own.
Been Kim @_beenkim
23K Followers 453 Following Research Scientist at Google DeepMind, PhD from MIT. Make machines empower people. @[email protected]
Demis Hassabis @demishassabis
357K Followers 125 Following Co-founder & CEO @GoogleDeepMind - working on AGI. Trying to understand the fundamental nature of reality. Also revolutionising drug discovery @IsomorphicLabs
Sanjeev Arora @prfsanjeevarora
21K Followers 32 Following Director, @PrincetonPLI and Professor @PrincetonCS. Seeks math/conceptual understanding of deep learning and large AI models.
@emilymbender@dair-co.. @emilymbender
58K Followers 2K Following Prof, Linguistics, UW // Faculty Director, CLMS // she/her // @[email protected] & bsky // rep by @ianbonaparte
Stat.ML Papers @StatMLPapers
20K Followers 0 Following Unofficial updates of statistical machine learning papers on arXiv
Ming-Wei Chang @mchang21
1K Followers 510 Following Research Scientist @GoogleDeepMind. BERT co-author. Gemini project.
Partha Talukdar @partha_p_t
4K Followers 215 Following Researcher @googleai, Faculty @iiscbangalore, Founder @kenomeio
Manish Gupta @ManishGuptaMG1
4K Followers 72 Following Dr. Manish Gupta is the Director of Google Research India, and the Infosys Foundation Chair Professor at IIIT Bangalore.
Daniela Witten @daniela_witten
49K Followers 754 Following dorothy gilford endowed chair and prof of stat/biostat @uw. all views my own.
Prof. Feynman @ProfFeynman
1.4M Followers 0 Following A universe of atoms, an atom in the universe. Tribute to the great explainer. Tweets about Science and Wisdom. Portrait by L.V Patten.
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Dylan Slack @dylanslack20
566 Followers 565 Following Researcher at ... Ph.D. @UCIbrenICS. Prev @awscloud and @googleAI. I tweet about misc findings + plug my papers
Allen Institute for A.. @allen_ai
54K Followers 361 Following AI for the Common Good. › Join us: https://t.co/DqTs1G4bGO › Get our newsletter: https://t.co/tvb1VpySfL
Lex Fridman @lexfridman
3.5M Followers 126 Following Host of Lex Fridman Podcast. Interested in robots and humans.
Gary Marcus @GaryMarcus
145K Followers 7K Following “A beacon of clarity”. Spoke at US Senate AI Oversight committee. Founder/CEO Geometric Intelligence (acq. by Uber). Rebooting AI & Taming Silicon Valley.
Eric Wallace @Eric_Wallace_
6K Followers 1K Following Researcher at OpenAI working to make language models more trustworthy, secure, and private.
Nitish Gupta @nitish_gup
1K Followers 483 Following Research Scientist at @GoogleAI in Natural Language Processing & Machine Learning PhD from University of Pennsylvania | Undergrad from IIT Kanpur
Sanjay Subramanian @sanjayssub
746 Followers 532 Following Building/analyzing NLP and vision models. PhD student @berkeley_ai. Formerly: @allen_ai, @penn
Judea Pearl @yudapearl
76K Followers 188 Following Student of causal inference, human reasoning, and history of ideas, all viewed through the sharp lens of artificial intelligence.
Mike Lewis @ml_perception
6K Followers 227 Following Llama3 pre-training lead. Partially to blame for things like the Cicero Diplomacy bot, BART, RoBERTa, kNN-LM, top-k sampling & Deal Or No Deal.
zhou Yu @Zhou_Yu_AI
9K Followers 837 Following Associate Professor at Columbia, advancing the frontier of NLP. Forbes 30 under 30. Amazon Alexa Prize winner.
arXiv Daily @Arxiv_Daily
48K Followers 2K Following Daily feed of this week's top research articles published to https://t.co/ULrW4yLt6n. AI Research Papers, Curated by @DeepAI
Milind Tambe @MilindTambe_AI
8K Followers 270 Following @Harvard Professor & Director Ctr for Computation & Society @HCRCS @GoogleAI Principal Scientist & Director #AIforSocialGood #AIforhealth #AIforConservation
Jason Weston @jaseweston
9K Followers 569 Following Research @MetaAI+NYU. Pretrain+FT: NLP from Scratch (2011). Multilayer attention+position embed+LLM: MemNets (2015). Recent (2023+):Sys 2 Attn, Self-Rewarding..
Pradeep Dasigi @pdasigi
1K Followers 460 Following Senior Research Scientist at Allen Institute for AI (AI2)
Max Welling @wellingmax
32K Followers 432 Following
Tal Linzen @tallinzen
16K Followers 893 Following Professor @nyuling and @NYUDataScience, research scientist @GoogleAI
Greg Durrett @gregd_nlp
6K Followers 752 Following CS professor at UT Austin. I do NLP most of the time. he/him
ZotBot @alexaucirvine
9 Followers 13 Following using AI to help alexa talk to you @UCIrvine's 2020 #AlexaPrize social bot competition team
Yejin Choi @YejinChoinka
19K Followers 330 Following professor at UW, director at AI2, adventurer at heart
Dan Jurafsky @jurafsky
27K Followers 297 Following Professor of linguistics and professor of computer science at Stanford and author of the James Beard award finalist "The Language of Food"
Anthony Platanios @eaplatanios
603 Followers 462 Following Research Scientist at Scaled Cognition. Previously Semantic Machines (@Microsoft) and @mldcmu.
Tim Rocktäschel @_rockt
29K Followers 1K Following Open-Endedness Team Lead @GoogleDeepMind, Professor of AI @AI_UCL, PI @UCL_DARK, @ELLISforEurope Scholar. ex @MetaAI (FAIR), @CompSciOxford. Opinions my own.
Thomas Wolf @Thom_Wolf
68K Followers 4K Following Co-founder and CSO @HuggingFace - open-source and open-science
ML Limericks @MLimericks
2K Followers 46 Following Limericks about machine learning. Originally by @fhuszar
Matthew Mercer @matthewmercer
935K Followers 2K Following Storyteller, VO Guy, Vincent Valentine, Minsc, Thirst Trap Ganondaddy, #TheLegendOfVoxMachina, CCO/DM of @CriticalRole. He/Him. Icon-@sephiramy.This paper by Feller is a key starting point for diffusion approaches & a great illustration of two principles of effective writing - It doesn’t obfuscate ideas with any more math than you absolutely need - It’s clear, narrative, expository writing anchored in helpful examples
Since these experiments have been popular, here is a recap that will be from now the thread for updates. The motivation for all this came from discussions at @neurips_conf with @tri_dao, @_albertgu, and @srush_nlp.
@SudhirK_Algrow This is so true. Even graduates from prestigious universities with impressive profiles are facing challenges in the job market and accepting offers that don't match their worth.
I've been asked what's the biggest thing in 2024 other than LLMs. It's Robotics. Period. We are ~3 years away from the ChatGPT moment for physical AI agents. We've been cursed by the Moravec's paradox for too long, which is the counter-intuitive phenomenon that "tasks that humans…
Ok, by popular demand: a starter set of papers you can read on the topic. "Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks": arxiv.org/abs/2311.09247 "Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to…
This Monday I gave the last lecture in my course on explainability 🫡 Readings/slides are available at utah-explainability.github.io + lecture recordings at youtube.com/playlist?list=… 🎬
Q-Learning is *probably* not the secret to unlocking AGI. But, combining synthetic data generation (RLAIF, self-instruct, etc.) and data efficient reinforcement learning algorithms is likely the key to advancing the current paradigm of AI research… TL;DR: Finetuning with…
A recent LLM hallucination benchmark is making rounds, and people are jumping to conclusions based on a table screenshot. The eval is so problematic in many ways. In fact, a trivial baseline can achieve 0% on hallucination. I cannot help but don my Peer Reviewer hat: - The study…
a number of weird definitions and weirdly specific points, but overall, worth reading it to see which areas are considered as priorities by WH. in this 🧵, let me copy-paste those few weird/interesting/specific points i found reading it. whitehouse.gov/briefing-room/…
I’m just an average researcher. But something I’ve learned in 30 years of being a researcher is that if you’ve convinced yourself at age 25 that you can teach others how to be great researchers, you’ve still got a lot to learn.
Enjoyed visiting UC Berkeley’s Machine Learning Club yesterday, where I gave a talk on doing AI research. Slides: docs.google.com/presentation/d… In the past few years I’ve worked with and observed some extremely talented researchers, and these are the trends I’ve noticed: 1. When…
My student sent me this list saying they have to improve themselves in many areas. Such a list can do more harm than good. While I appreciate author's intention to motivate one for greatness, I don't think it can be planned. But you can plan to be a "good researcher."
Enjoyed visiting UC Berkeley’s Machine Learning Club yesterday, where I gave a talk on doing AI research. Slides: docs.google.com/presentation/d… In the past few years I’ve worked with and observed some extremely talented researchers, and these are the trends I’ve noticed: 1. When…
Israeli in the US. Thoughts running circles in my head for days.
Excited to share work from my FAIR internship on understanding the effects of RLHF on LLM generalisation and diversity: arxiv.org/abs/2310.06452 While RLHF outperforms SFT in-distribution and OOD, this comes at the cost of a big drop in output diversity! Read more below🧵
A tutorial on neural theorem proving: github.com/wellecks/ntptu… Interactive notebooks for learning about combining neural language models with formal proof assistants. Part I) Build and evaluate a next-step suggestion tool Part II) LLM cascades and Draft, Sketch, Prove
guilty; i had never heard of expectile regression until arxiv.org/abs/2110.06169, and had to spend 15 minutes to understand the loss function: kyunghyuncho.me/expectile-regr…
For some reason, everyone in my mentions is talking about how much they love @tqchenml 's course on DL systems dlsyscourse.org/lectures/ . While I am, of course, a bit jealous, it sounds like a must-take course.
Back-propagation for feed-forward architectures is a special case of reverse-mode automatic differentiation. It computes the gradient of a loss function with a cost comparable to the computation of the loss function itself. en.wikipedia.org/wiki/Backpropa…
🚨New paper🚨 "The Bias Amplification Paradox in Text-to-Image Generation" Paper: yanaiela.github.io/papers/bias-am… Does Stable Diffusion amplify gender-occupation biases present in the training data? Well, the answer is nuanced.
Introducing 🎙️DialogStudio🎙️, the largest and most diverse dialogue dataset collection with diverse goals （e.g. task-oriented, open-domain, NLU, etc.) and different domains (e.g. finance, insurance software, movie, etc.) github.com/salesforce/Dia… huggingface.co/datasets/Sales… #NLP #AI
Excited to share our latest survey paper: "Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems"! 🚀 ArXiv: arxiv.org/abs/2307.08423.
Trends for United States
You might like
