Haoxiang Wang @Haoxiang__Wang
Final-year Machine Learning PhD candidate from UIUC. Will join NVIDIA as a research scientist. Past intern at Apple/Amazon/Waymo. haoxiang-wang.github.io Champaign, IL Joined August 2014-
Tweets605
-
Followers898
-
Following925
-
Likes3K
SnapKV LLM Knows What You are Looking for Before Generation Large Language Models (LLMs) have made remarkable progress in processing extensive contexts, with the Key-Value (KV) cache playing a vital role in enhancing their performance. However, the growth of the KV
Check out our latest SOTA open-source reward model based on LLaMA3-8B-it! The RM readily serves to provide signals for subsequent iterative RLHF, see a demo in huggingface.co/sfairXC/Fsfair… which improves zephyr-set with alpaca lc win rate 8% to 34.79%
Check out our latest SOTA open-source reward model based on LLaMA3-8B-it! The RM readily serves to provide signals for subsequent iterative RLHF, see a demo in huggingface.co/sfairXC/Fsfair… which improves zephyr-set with alpaca lc win rate 8% to 34.79%
🚀 Launching our BPO (Bootstrapped Preference Optimization)! 🤔️ MLLM based on pretrained LLMs demonstrate pretraining bias problem. ✅ We design strategies to bootstrap preference data from the MLLM, which is used to improve itself.
New open-source release for the Mistral AI Hackathon Mistral 7B v0.2 Base: models.mistralcdn.com/mistral-7b-v0-… - 32k context window - Rope Theta = 1e6 - No sliding window This is the raw pretrained model behind Mistral-7B-Instruct-v0.2 Also, new fine-tuning repo: github.com/mistralai-sf24…
Weights drop ⚠️ We released our pre-trained model for the cup arrangement task trained on 1400 demos! We aim to enable anyone to deploy UMI on their robot to arrange any "espresso cup with saucer" they buy on Amazon. github.com/real-stanford/…
TiC-CLIP is accepted at #ICLR2024. Now releasing the code, camera ready and new results. A benchmark and methods for continual pretraining of large image-text models Code for train/eval and data: github.com/apple/ml-tic-c… Paper: arxiv.org/abs/2310.16226 openreview.net/forum?id=TLADT…
TiC-CLIP is accepted at #ICLR2024. Now releasing the code, camera ready and new results. A benchmark and methods for continual pretraining of large image-text models Code for train/eval and data: github.com/apple/ml-tic-c… Paper: arxiv.org/abs/2310.16226 openreview.net/forum?id=TLADT…
Some text data is private & cannot be shared... Can we generate synthetic replicas with privacy guarantees?🤔 Instead of DP-SGD finetuning, use Aug-PE with inference APIs! Compatible with strong LLMs (GPT-3.5, Mistral), where DP-SGD is infeasible. 🔗alphapav.github.io/augpe-dpapitext [1/n]
We just released the code for our #ICLR2024 publication PGDVS and hope this can spur more efforts towards a generalized dynamic novel view synthesis, making the immersive experience more affordable. - paper: arxiv.org/abs/2310.08587 - code: github.com/apple/ml-pgdvs
Excited that LM-Infinite has been accepted into #NAACL2024 ! It is the first-of-its-kind zero-shot length generalizations for language models, with 200M length inference and downstream (Retrieval, Qasper) improvements! Great thanks to all my collaborators! arxiv.org/abs/2308.16137
Happy to share R-Tuning got accepted to #NAACL2024 main! We introduce Refusal-Aware Instruction Tuning to tackle hallucination in LLMs. So that the LLMs could say I Don't Know now! Goal: Alignment for Honesty Paper: arxiv.org/abs/2311.09677
Very happy to get 9 papers accepted by NAACL2024, especially Chi Han’s paper has got multiple perfect review scores. This method can generalize LLM to length of 200M. Chi will be on academic job market next year! arxiv.org/abs/2308.16137
[1/4] So, I decided to seriously use JAX, and it didn't take long for me to realize its power. With just a couple hundred lines of code, you can do data&tensor parallelism on @huggingface transformers. I've created a toolkit to make this more accessible. github.com/luyug/magix
Today while testing @AnthropicAI 's new model Claude 3 Opus I witnessed something so astonishing it genuinely felt like a miracle. Hate to sound clickbaity, but this is really what it felt like. Important context: I've been working on NLP for my mother tongue - the Circassian…
Are you interested in SOTA compact CLIP models? 🚀🚀 Check out our open-sourced repo for a family of MobileCLIP models, including a ViT-B@224 with 77.2% IN-top1 accuracy. More highlights in 🧵 Paper (appearing in CVPR 2024): arxiv.org/abs/2311.17049 Repo: github.com/apple/ml-mobil…
**Training dynamics of attention** 1/📜Introducing our latest paper: "Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality." Link: [arxiv.org/abs/2402.19442] Joint work with @siyuc3141, @HeejuneSheen, and @0920wth
I am really excited to reveal what @GoogleDeepMind's Open Endedness Team has been up to 🚀. We introduce Genie 🧞, a foundation world model trained exclusively from Internet videos that can generate an endless variety of action-controllable 2D worlds given image prompts.
Curious about how severe the alignment tax is on LLMs' general capabilities? Eager to mitigate the alignment tax? We explored a frustratingly easy approach: Model Averaging. It's astonishingly effective, outperforming numerous baselines! 🔎Paper: arxiv.org/abs/2309.06256
Amazed by how fast Groq is? Want to make your LLM inference even faster? We propose Cascade Speculative Drafting, a speculative execution algorithm that comprises multiple draft models through cascades, achieving up to an 81% additional speedup over speculative decoding in our…
New (2h13m 😅) lecture: "Let's build the GPT Tokenizer" Tokenizers are a completely separate stage of the LLM pipeline: they have their own training set, training algorithm (Byte Pair Encoding), and after training implement two functions: encode() from strings to tokens, and…
🚀 Iterative DPO is efficient theoretically and empirically! 🚀 We've got extensive empirical support for GSHF now! 📊 Joint work with Wei @weixiong_1, Chenlu @ye_chenlu, Ziqi @wzq016, Han @han_zhong1, Heng @elgreco_winter, Nan @nanjiang_cs , Tong arxiv.org/abs/2312.11456
Secure Learning Lab (.. @uiuc_aisecure
940 Followers 289 Following We are a computer science research group led by Bo Li at UIUC, focusing on responsible and trustworthy machine learning.
Han Zhao @hanzhao_ml
3K Followers 1K Following Assistant Professor @IllinoisCS; Ph.D. @mldcmu; Interested in machine learning and AI.
Bill Yuchen Lin 🤖 @billyuchenlin
6K Followers 2K Following Research @allen_ai. I evaluate (multi-modal) LLMs, build agents, and study the science of LLMs. Previously: @GoogleAI & @MetaAI FAIR @nlp_usc
Yao Fu @Francis_YAO_
14K Followers 2K Following PhD @EdinburghNLP on LLMs and Machine Reasoning. Ex. @Columbia @PKU1898 @MITIBMLab @allen_ai AGI has yet to come, so keep running
Ananya Kumar @ananyaku
4K Followers 471 Following Researcher at @openai Previously PhD at Stanford University (@StanfordAILab) advised by Percy Liang and Tengyu Ma
Yiping Lu @2prime_PKU
3K Followers 2K Following Kernel, ML for PDE, Robust learning,non-parametric stats/🌈/PKU👉Stanford👉NYU Courant👉Northwestern IEMS/ Previous Intern @RIKEN_AIP
Jason Lee @jasondeanlee
10K Followers 3K Following Associate Professor at Princeton and Research Scientist at Google DeepMind. ML/AI Researcher working on foundations of LLMs and deep learning
Furong Huang @furongh
4K Followers 2K Following Assistant professor of @umdcs @umiacs @ml_umd at UMD. Researcher in #AI/#ML, #Trustworthy AI/ML, #EthicalAI, AI #Democratization, AI for ALL.
Dinghuai Zhang 张鼎.. @zdhnarsil
2K Followers 1K Following PhD student at @Mila_Quebec. Ex intern at FAIR Labs @MetaAI. Previous math undergraduate at @PKU1898.
Tianyin Xu @tianyin_xu
4K Followers 998 Following Watchman in a cornfield @IllinoisCS @ECEILLINOIS @ACMSIGOPS
Jian Kang @jiank_uiuc
1K Followers 844 Following Assistant Professor of Computer Science @UofR | PhD @IllinoisCS | Ex-intern @MetaAI Trying to make graph learning reliable
Xiaolong Wang @xiaolonw
11K Followers 955 Following Assistant Professor @UCSDJacobs Postdoc @berkeley_ai PhD @CMU_Robotics
Linyi Li @limyikli
293 Followers 354 Following Researcher in ML & Security https://t.co/ya677rH62z Alumni @ UIUC & Tsinghua he/him/his
Huaxiu Yao @HuaxiuYaoML
3K Followers 527 Following Assistant Professor of Computer Science @UNC @unccs @uncsdss | Postdoc @StanfordAILab | Ph.D. @PennState | #foundationmodels, #AISafety, #AIforScience | he/him
DK Xu @DongkuanXu
2K Followers 2K Following Assistant Professor @NCState. Co-Founder @GentopiaAI. Artificial General Intelligence. Ex- @MSFTResearch, @ https://t.co/JuUn6gRp78, @NECLabsAmerica. Big Fan of @NFL.
Qi Lei @Qi_Lei_
3K Followers 1K Following Assistant professor in Math and Data Science, NYU, Postdoc at Princeton ECE, PhD from UT Austin, interested in machine learning, deep learning and optimization
Cawmpoy @cawmpoy52398
0 Followers 51 Following
EthelJerome @W5u47rdp489Z3
0 Followers 100 Following
Eddy Emmanuel @youngboi_eddy
107 Followers 427 Following Machine learning //Artificial intelligence//crypto enthusiast. GitHub: https://t.co/pLyM6JSfh5 LinkedIn:https://t.co/iLoDYlwIXD
Zabir Al Nazi Nabil @PseudoEmpirical
58 Followers 261 Following Self-taught SWE, Open Source Enthusiast & Contributor, Sci-Fi Connoisseur. Interested in AGI, LLM, XAI. CS PhD Student @UCRiverside
Jason Pho 🔜 GDC @Jsavetheworld
154 Followers 599 Following @miHoYo | Love good stories, community, helpful tech | Enjoy finding good questions
Wei Xiong @weixiong_1
186 Followers 176 Following PhD Student @IllinoisCS, Practice Math for 2.5 Years
Vishal Goklani @vgoklani_ai
621 Followers 5K Following Twitter Nerd... Interested in Deep Learning (self-supervised learning & LLMs), Astrophysics (exoplanets), and Cosmology (CMB).... I like to build things
Martin Fan @perfectoid_ai
393 Followers 8K Following
Zhao XU @BillHsu98
25 Followers 165 Following MPhil of @hkust | Previously: Bachelor of @Tsinghua_Uni | Research intern @AlibabaGroup DAMO Academy | Visiting scholar @penn_state | A lifelong learner
Zhuokai Zhao @zhuokaiz
2 Followers 21 Following Final-year CS PhD Candidate at @UChicago. Research in data-centric and trustworthy ML. Previously @Meta, @Twitch, @Siemens, @HopkinsEngineer, @ECEILLINOIS.
Yifeng Ding @YifengDing_
233 Followers 580 Following Ph.D. student @IllinoisCS. Interested in Large Language Models for Code.
li ii iq j @iq_li80427
47 Followers 320 Following
Pensé FFun @inftyCategory
100 Followers 6K Following
de jia @dejia49220082
21 Followers 817 Following
_Lysandra @Lysandr38860865
3 Followers 563 Following
CstlCscd_37 @cstlcscd40015
6 Followers 358 Following
Slofouski @slofouski90669
123 Followers 2K Following
Shaobo Wang @ShaoboWang6
129 Followers 668 Following CS Master @sjtu1896 | Life can only be understood backwards; but it must be lived forwards.
Shyam Pathade @Shyamptwt
209 Followers 273 Following 🎛️ Al Apprentice | Model Tinkerer |Learning insights and algorithms
Roger Luo 罗秀哲 @rogerluorl18
663 Followers 485 Following PhD student in University of Waterloo. Associate graduate student in Perimeter Institute.
leloy @leloykun
831 Followers 4K Following ex-ML Research Eng. @ExpedockAI • 2x IOI & 2x ICPC World Finalist • Multi-Modal ML • Document Information Extraction • Non-Euclidean Geometry • Math @ AdMU
Xiyang Wu @wu_xiyang
254 Followers 904 Following Ph.D. Student at @gammaumd @eceumd @umiacs @UofMaryland. Previous: @EmoryUniversity @GeorgiaTech @TJU1895.
Hussein Dia @HusseinDia
3K Followers 3K Following Future Mobility Professor. Transport Technology & Decarbonisation. Researching Innovations for Sustainable Transport.
채원에밀리_Chaew.. @Chaewonemily7
85 Followers 1K Following 예수의 전문적인 마약중독자 † Fed에서 기업가로 변신 서번트 리더 @instagram🇺🇸 채원 에밀리 #채원밀리 chaewon Emily
Ross @ma1547372858
15 Followers 1K Following
Jiacheng Lin @jclin808
56 Followers 173 Following CS PhD student @ UIUC, advised by Prof. @jimeng; NLP&Bio&Healthcare; Undergrad&MS @Tsinghua_Uni; Intern@UW, Microsoft Research AI4Science, MSRA
Sadia Afrin Purba @sadiaafrinpurba
48 Followers 633 Following All in all I'm just another brick in the wall
HaoyueBai @haoyue_bai
937 Followers 839 Following Ph.D. student at Computer Science Department @UWMadisonCS, MPhil @HKUSTCSE.
Mo Zhou @MoZhou_7
28 Followers 177 Following CS PhD student at @DukeU, working on deep learning theory and non-convex optimization
NEC Labs America @NECLabsAmerica
591 Followers 2K Following @NEC Labs America delivers high-impact #technology #research. Located in Princeton, NJ & San Jose, CA. #AI #MachineLearning #DataScience #OpticalNetworking
Ryannnsi @Ryannnsi1
4 Followers 94 Following
Xuheng Li @xuhengli_
311 Followers 807 Following CS PhD student @UCLA, supervised by @QuanquanGu | RL, deep learning theory, diffusion model | Previously BSc @PKU1898 | Stargazer
Yu Meng @yumeng0818
1K Followers 160 Following Asst. Professor @CS_UVA, Past: PhD from @IllinoisCS, visiting researcher @princeton_nlp, Google PhD Fellow. NLP/ML/LLM
LIWEI WANG @LIWEIWANG_HR
12 Followers 245 Following
Hung Le @lqh_4rt3mis
68 Followers 269 Following
Harry Tran @harrytraneta
180 Followers 856 Following Solopreneur making: Multipurpose online forms and document merges https://t.co/8UO0S5fL5X
Quan Xiao @QuanXiao8
30 Followers 87 Following PhD student in ECSE at Rensselaer Polytechnic Institute, optimization and machine learning
Shenao Zhang @ShenaoZhang
269 Followers 965 Following PhD student @NorthwesternU | Student Researcher @MSFTResearch. Ex-intern @MSFTReserch, ByteDance, and Tencent AI | Previously @GeorgiaTech. LLM, RL, agent.
Yi Ma @YiMaTweets
71K Followers 123 Following Chair Professor in AI, Director of IDS, Head of CS, HKU; Professor of EECS, Berkeley; Author of Book: High-Dim Data Analysis, https://t.co/gwaqMJp8av.
Yann LeCun @ylecun
711K Followers 718 Following Professor at NYU. Chief AI Scientist at Meta. Researcher in AI, Machine Learning, Robotics, etc. ACM Turing Award Laureate.
AK @_akhaliq
310K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo follow on Hugging Face: https://t.co/q2Qoey80Gx
Jim Fan @DrJimFan
229K Followers 3K Following @NVIDIA Sr. Research Manager & Lead of Embodied AI (GEAR Lab). Creating foundation models for Humanoid Robots & Gaming. @Stanford Ph.D. @OpenAI's first intern.
Gautam Kamath @thegautamkamath
44K Followers 507 Following Assistant Prof of CS @UWaterloo, Faculty @VectorInst, Canada @CIFAR_News AI Chair. Co-EiC @TmlrOrg. I lead @TheSalonML. Privacy, robustness, machine learning.
Jia-Bin Huang @jbhuang0604
51K Followers 285 Following Associate Professor @umdcs; Part-time Research Scientist @Meta. I like pixels.
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Yuandong Tian @tydsh
16K Followers 806 Following Research Scientist and Senior Manager in Meta AI (FAIR). AI-guided Optimization and Representation Learning. Novelist in spare time. PhD in @CMU_Robotics.
Gabriel Peyré @gabrielpeyre
92K Followers 449 Following @CNRS researcher at @ENS_ULM. One tweet a day on computational mathematics.
Google DeepMind @GoogleDeepMind
944K Followers 275 Following We’re a team of scientists, engineers, ethicists and more, committed to solving intelligence, to advance science and benefit humanity.
Pin-Yu Chen @pinyuchenTW
3K Followers 840 Following Principal research scientist@IBM Research & Chief Scientist@RPI-IBM AI Research Collaboration & PI@MIT-IBM AI Lab. IJCAI Computers & Thought Award Winner.
NeurIPS Conference @NeurIPSConf
112K Followers 35 Following New Orleans, Dec 10-16, 23. https://t.co/ga8aOw615g Tweets to this account are not monitored. Please send feedback to [email protected].
Andrej Karpathy @karpathy
979K Followers 905 Following 🧑🍳. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥
Dan Roy @roydanroy
45K Followers 2K Following ML / AI researcher, emphasis on theory. Research Director and Canada CIFAR AI Chair, @VectorInst Professor, @UofT (Statistics/CS)
Secure Learning Lab (.. @uiuc_aisecure
940 Followers 289 Following We are a computer science research group led by Bo Li at UIUC, focusing on responsible and trustworthy machine learning.
Prof. Anima Anandkuma.. @AnimaAnandkumar
25K Followers 2K Following Bren Professor @caltech, Fmr Sr Director of #AI research @nvidia, Fmr Principal Scientist @awscloud, AI+Science, PDE, Neural operators. Views my own.
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Han Zhao @hanzhao_ml
3K Followers 1K Following Assistant Professor @IllinoisCS; Ph.D. @mldcmu; Interested in machine learning and AI.
Shane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)
lmsys.org @lmsysorg
37K Followers 172 Following Large Model Systems Organization. We created Vicuna and Chatbot Arena! Compare 30+ LLMs (GPT-4/Claude/Llamas) side-by-side at https://t.co/IDFeIDIOtm
Wei Xiong @weixiong_1
186 Followers 176 Following PhD Student @IllinoisCS, Practice Math for 2.5 Years
Zhe Gan @zhegan4
2K Followers 321 Following Staff Research Scientist @Apple AI/ML. Ex-Principal Researcher @Microsoft Azure AI. Working on building large-scale vision and multimodal foundation models.
Charles Qi @charles_rqi
7K Followers 220 Following Autopilot and AI @Tesla | Prev: Research Scientist & Manager @Waymo | Postdoc @FAIR, PhD @Stanford | COO at the Lighthouse Mentorship Program.
Mengdi Wang @MengdiWang10
1K Followers 265 Following Princeton professor in AIML, optimization and data science. Program Chair @ICLR2023. Formerly @MIT @GoogleDeepmind @Tsinghua
Jason Weston @jaseweston
9K Followers 569 Following Research @MetaAI+NYU. Pretrain+FT: NLP from Scratch (2011). Multilayer attention+position embed+LLM: MemNets (2015). Recent (2023+):Sys 2 Attn, Self-Rewarding..
Haotian Liu @imhaotian
6K Followers 397 Following building intelligence @xAI, creator of #LLaVA, cs @UWMadison, prev @MSFTResearch
Yangqing Jia @jiayq
12K Followers 263 Following Founder @leptonai. @UCBerkeley alumni. ex @google & @facebook. ex vp @AlibabaGroup. Open source work on caffe, @pytorch, @tensorflow, & @onnxai.
Yu Meng @yumeng0818
1K Followers 160 Following Asst. Professor @CS_UVA, Past: PhD from @IllinoisCS, visiting researcher @princeton_nlp, Google PhD Fellow. NLP/ML/LLM
Yite Wang @YW91856288
11 Followers 138 Following PhD student at UIUC working on deep learning and numerical methods.
Haonan Wang @HaonanWang97
182 Followers 289 Following CS Ph.D. at National University of Singapore 🇺🇸UIUC-BS done 🇸🇬NUS-PhD doing
Anand Bhattad @anand_bhattad
2K Followers 293 Following Research Assistant Professor @TTIC_Connect | Exploring Knowledge in Generative Models | PhD from @illinoisCS | UG @surathkal_nitk
Ruslan Shaydulin @ruslanquantum
289 Followers 118 Following Quantum algorithms researcher @jpmorgan. Views my own.
Nomic AI @nomic_ai
14K Followers 50 Following Building explainable and accessible AI https://t.co/bbYqCdL8vQ
Wing Lian (caseus) @winglian
9K Followers 2K Following @axolotl_ai OSS maintainer. Axolotl AI founder. AI/ML tinkerer. Building tools for everyone.
Nicolas Delfosse @nic_delfosse
3K Followers 1K Following Principal Researcher working on quantum computing and quantum error correction @IonQ_Inc.
Mehrdad Farajtabar @MFarajtabar
2K Followers 145 Following Research Scientist at @Apple, ex-@DeepMind, ex-@GeorgiaTech
Eric @ericmitchellai
4K Followers 487 Following I like AI & music. Working on making LLMs easier & safer to use. Final year PhD student at Stanford advised by Chelsea Finn & Chris Manning.
Yizhe Zhang @YizheZhangNLP
1K Followers 442 Following Research Scientist at Apple MLR | ex-researcher @ Microsoft Research, Meta AI | PhD @ Duke University
Jason Ramapuram @jramapuram
789 Followers 394 Following ML Research Scientist MLR | Formerly: DeepMind, Qualcomm, Viasat, Rockwell Collins | Swiss-minted PhD in ML | Barista alumnus ☕ @ Starbucks | 🇺🇸🇮🇳🇱🇻🇮🇹
Josh Susskind @jsusskin
2K Followers 538 Following Apple ML research: foundations, perception, action, future technology, creativity, curiosity, compositionality, scientific jazz!
Jiatao Gu @thoma_gu
3K Followers 2K Following Machine Learning Researcher at @Apple ML Research (MLR) based in NYC | ex-FAIRer | PhD from HKU | Research on Generative AI for multimodalities. また日本語もできます。
Jiawei Liu @JiaweiLiu_
2K Followers 957 Following Simplifying the making of great software. PhD Student @plfmse @IllinoisCS.
Sachin Goyal @goyalsachin007
765 Followers 715 Following PhD student @ CMU MLD || Microsoft Research || UG @ IIT Bombay
Together AI @togethercompute
27K Followers 304 Following The future of AI is open-source. Let's build together.
Tianlin @linylinx
6K Followers 579 Following ML Tech Lead @sourceful ⏩: @illumina AI Lab @qualcomm AI, PhD @LSEStatistics 📜 generative models 🤪 joking not joking
Fuzhao Xue @XueFz
4K Followers 542 Following Ph.D. candidate@NUSingapore, Intern of GEAR @NVIDIA | Google PhD Fellow | LLM, Foundation Model Scaling | Ex-@GoogleBrain | Zero-shot Cooking Learner🧑🍳
Max Tegmark @tegmark
145K Followers 29 Following Known as Mad Max for my unorthodox ideas and passion for adventure, my scientific interests range from artificial intelligence to the ultimate nature of reality
Ziqi Wang @wzq016
149 Followers 316 Following Ph.D. student @IllinoisCS, Prev undergrad @Tsinghua_Uni, Prev intern @Google
Hyung Won Chung @hwchung27
18K Followers 231 Following Research Scientist @OpenAI. Past: @Google Brain / PhD @MIT
Andy Zou @andyzou_jiaming
3K Followers 63 Following PhD student at CMU, working on AI Safety and Security
Jacob Andreas @jacobandreas
14K Followers 958 Following Teaching computers to read. Assoc. prof @MITEECS / @MIT_CSAIL (he/him). https://t.co/5kCnXHjtlY https://t.co/2A3qF5vdJw
Mistral AI @MistralAI
90K Followers 0 Following Fast, open-source and secure language models. Join us https://t.co/INALdNGvCP
Chi Han @Glaciohound
220 Followers 230 Following CS PhD student at UIUC, interested in language models and their understanding.
Song Mei @Song__Mei
1K Followers 571 Following Assistant Professor at UC Berkeley, Department of Statistics and EECS. Researcher working on foundations of AI and deep learning.
Suchin Gururangan @ssgrn
4K Followers 250 Following he/him Research scientist 🦙 Llama team, @meta GenAI PhD @uwcse + @uwnlp
Rahul Goel @rahul_nlu
2K Followers 498 Following Making LLM agents come to life. Modeling Lead Bard@Google. Previously: NLU@Google Assistant, Alexa Conversations.
Edward Grefenstette @egrefen
36K Followers 776 Following FR/US/GB AI/ML Person, Director of Research at @GoogleDeepMind, Honorary Professor at @UCL_DARK, @ELLISforEurope Fellow. All posts are personal.
jason @agikoala
2K Followers 24 Following secondary account (main is @_jasonwei) @agihippo is a buddy of mine
yi 🦛 @agihippo
3K Followers 81 Following secondary account, hardcore fans only. friend of @agikoala the great researcher, main account: @yitayml warning: hot takes.
Saurabh Garg @saurabh_garg67
864 Followers 579 Following Building next-gen AI at @MistralAI | prev/ PhD @mldcmu; CS @iitbombay (undergrad); Collab @GoogleAI @awscloud @apple
Anurag Ranjan @anuragranj
3K Followers 504 Following Researcher @Apple. 3D. PhD @MPI_IS. opinions my own.
Do models need to reason in words to benefit from chain-of-thought tokens? In our experiments, the answer is no! Models can perform on par with CoT using repeated '...' filler tokens. This raises alignment concerns: Using filler, LMs can do hidden reasoning not visible in CoT🧵
Data condition: On our task, LMs fail to converge when trained on only filler-token sequences (ie Question …… Answer). Models converge only when the filler training set is augmented with additional, parallelizable CoTs, otherwise filler-token models remain at baseline accuracy
Announcing the #ILLINOIS Siebel School of Computing and Data Science at The Grainger College of Engineering, made possible with a $50 MM gift from Thomas M. Siebel. With our #5 in-the-nation computer science program and 21 blended degree programs, the best is yet to come! 🔸🔹
Check out our latest SOTA open-source reward model based on LLaMA3-8B-it! The RM readily serves to provide signals for subsequent iterative RLHF, see a demo in huggingface.co/sfairXC/Fsfair… which improves zephyr-set with alpaca lc win rate 8% to 34.79%
First Llama 3 8b instruct --> reward model is SOTA open model on RewardBench. kudos @hendrydong and team huggingface.co/sfairXC/Fsfair…
@natolambert Also a demo is that we align the zephyr-7b-sft (with 8% alpaca eval win rate) to sfairXC/FsfairX-Zephyr-Chat-v0.1 with 34.79%. Another message is that we only use the rm in rewardbench to label sample instead of gpt4 but also get pretty good results.
@natolambert We derive the online iterative rlhf/dpo and establish the mathematical foundation in arxiv.org/pdf/2312.11456… . Then we realize that the community doesn't like math.... So we are working on a separate exp paper with minimal equations.
Llama3 was trained on 15 trillion tokens of public data. But where can you find such datasets and recipes?? Here comes the first release of 🍷Fineweb. A high quality large scale filtered web dataset out-performing all current datasets of its scale. We trained 200+ ablation…
We have just released 🍷 FineWeb: 15 trillion tokens of high quality web data. We filtered and deduplicated all CommonCrawl between 2013 and 2024. Models trained on FineWeb outperform RefinedWeb, C4, DolmaV1.6, The Pile and SlimPajama!
Happy to see that the rejection sampling finetuning (we call it RAFT, reward ranked finetuning arxiv.org/pdf/2304.06767…) also contributes to the post-fine tuning of llama3 Here DPO is short for direct POLICY optimization. Does it mean they skip RM but use some other algorithm ?
🥁 Launching a new dataset: Capybara-Preferences, built with distilabel 1.0 ⚗️! Hard at work fine-tuning Llama 3? Here's the dataset you've been waiting for. Initial results with ORPO & this dataset are 🔥 huggingface.co/datasets/argil… 🧵What makes this dataset so special?
🔥llm.c update: Our single file of 2,000 ~clean lines of C/CUDA code now trains GPT-2 (124M) on GPU at speeds ~matching PyTorch (fp32, no flash attention) github.com/karpathy/llm.c… On my A100 I'm seeing 78ms/iter for llm.c and 80ms/iter for PyTorch. Keeping in mind this is fp32,…
Proud to be part of the team to make both Llama Guard 2 and Llama 3 happen! This is indeed a tough and fulfilling journey! Check out our models!
It’s here! Meet Llama 3, our latest generation of models that is setting a new standard for state-of-the art performance and efficiency for openly available LLMs. Key highlights • 8B and 70B parameter openly available pre-trained and fine-tuned models. • Trained on more…
Llama 3 has been my focus since joining the Llama team last summer. Together, we've been tackling challenges across pre-training and human data, pre-training scaling, long context, post-training, and evaluations. It's been a rigorous yet thrilling journey: 🔹Our largest models…
🎨Spent some time refactoring the 2021 post on diffusion model with new content: lilianweng.github.io/posts/2021-07-… ⬇️ ⬇️ ⬇️ 🎬Then another short piece on diffusion video models: lilianweng.github.io/posts/2024-04-… (Yes, I had an intensive weekend🥹)
Congrats @ElanRosenfeld on a great thesis that moves our understanding of distribution shift forward! w/ @risteski_a @boazbaraktcs @ShalitUri
Google presents RecurrentGemma Moving Past Transformers for Efficient Open Language Models We introduce RecurrentGemma, an open language model which uses Google's novel Griffin architecture. Griffin combines linear recurrences with local attention to achieve excellent
LLM2Vec Large Language Models Are Secretly Powerful Text Encoders Large decoder-only language models (LLMs) are the state-of-the-art models on most of today's NLP tasks and benchmarks. Yet, the community is only slowly adopting these models for text embedding tasks,
magnet:?xt=urn:btih:9238b09245d0d8cd915be09927769d5f7584c1c9&dn=mixtral-8x22b&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=http%3A%2F%https://t.co/OdtBUsbeV5%3A1337%2Fannounce
Trends for United States
You might like
