Bill Xu @billxbf
Cyberpunk. Research @ Samsung AI Center. On better generative reasoning/planning in foundation models. billxbf.github.io CA Joined May 2022-
Tweets322
-
Followers183
-
Following123
-
Likes409
Yes! We are looking for contributors for OpenDevin! Here are some ways to get started: 1. Join discussions on github, slack, or discord: github.com/openDevin/Open… 2. Take a look at the "good first issues" and try to work on them: github.com/OpenDevin/Open…
Yes! We are looking for contributors for OpenDevin! Here are some ways to get started: 1. Join discussions on github, slack, or discord: github.com/openDevin/Open… 2. Take a look at the "good first issues" and try to work on them: github.com/OpenDevin/Open…
The first chapter of the game of scale focus on scaling text data, which peaks at GPT-4 and concluded by Llama 3. The second chapter of this game would be unified video-language generative modeling and iterative reinforcement learning from X feedback. yaofu.notion.site/Apr-2024-Llama…
Key takeaways 👉 n param: 7B -> 8B training data: 2T -> 15T vocab size: 32k -> 128k harder data curation better data mixer
Key takeaways 👉 n param: 7B -> 8B training data: 2T -> 15T vocab size: 32k -> 128k harder data curation better data mixer
Some real $10M worth of inspiring experiments and evidences 😮
Some real $10M worth of inspiring experiments and evidences 😮
This competition is so intriguing in any sense that I can’t resist back to Kaggle. kaggle.com/competitions/a…
RWKV-6 is out! huggingface.co/BlinkDL/rwkv-6… - Available in both 1.6B and 3B - Trained on 2.5T tokens - Can handle 100+ languages Upcoming model: RWKV-6 7B model ^^
Interesting to see it outperforms similar-sized mistral 8x7b in most benchmarks 🤔 Can we draw conclusion that Mamba (vs transformers) = higher training time for higher inference throughput + longer context? @AI21Labs @tri_dao Mamba out
Whilst many benchmarks, every practitioner has his own ranking of open-source (smaller) LLMs. And to me, after many experiments: 1’ Mistral-7B 2’ Llama2-13B 3’ Llama2-7B 4’ Gemma-2B 5’ Gemma-7B 😅pretty affirmative about some obvious inconsistency, but opinions are my own.
The (probably only) good point of commercial GPUs (eg 4090) over server GPUs is the “zzz” sound and a free warmer at home when you start training 🎶
🤨Should you care about GFlowNets? What are they anyway?🧐 Learn about how GFlowNets speed up drug discovery and help large language models reason better in my new video!🔬📚 youtu.be/o0Ju9NQa5Ko
Is Cosine-Similarity of Embeddings Really About Similarity? Netflix cautions against blindly using cosine similarity as a measure of semantic similarity between learned embeddings, as it can yield arbitrary and meaningless results. 📝arxiv.org/abs/2403.05440
Found more bugs for #Gemma: 1. Must add <bos> 2. There’s a typo for <end_of_turn>model 3. sqrt(3072)=55.4256 but bfloat16 is 55.5 4. Layernorm (w+1) must be in float32 5. Keras mixed_bfloat16 RoPE is wrong 6. RoPE is sensitive to y*(1/x) vs y/x 7. (Fixed) RoPE should be float32…
Definitely one of a few high quality papers these days.
Definitely one of a few high quality papers these days.
Introducing Sora, our text-to-video model. Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions. openai.com/sora Prompt: “Beautiful, snowy…
Nakidep @nakidep1794
0 Followers 206 Following Life itself is a journey, we are all worthy and should strive to travel to different lives.
LesleyHenley @k63se18cToJasRR
1 Followers 103 Following
Andrzej Białecki @Kaszanas
438 Followers 2K Following PhD student @WUT_edu Esports Research Science • I write in (Python, Go, Rust) • Sports Professional RG: https://t.co/l5qNRtn2K7…
Jun Kai @ljunkai_
52 Followers 42 Following I design AI solutions for Co. @AWS ● Talks about GenAI landscape & technical concepts ● Sharing my opinions based on hands-on experience.
The Lone Ranger @AbdullahMdKhan
54 Followers 2K Following
Ervin Lang @ervinlang
49 Followers 1K Following
Marko @MarkoVelich
145 Followers 2K Following Director of Engineering at Photomath, ex-Facebook, ex-LEGO Engineering Manager with focus on Machine Learning Passion for building amazing engineering teams
⿻ barton 🦺𑗊 @bmorphism
2K Followers 4K Following applied categorical duck cyberneticist • building for agencies in the 21st century • inventor of the operadic cognitive diagram cognitive continuation standard
Omar Yasser @OmarYasser314
6 Followers 865 Following
Jayoo Hwang @JayooHwang
90 Followers 882 Following Independent deep learning researcher (LLMs, multimodal, agents) @ml_collective, BSc UCalgary
Taimoor Hassan 🇵�.. @mtaimoorhas
6K Followers 7K Following 22 | 8x Startups SOLD - 12 Built | https://t.co/HwhGKjCbah ($7.5k) | 🇵🇰 National Winner 🏆 World Finalist @MSFTimagine 2021 | AI • SaaS • NoCode | Indie Hacker
Banghua Zhu @BanghuaZ
2K Followers 804 Following PhD @Berkeley_EECS, statistics, info theory, LLM, RL, Human-AI Interactions.
Pacific Robots @pacificrobots
69 Followers 610 Following
Sergey Bunas (e/acc) @sergeybunas
671 Followers 464 Following Crafting AI @ https://t.co/BnTPNTJ38O Previously: - Founder @ https://t.co/uPwKdsJ65V (AI replies for Twitter) - Senior Engineer @ https://t.co/kAgBvInjdZ (YC19)
AI Deeply @AiDeeply
403 Followers 5K Following AI is reshaping the world. Who are the people and companies driving the change? Visit our website to search more than 5,000 profiles.
jose @jose08050145
0 Followers 77 Following
Jason Cwik @jasoncwik
104 Followers 131 Following Director, ECS Infrastructure, Dell EMC. #iworkfordell but all opinions are my own.
Lee (Caoyuan) Li @GrassLee123
44 Followers 1K Following
Christopher Snyder @DrChrisSnyder
24 Followers 137 Following
Yizhi Li @yizhilll
269 Followers 407 Following PhD Student @Manchester_NLP; Multimodal Art Projection research community (https://t.co/i2hhDpkRTV)
Zhen Wang @zhenwang9102
474 Followers 448 Following
Humans of the Latent .. @latenthumans
88 Followers 590 Following Exploring the art of #Synthography [email protected]
Jade @Jade72007337861
9 Followers 880 Following
Rebecca Wang @ZhaoyueWan75195
19 Followers 168 Following #Master student at #McGill/#Mila. Currently at #UofT. Want to build safe RL, AI alignment. Want to bridge non-tech stakeholders with AI researchers
Bella💋 @June_WTOP
1K Followers 3K Following Love is the soul of everyone. Love is a kind of emotional dependence in people's hearts. Love is a part of one's life.🌟🌟☀️☀️
Universa @UniversaAI
114 Followers 134 Following Welcome to UNIVERSA, an ambitious open-source initiative aimed at transcending traditional Al development. Our podcast : @Universaaipod #AI
Łukasz Hanusik @bdsmsystems
142 Followers 2K Following DjPizza™ at night, Deviloper at rest, Anti-pattern Architect, advocate at B.D.S.M Business Development Sales Marketing. Always Habibis ❤️
Ashutosh Mehra @ashutoshmehra
1K Followers 5K Following Senior Principal Scientist at Adobe. Working on Acrobat AI Assistant, LLMs, and document ML.
Shivshankar Shukla @02__shanks
90 Followers 1K Following Frying neurons, otaku wisdom, one byte at a time 🎬 | IITR'24
konilse @anas9r
9 Followers 442 Following
Felipe Cardoso @darkfelix1989
20 Followers 120 Following
ElzaDemaree @DemareeElz57361
129 Followers 2K Following
Nikolai Yakovenko @ivan_bezdomny
8K Followers 6K Following DeepNewz -- realtime news powered by AI. Check out our website and GPT Store app. iOS app coming soon! @deepnewsbot AI News @deepnftvaluebot NFT pricing
Yifan Xie @YifanX
1K Followers 587 Following words are of my overfitted mental model of the world doing NLP stuff for DeepNewz, and building some NFT valuation models
Georgi Gerganov @ggerganov
38K Followers 243 Following Not AI | 0x0e59 0x2550 24th at the Electrica puzzle challenge
Zeyuan Allen-Zhu @ZeyuanAllenZhu
8K Followers 273 Following physics of language models @ Meta / FAIR IOI - USACO - MCM - ACM/ICPC - Codejam Tsinghua - MIT - Princeton/IAS - MSR - FAIR
Najoung Kim 🫠 @najoungkim
2K Followers 493 Following At @BULinguistics and visiting @GoogleAI part-time. 🤖🔠🐱
Xin Eric Wang @xwang_lk
7K Followers 1K Following Multimodal and Embodied AI Researcher / Professor @UCSC. Director of https://t.co/Y4swOBag21. AI for Humanity in the long run. he/him
Arthur Mensch @arthurmensch
40K Followers 873 Following Co-founder and CEO @MistralAI. Apply https://t.co/yHGRZAtjcx
Tal Linzen @tallinzen
16K Followers 893 Following Professor @nyuling and @NYUDataScience, research scientist @GoogleAI
Chuang Gan @gan_chuang
4K Followers 456 Following Faculty Member at UMass Amherst; Principal researcher at MIT-IBM Watson AI Lab; Homepage: https://t.co/oXP6pqXCpo
Banghua Zhu @BanghuaZ
2K Followers 804 Following PhD @Berkeley_EECS, statistics, info theory, LLM, RL, Human-AI Interactions.
lmsys.org @lmsysorg
37K Followers 172 Following Large Model Systems Organization. We created Vicuna and Chatbot Arena! Compare 30+ LLMs (GPT-4/Claude/Llamas) side-by-side at https://t.co/IDFeIDIOtm
Sergey Bunas (e/acc) @sergeybunas
671 Followers 464 Following Crafting AI @ https://t.co/BnTPNTJ38O Previously: - Founder @ https://t.co/uPwKdsJ65V (AI replies for Twitter) - Senior Engineer @ https://t.co/kAgBvInjdZ (YC19)
Beidi Chen @BeidiChen
6K Followers 343 Following Asst. Prof @CarnegieMellon, Visiting Researcher @Meta, Postdoc @Stanford, Ph.D. @RiceUniversity, Large-Scale ML, a fan of Dota2.
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Hao Liu @haoliuhl
4K Followers 155 Following phd student @berkeley_ai https://t.co/ZNJawlrerS machine learning, neural networks.
Nathan Lambert @natolambert
25K Followers 690 Following Figuring out AI @allen_ai, "rl boi" DM me papers. Writes @interconnectsai, talks @retortai Has phd and some credentials
Yizhi Li @yizhilll
269 Followers 407 Following PhD Student @Manchester_NLP; Multimodal Art Projection research community (https://t.co/i2hhDpkRTV)
Binyuan Hui @huybery
6K Followers 318 Following 🐚 Core maintainer at Qwen and OpenDevin. || Code Generation, Text-to-SQL, Large Language Models.
Subbarao Kambhampati .. @rao2z
16K Followers 29 Following AI researcher & teacher @SCAI_ASU. Works on Human-Aware AI. Former President of @RealAAAI; Chair of @AAAS Sec T. Here to tweach #AI. YouTube Ch: https://t.co/4beUPOmMW6
Nikolai Yakovenko @ivan_bezdomny
8K Followers 6K Following DeepNewz -- realtime news powered by AI. Check out our website and GPT Store app. iOS app coming soon! @deepnewsbot AI News @deepnftvaluebot NFT pricing
Yifan Xie @YifanX
1K Followers 587 Following words are of my overfitted mental model of the world doing NLP stuff for DeepNewz, and building some NFT valuation models
Yu Su @ysu_nlp
6K Followers 857 Following Dist. Assist. Prof.@OhioState, Director @osunlp, 20% Researcher@Microsoft. I like to think about intelligence, artificial or biological
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Ari Weinstein @AriX
17K Followers 8K Following Co-founder @SoftwareAppsInc. Previously managed Shortcuts and SiriKit at Apple, and co-founded Workflow. @[email protected]
Denny Zhou @denny_zhou
9K Followers 420 Following @GoogleDeepMind founder & lead of Reasoning Team. Build LLMs to reason. Opinions my own.
Demis Hassabis @demishassabis
357K Followers 125 Following Co-founder & CEO @GoogleDeepMind - working on AGI. Trying to understand the fundamental nature of reality. Also revolutionising drug discovery @IsomorphicLabs
Eugene Vinitsky @EugeneVinitsky
13K Followers 2K Following Anti-cynic. Artificial narrow intelligence. Autonomous vehicles, multi-agent learning, and transportation. RS at Apple, Asst. Prof at @nyutandon. He/him.
Josselyn Ordóñez @JossySoo
19 Followers 158 Following Administradora 💻. Interesada en Innovación, Tecnología y Desarrollo Social
Joseph Suarez (e/🐡.. @jsuarez5341
2K Followers 63 Following MIT PhD candidate, creator of Neural MMO (https://t.co/NaaDv6UQlN), PufferLib (https://t.co/43D0orh0lJ). Open-source RL
Elon Musk @elonmusk
181.6M Followers 584 Following
Tivadar Danka @TivadarDanka
66K Followers 457 Following I make math accessible for everyone. Mathematician with an INTJ personality. Chaotic good. Writing https://t.co/jYkO4bz6lL
Peyman Milanfar @docmilanfar
67K Followers 264 Following Distinguished Scientist at Google Research. Computational Imaging, Machine Learning, and Vision. Tweets = personal opinions. May change or disappear over time.
Alex Cabrera @a_a_cabrera
1K Followers 491 Following PhD candidate @cmuhcii @scsatcmu. Humans + AI = ???
Ziyu Yao @ZiyuYao
1K Followers 544 Following Asst Prof @GeorgeMasonU CS interested in #NLProc #AI. Alum @OhioState. Prev intern @LTIatCMU @MSFTResearch @FujitsuAmerica @Tsinghua_Uni.
merve @mervenoyann
56K Followers 4K Following open-sourceress at @huggingface 🧙🏻♀️ proud mediterrenean 🍋 I do TL;DR on ML papers
Giannis Daras @giannis_daras
4K Followers 399 Following Ph.D. candidate, Computer Science @UTAustin, working with @AlexGDimakis. Research Scientist Intern @nvidia. Ex: @google, @explosion_ai, @ntua
kache (dingboard.com) @yacineMTB
53K Followers 3K Following i'm a swe. go to https://t.co/pWRBfY8kn2 - AI image editing IN YOUR BROWSER! follow to watch a self funded founder beat VC backed AI startups with @dingboard_
Nick Cui @NickCui2023
13 Followers 98 Following
Bill Xing @BillXing7
191 Followers 1K Following Tech Investor, Investment Vice President at 5Y Capital, https://t.co/SQAjATQWSX
Max Woolf @minimaxir
19K Followers 460 Following Data Scientist at @BuzzFeed in San Francisco // AI content generation R&D // Mastodon: @[email protected]
Costa Huang @vwxyzjn
3K Followers 1K Following RLHF @huggingface 🤗; main dev of @cleanrl_lib; CS PhD @DrexelUniv; Ex @CuraiHQ @weights_biases @NVIDIAAI @riotgames.
George Hotz 🌑 @realGeorgeHotz
248K Followers 174 Following President @comma_ai. Founder @__tinygrad__
Graham Neubig @gneubig
31K Followers 588 Following Associate professor at CMU, studying natural language processing and machine learning.
YOASOBI @YOASOBI_staff
1.1M Followers 108 Following We are YOASOBI from JAPAN!Composer:Ayase→@Ayase_0404 Vocal:ikura→@ikutalilas Songs: https://t.co/iLAra1R7MeYes! We are looking for contributors for OpenDevin! Here are some ways to get started: 1. Join discussions on github, slack, or discord: github.com/openDevin/Open… 2. Take a look at the "good first issues" and try to work on them: github.com/OpenDevin/Open…
That's amazing . Are you guys looking for contributors , not sure how to start ?
this but for the subset of the bored population that would talk to bots on lmsys for fun
proper way to model social media is that the average user spends at most 300ms-3s looking at a tweet, does not read it, does not pause to think about it, but still instantly reacts with whatever emotion the vibe of the post gave them. then they instantly forget and keep scrolling
Excited to share our work at @GoogleDeepMind! We propose Naturalized Execution Tuning (NExT), a self-training method that drastically improves the LLM's ability to reason about code execution, by learning to inspect execution traces and generate chain-of-thought rationales 🧵👇
@deliprao hey third time could be the charm!! not judging until i get the model weights 😂
@deliprao you sure there is long-term reputation? I thought internet (and research community) has no memory of subpar things (famous) people did
The first chapter of the game of scale focus on scaling text data, which peaks at GPT-4 and concluded by Llama 3. The second chapter of this game would be unified video-language generative modeling and iterative reinforcement learning from X feedback. yaofu.notion.site/Apr-2024-Llama…
Today's LLM leaderboard chasing is like yesterday's ImageNet climbing but with more players.
@billxbf It's a crazy test idea. News soon! Note license
After a prolonged two and a half year. I finally got promoted to L4 SWE. Hopefully I can find a place to do full time research not just 20%🥲 in the near future.
According to the license, you must name all models that use llama 3 in any way “LLaMa 3 XXX” llama.meta.com/llama3/license/ They don't say that you can't give your models nicknames though... "LLaMa 3 Robert Archibald Percival Fortescue Language Model" aka "BobLM"
after two years of blood sweat and tears, i cannot describe how it feels to tie the bow on this device🥳😭 can’t wait to see what folks hack and build with Frame 🤘🏼⚒️ @brilliantlabsAR
I transitioned from research to startup world at least two years too late 😅
Llama 3: Better data is all you need
Let me show something that is ACTUALLY DIFFERENT. @perplexity_ai is NOT ABLE TO deal with new arxiv papers while our chrome extension, elmo.chat, does an excellent job. See this thread for details. Proof in this thread. You are welcome to check it out. Dude, this…
Honored and proud of our designers!
This is the first time we see a new architecture making🍎to🍎 comparison at scale with Llama-7B trained on the same 2T tokens and win (unlimited context length, lower ppl, constant kv at inference, ...)! Very excited to be part of the team! Thanks for the lead @violet_zct…
How to enjoy the best of both worlds of efficient training (less communication and computation) and inference (constant KV-cache)? We introduce a new efficient architecture for long-context modeling – Megalodon that supports unlimited context length. In a controlled head-to-head…
Meta announces Megalodon Efficient LLM Pretraining and Inference with Unlimited Context Length The quadratic complexity and weak length extrapolation of Transformers limits their ability to scale to long sequences, and while sub-quadratic solutions like linear attention and
Natural Language Processing allows machines to communicate with and learn from humans. A Language Model (LM) assigns probabilities to sentences. It can be use to fix typos and grammar or respond to questions. Using n-grams allows us to create simple statistical LMs.
Happy to share our survey preprint on using generative models for recommender systems. Awesome collaboration across industry and academia! This is my first paper after GDM. :) Paper: arxiv.org/abs/2404.00579
📘 New Research Alert📊 "A Review of Modern #RecommenderSystems Using Generative Models (Gen-RecSys)" is online. link: arxiv.org/abs/2404.00579 An important milestone in generative information-seeking research. #recsys #generative #llm #evaluation #harm #foundationmodel
New multimodal model in town: Idefics2! 💪 Strong 8B-parameters model: often on par with open 30B counterparts. 🔓Open license: Apache 2.0. 🚀 Strong improvement over Idefics1: +12 points on VQAv2, +30 points on TextVQA while having 10x fewer parameters. 📚 Better data:…
Trends for United States
You might like
