Misha Laskin @MishaLaskin
Staff Research Scientist @DeepMind. Previously @berkeley_ai. YC alum. mishalaskin.com NYC Joined August 2013-
Tweets680
-
Followers8K
-
Following175
-
Likes2K
How do LLMs scale to million token context window? Ring Attention is a nice trick to parallelize long sequence across devices and rotate them in a ring with zero overhead scaling. In our new blog, we cover the tricks behind this magic. It looks like this (1/5🧵)
Today @Astranis is introducing a revolutionary new satellite for high orbits. We call it Omega. It is pound-for-pound higher performance than any satellite on orbit today. First flight unit will be completed next year, with launches starting in 2026.
Introducing Captions for web. Video editing made simple, thanks to AI — now on a bigger screen. One-click editing, right from your browser.
the issue with getting reliable outputs from LLMs as a user is that you don't know what prompts were used during RLHF when the model was aligned so you are forced to manually explore the space of possible prompts
the issue with getting reliable outputs from LLMs as a user is that you don't know what prompts were used during RLHF when the model was aligned so you are forced to manually explore the space of possible prompts
I'm starting a company with @brian_ichter, @chelseabfinn, @svlevine, @hausman_k, @QuanVng, and @SurajNair_1 called Physical Intelligence (π.com!). We're bringing general-purpose AI into the physical world.
tokenistic interpretability
We are excited to share Large World Model (LWM), a general-purpose 1M context multimodal autoregressive model. It is trained on a large dataset of diverse long videos and books using RingAttention, and can perform language, image, and video understanding and generation.
congrats to chris and the polycam team!
congrats to chris and the polycam team!
Excited to share @Polycam3D has raised $18M from @leftlanecap, @adjacent and @Adobe! Today we are announcing the launch of our #VisionPro for viewing captures in immersive 3D. A thread 🧵
I'm excited to share a preview of what I've spent the last few months working on at @GoogleAI: SPO, a new RLHF algorithm with strikingly simple implementation (no reward models) and shockingly strong guarantees (handles messy, intransitive prefs.): arxiv.org/abs/2401.04056
Google DeepMind announces Vision-Language Models as a Source of Rewards paper page: huggingface.co/papers/2312.09… Building generalist agents that can accomplish many goals in rich open-ended environments is one of the research frontiers for reinforcement learning. A key limiting…
Buried in the news due to Gemini launch - but AlphaCode 2 is really impressive too. Scores at the 15% of Codeforces participants.
Buried in the news due to Gemini launch - but AlphaCode 2 is really impressive too. Scores at the 15% of Codeforces participants.
Excited to finally share what I’ve been working on over the past year. Gemini is a really capable SOTA model with strong reasoning and coding abilities. It’s multimodal - can understand images, videos, audio, and text. It was a really intense and collaborative effort!…
I’m very excited to share our work on Gemini today! Gemini is a family of multimodal models that demonstrate really strong capabilities across the image, audio, video, and text domains. Our most-capable model, Gemini Ultra, advances the state of the art in 30 of 32 benchmarks,…
We’re thrilled to announce two online LLMs we’ve trained: pplx-7b-online and pplx-70b-online! Built on top of open-source LLMs and fine-tuned to use knowledge from the internet. They are now available via Labs and in a first-of-its-kind live-LLM API. pplx.ai/online-llms
📢 Announcing a breakthrough in science robotics @SciRobotics - 𝙑𝙞𝙨𝙪𝙖𝙡 𝘿𝙚𝙭𝙩𝙚𝙧𝙞𝙩𝙮 🏳️🌈 any object 🌈 any rotation 📉 a low-cost hand (D'Claw) 📷single camera A single policy capable of in-hand reorientation of novel & complex objects (thread👇)
I find the OAI development today quite sad. The work Sam and Greg have contributed to has been inspiring for years now. Even before ChatGPT there was incredible research impact - gym, Rubik’s cube robot, scaling laws, etc. Hard, transformative work over many years.
We all know that in-context learning emerges in transformers... but our new work shows that it can actually then disappear, after long training times! We dive into this **transience** phenomenon. arxiv.org/abs/2311.08360 🧵👇1/N
AK @_akhaliq
310K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo follow on Hugging Face: https://t.co/q2Qoey80Gx
Jim Fan @DrJimFan
229K Followers 3K Following @NVIDIA Sr. Research Manager & Lead of Embodied AI (GEAR Lab). Creating foundation models for Humanoid Robots & Gaming. @Stanford Ph.D. @OpenAI's first intern.
Eric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0p
Soumith Chintala @soumithchintala
187K Followers 883 Following Cofounded and lead @PyTorch at Meta. Also dabble in robotics at NYU. AI is delicious when it is accessible and open-source.
Danijar Hafner @danijarh
14K Followers 869 Following Building AI that makes autonomous decisions using world models, artificial curiosity, and temporal abstraction @DeepMind
Karol Hausman @hausman_k
22K Followers 141 Following @Physical_int ex: researcher @GoogleAI/@DeepMind, adj. Prof. @Stanford. Into robots, AI, NBA, philosophy, soccer and almond croissants. 🇵🇱🇺🇸
Michael Black @Michael_J_Black
59K Followers 643 Following Director, Max Planck Institute for Intelligent Systems (@MPI_IS). Chief Scientist @meshcapade. Building 3D digital humans using vision, graphics, and learning.
Eugene Vinitsky @EugeneVinitsky
13K Followers 2K Following Lets make multi-agent learning easy. Anti-cynic. RS at Apple, Asst. Prof at @nyutandon. He/him.
Shane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)
yobibyte @y0b1byte
15K Followers 2K Following Kurin ViTaly, senior research scientist @IsomorphicLabs, ML PhD from @UniofOxford on RL, Multitask learning & Graphs
Tom Goldstein @tomgoldsteincs
23K Followers 2K Following Professor at UMD. AI security & privacy, algorithmic bias, foundations of ML. Follow me for commentary on state-of-the-art AI.
Sander Dieleman @sedielem
50K Followers 2K Following Research Scientist at Google DeepMind. I tweet about deep learning (research + software), music, generative models (personal account).
Noam Brown @polynoamial
34K Followers 612 Following Researching reasoning @OpenAI | Co-created Libratus/Pluribus, the first superhuman no-limit poker AIs | Co-created CICERO | PhD from @SCSatCMU
Sara Hooker @sarahookr
39K Followers 7K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.
Ethan Caballero is bu.. @ethanCaballero
8K Followers 2K Following ML PhD student @Mila_Quebec ; previously @GoogleDeepMind
Nathan Lambert @natolambert
25K Followers 690 Following Figuring out AI @allen_ai, "rl boi" DM me papers. Writes @interconnectsai, talks @retortai Has phd and some credentials
Andrew Carr (e/🤸) @andrew_n_carr
15K Followers 3K Following science @getcartwheel AI writer @tldrnewsletter advisor @arcade_ai Past - Codegen @OpenAI, Brain @GoogleAI, world ranked Tetris player
Gunbir Singh Baveja @g_baveja
19 Followers 90 Following visiting researcher @kaist_ai advised by @JosephLim_AI; sophomore @UBC
Grace Isford @graceisford
7K Followers 2K Following Partner @Lux_Capital investing in the future 🚀 | board @ecorner (STVP) previously @canvasvc @stanfordwib @joinhandshake @stanford
Stefan Juang @StefanJuang
147 Followers 1K Following The final goal of AI is not just to create intelligent machines, but to understand intelligence itself.
Stephanie Zhan @stephzhan
19K Followers 2K Following GP @Sequoia. Preseed/Seed/A. Boards @linear @middeskhq @recroom Seeds: @replicatehq, https://t.co/qBx7xi7Qrd, 5 in stealth: robotics, AI agents, & more. AI, SaaS, dev tools.
Mikkel @Mikkel86881951
423 Followers 2K Following
Charles Packer @charlespacker
660 Followers 311 Following Building https://t.co/RKVR6kpMCl 📚🦙 | PhD student at @berkeley_ai @ucbrise @BerkeleySky
Kilian Haefeli @khshind
234 Followers 344 Following Exploring crevasses of Deep Learning at ETH Zurich & UofT | Previously: @Aleph__Alpha, @Logitech, and exfounder at Airica
Juyong Lee @jylee_ai
29 Followers 62 Following
Milin Bhade @MilinBhade
57 Followers 1K Following Post Grad Student at IISc, Bangalore Masters in Computer Science & Automation
Howie Xu @H0wie_Xu
4K Followers 569 Following Entrepreneur, AI/Security Executive | ex CEO TrustPath, SVP of AI at Palo Alto Networks, Greylock EIR, Founder of VMware networking
Dana Mahmood @deordered
24 Followers 731 Following Fine-tuning AI models oftentimes & practicing philosopher at other times.
Electronicsseeker @libertarian108
9 Followers 1K Following
Barbad @BarbadForoughi
0 Followers 1K Following
Krystian Weissgerber @k_weissgerber
16 Followers 37 Following Prompt engineer @ Orange Poland 🟧 AI Student @ Koźmiński University
Eli Brosh @EliBrosh
47 Followers 132 Following Head of AI Research at https://t.co/TjQKarfKqP, Machine learning junkie, Coffee snob
HinePo @Hine__Po
191 Followers 440 Following Head of AI & Data. Data science tech lead. Chemical engineer. Kaggle Competitions Expert (top 1%).
Karishma @KThakrar1
16 Followers 579 Following
Janhavee Shinde @SJanhavee
61 Followers 2K Following
Xiaolong Yang @yang_appstats
388 Followers 3K Following AM student of political methodology @HarvardGSAS. 東大教養の人間だった。因果推論。
0xShangri-La @jediming
104 Followers 2K Following
Li_F2_H2 @Li_F2_H2
70 Followers 445 Following
Aryan Pandey (Look fo.. @AryanPa66861306
1K Followers 3K Following Half Machine Learning Engineer || DevOps and Machine Learning || Open Source at OpenVINO
19890723 @tDzISWb22CPtFrP
4 Followers 2K Following
زِرِنگ @premature79
400 Followers 918 Following
Gagan Jain @gaganjain1582
53 Followers 748 Following Research Associate @GoogleDeepMind | IIT Bombay'22
Vinod Valloppillil @vinodv
450 Followers 743 Following Enterprise + AI. Partner/Dir PM Azure AI. ex-$GOOG (led Cloud AI Language & Vision PM), $DBX (search, ML), Startups (3 exits), early $MSFT (OS, web).
Winner Chukwuemeka @WinnerAzubuike
22 Followers 100 Following Building @ https://t.co/DmZmYKfumD (Startup advisory platform) applying to YC S24 || make something people want!
Daanish @danishabbir
632 Followers 5K Following elk again. before: startup founder, ml eng (e.g. @nvidia), ee + english (@stanford)
Ronald Simons @RonaldSimons
87 Followers 736 Following CEO building investor relationships at Treenia, an Early-Stage Data Point Domain Name Registrar Startup | Reader | Chess FIDE Legend | Views are my own
Skinny Satan @SkinnySatans
645 Followers 96 Following Greetings everyone. We are the Satanic Tech cult, welcome to the darkside of the Geekzone follow me to know more
T J @tdj11100
313 Followers 4K Following TJ completed a Ph.D. in Physics and then moved into the tech world.
Tyler Bruno @tylerbruno05
902 Followers 415 Following CS/AI Undergrad at @dartmouth. Transforming curiosity into actions in pursuit of a better world.
Joe Fredrick @fredric11642
3 Followers 54 Following
ma @ma52987379
0 Followers 120 Following
Javier Buitrago 🚢 @javbuitrago
273 Followers 893 Following Leading technical recruiting @playground_ai
Nate Boyd @n8boyd
695 Followers 2K Following Invest in and help build deep tech & AI startups ~ dad & partner ~ curious & skeptical
TommyTang @Tommy_Tang_930
19 Followers 223 Following
Pavan @pavantechworld
0 Followers 211 Following
Ole Jonas @friendly_tweedy
50 Followers 500 Following
Mishari Almishari @malmishari
4K Followers 518 Following
AK @_akhaliq
310K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo follow on Hugging Face: https://t.co/q2Qoey80Gx
Jim Fan @DrJimFan
229K Followers 3K Following @NVIDIA Sr. Research Manager & Lead of Embodied AI (GEAR Lab). Creating foundation models for Humanoid Robots & Gaming. @Stanford Ph.D. @OpenAI's first intern.
Sergey Levine @svlevine
80K Followers 122 Following Associate Professor at UC Berkeley Co-founder, Physical Intelligence
Soumith Chintala @soumithchintala
187K Followers 883 Following Cofounded and lead @PyTorch at Meta. Also dabble in robotics at NYU. AI is delicious when it is accessible and open-source.
Danijar Hafner @danijarh
14K Followers 869 Following Building AI that makes autonomous decisions using world models, artificial curiosity, and temporal abstraction @DeepMind
Jeremy Howard @jeremyphoward
222K Followers 5K Following 🇦🇺 Co-founder: @AnswerDotAI & @FastDotAI ; Hon Professor: @UQSchoolITEE ; Digital Fellow: @Stanford
Karol Hausman @hausman_k
22K Followers 141 Following @Physical_int ex: researcher @GoogleAI/@DeepMind, adj. Prof. @Stanford. Into robots, AI, NBA, philosophy, soccer and almond croissants. 🇵🇱🇺🇸
Horace He @cHHillee
24K Followers 449 Following Working at the intersection of ML and Systems @ PyTorch "My learning style is Horace twitter threads" - @typedfemale
Anthropic @AnthropicAI
262K Followers 26 Following We're an AI safety and research company that builds reliable, interpretable, and steerable AI systems. Talk to our AI assistant Claude at https://t.co/aRbQ97uk4d.
Richard Socher @RichardSocher
101K Followers 971 Following CEO @youSearchEngine Investing at @aixventuresHQ Before: Stanford Adj Prof in AI/NLP, Chief Scientist at Salesforce, MetaMind
Shane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)
Ilya Sutskever @ilyasut
370K Followers 2 Following towards a plurality of humanity loving AGIs @openai
Oriol Vinyals @OriolVinyalsML
167K Followers 82 Following VP of Research & Deep Learning Lead, Google DeepMind. Gemini co-lead. Past: AlphaStar, AlphaFold, AlphaCode, WaveNet, seq2seq, distillation, TF.
Sander Dieleman @sedielem
50K Followers 2K Following Research Scientist at Google DeepMind. I tweet about deep learning (research + software), music, generative models (personal account).
Richard Sutton @RichardSSutton
26K Followers 37 Following Student of mind and nature, libertarian, chess player, cancer survivor. @ Keen Technologies, UAlberta, Amii, RLAI, The Royal Society, RichSutton.eth
Jack Clark @jackclarkSF
68K Followers 5K Following @AnthropicAI, ONEAI OECD, co-chair @indexingai, writer @ https://t.co/3vmtHYkaTu Past: @openai, @business @theregister. Neural nets, distributed systems, weird futures
Natasha Jaques @natashajaques
25K Followers 1K Following Senior Research Scientist at @GoogleAI and Assistant Professor @uwcse. Social Reinforcement Learning in multi-agent and human-AI interactions. PhD from @MIT.
Joshua Achiam ⚗️ @jachiam0
14K Followers 949 Following Human. Trying to make safe alchemy machines. Thinking about humanist alchemism (h/alc ⚗️, maybe). Main author of https://t.co/cKuSh210l1
Jonathan Frankle @jefrankle
16K Followers 684 Following Chief Scientist, Neural Networks @Databricks via MosaicML. PhD @MIT_CSAIL. BS/MS @PrincetonCS. DC area native. Making AI efficient for everyone at @DbrxMosaicAI
Stephanie Zhan @stephzhan
19K Followers 2K Following GP @Sequoia. Preseed/Seed/A. Boards @linear @middeskhq @recroom Seeds: @replicatehq, https://t.co/qBx7xi7Qrd, 5 in stealth: robotics, AI agents, & more. AI, SaaS, dev tools.
Daniel Han @danielhanchen
7K Followers 941 Following Building @UnslothAI. Finetune LLMs 30x faster https://t.co/aRyAAgKOR7. Prev ML at NVIDIA. Hyperlearn used by NASA. I like maths, making code go fast
Richie Steigerwald @richie_internet
107 Followers 120 Following Math, Philosophy, Design, Engineering, Fairness, and Games. DeepMind. Black Lives Matter. he/him
Enrique Piqueras @epiqueras1
2K Followers 234 Following Organizing the world's information and making it universally accessible and useful using JAX @Google @Deepmind.
Tony Z. Zhao @tonyzzhao
12K Followers 785 Following CS PhD student @Stanford. Aspiring full-stack roboticist. Prev Deepmind, Tesla, GoogleX, Berkeley.
killian @hellokillian
23K Followers 438 Following building a universal interface between language models and computers ● https://t.co/yJVGuC0xlD
Mark Chen @markchen90
10K Followers 246 Following Head of Frontiers Research at OpenAI. Coach for the USA IOI Team.
Ideogram @ideogram_ai
39K Followers 0 Following Helping people become more creative. It's pronounced eye-diogram. Join our lovely community at https://t.co/aKDNl4OOQf.
John Arnold @JohnArnoldFndtn
78K Followers 388 Following Co-chair of Arnold Ventures. Fighting special interests and status quo bias to build better systems for people.
Trending GitHub Repos.. @trending_repos
18K Followers 0 Following Tweeting the most starred GitHub repository of the: 📈 day - every day 🏅 week - every Monday 🏆 month - every 1st of the month
Sherjil Ozair @sherjilozair
6K Followers 3K Following prev: autopilot @tesla, deep learning @googledeepmind, phd https://t.co/dxgb6gimCf, cs @iitdelhi
Julian Schrittwieser @Mononofu
4K Followers 68 Following Principal Research Engineer at DeepMind Gemini RL AlphaGo, AlphaZero, MuZero, AlphaCode, AlphaTensor 日本語を勉強している
Mikhail Parakhin @MParakhin
17K Followers 21 Following
Aman Sanger @amanrsanger
15K Followers 656 Following building @cursor_ai at @anysphere https://t.co/EdcQJ2dv0J | https://t.co/vJ5zNuT6WO
bleedingedge.ai @bleedingedgeai
10K Followers 6 Following
Louis Castricato @lcastricato
3K Followers 477 Following Math @uwaterloo, RLHF @BrownCSDept, Goosefluencer. x-RS @aieleuther, x-Head of LLMs @stabilityai, x-lead @CarperAI. co-founder @synth_labs. We're hiring.
Sam Whitmore @sjwhitmore
12K Followers 2K Following building @newcomputer. not a cat (or a man) in real life. I like to run a lot! @kensho @harvard @StuyNY
AI Pub @ai__pub
72K Followers 343 Following AI papers and AI research explained, for technical people. Get hired by the best AI companies: https://t.co/MySVjUGOQ3
Roshan Rao @proteinrosh
2K Followers 578 Following he/him. Proteins, evolutionary models, unsupervised learning. Prev: RS @MetaAI, PhD @berkeley_ai.
Stability AI @StabilityAI
190K Followers 31 Following We are building the foundation to activate humanity's potential.
DJ Strouse @djstrouse
1K Followers 621 Following Reasoning about reasoning. Technically a member of staff @GoogleDeepMind. Previously, PhD @Princeton.
Fabio Pardo @PardoFab
1K Followers 442 Following Research Scientist at @GoogleDeepMind Toronto. Previously at @ImperialCollege, @Sorbonne_Univ_ and @ENS_ULM. Author of the Tonic RL library.
Mohammed AlQuraishi @MoAlQuraishi
10K Followers 359 Following MLing biomolecules en route to structural systems biology. Asst Prof of Systems Biology and CS @Columbia. Prev. @Harvard SysBio; @Stanford Genetics, Stats.
AIX Ventures @aixventureshq
2K Followers 129 Following AI-focused venture firm investing in early-stage companies. https://t.co/kxnv03Rtlz. @pabbeel @antgoldbloom @chrmanning @richardsocher @shaunbjohnson
Viktor Blåsjö @viktorblasjo
4K Followers 794 Following History of mathematics; implications for historiography and philosophy of science, education; polemics thereof.
Brendan O'Donoghue @bodonoghue85
3K Followers 1K Following Research scientist at @GoogleDeepMind, working on generative models, deep learning, RL. PhD from @stanford.
Patrick McKenzie @patio11
164K Followers 796 Following I work for the Internet and am an advisor to @stripe. These are my personal opinions unless otherwise noted.
Max Roser @MaxCRoser
287K Followers 1K Following Data to understand global problems and research to make progress against them. Founder of @OurWorldInData / Professor at @UniofOxford's @BlavatnikSchool
Richard Morris @ahistoryinart
76K Followers 3K Following Brief lives of great painters, dealer in 19thC/20th British and European art https://t.co/SdtZyAiQG3 email: [email protected]
Horace Dediu @asymco
69K Followers 424 Following https://t.co/iOcMYL7Ksg https://t.co/XbOebFdfbw https://t.co/jXz3kWqoMC
Steve Stewart-William.. @SteveStuWill
193K Followers 238 Following Psychology, evolution, science. Author of "The Ape That Understood the Universe" (2018) and "Darwin, God and the Meaning of Life" (2010). Backup: @SteveStuWill2
Massimo @Rainmaker1973
2.0M Followers 220 Following Engineer. Selecting and curating pictures and videos trying to add context, source and explanation to science, tech, art and weather topics
Michael Nielsen @michael_nielsen
96K Followers 6K Following Searching for the numinous 🇦🇺 🇨🇦, home in 🇺🇸 Research @AsteraInstitute https://t.co/maezekzRUb
David Warde-Farley �.. @dwf
6K Followers 2K Following Scientist @DeepMind. (Dormant) personal account; tweets reflect my views alone. he/him
Sergey Ovchinnikov �.. @sokrypton
12K Followers 3K Following Scientist, Assistant Professor @MITBiology, #FirstGen, ProteinBERTologist
Programming Wisdom @CodeWisdom
280K Followers 2K Following Programming wisdom and quotes throughout the years. The Knuth, the whole Knuth, and nothing but the Knuth, so help me Codd.
Francis Davidson @FDavidsonT
3K Followers 1K Following Founded Sonder in college, took it public in a $2.2B IPO 8 years later. Fascinated by creativity, prediction, culture, hospitality, cities and education.
𝔊𝔴𝔢𝔯𝔫 @gwern
42K Followers 88 Following Internet besserwisser; pedantic, mean reply guy. 𝘞𝘢𝘵𝘢𝘴𝘩𝘪 𝘬𝘪𝘯𝘪𝘯𝘢𝘳𝘪𝘮𝘢𝘴𝘶! (Follow requests ignored due to terrible UI.)
andy jones @andy_l_jones
4K Followers 326 Following engineering & research at @AnthropicAI. DC, SF, London
Ravi Gupta @GuptaRK22
21K Followers 410 Following @Sequoia || @Instacart COO/CFO before || @benchling, @faire_wholesale, @instacart, @meter, @remote, @sierraplatform, @tryramp & others || https://t.co/nbHEiIMCUT
Brandon Amos @brandondamos
14K Followers 2K Following research scientist @MetaAI (FAIR) | optimization, machine learning, control, and reinforcement learning | PhD from @SCSatCMUToday I’m thrilled to announce @Lux_Capital's NYC AI Directory & NYC AI Map - 2 resources for the burgeoning AI talent ecosystem READ MORE👇 luxcapital.com/news/the-great… NYC AI Directory: airtable.com/appK49oThZBOTS… NYC AI Map: felt.com/map/LUX-NYC-tk…
Thanks for all the love and support today! We're hiring: perplexity.ai/hub/careers. Come build the future with us. Together, we can shape how people consume information online in the years to come!
we put the 01 into @Grimezsz spider
Easily Fine-tune @AIatMeta Llama 3 70B! 🦙 I am excited to share a new guide on how to fine-tune Llama 3 70B with @PyTorch FSDP, Q-Lora, and Flash Attention 2 (SDPA) using @huggingface build for consumer-size GPUs (4x 24GB). 🚀 Blog: philschmid.de/fsdp-qlora-lla… The blog covers: 👨💻…
We have just released 🍷 FineWeb: 15 trillion tokens of high quality web data. We filtered and deduplicated all CommonCrawl between 2013 and 2024. Models trained on FineWeb outperform RefinedWeb, C4, DolmaV1.6, The Pile and SlimPajama!
My only regret with Lambda School is that I’m not wealthy enough to pay Austen & Lambda’s fines out of pocket because I’ve payed about that much in income tax since I graduated.
We studied In-Context learning with hundreds to thousands of examples. My favorite example: I sent *one million* tokens to Gemini 1.5 Pro for linear classification with 64 dimensional integer-valued vectors and many-shot learning performs similarly to k-Nearest Neighbours.
We're joining AI Grant 🅱️🎉
@browserbasehq is building the piece of infrastructure that every AI application needs: a programmable web browser.
🚀 Today I’m excited to announce Superblocks Embedded Apps – Build 10x faster and embed Superblocks into your legacy internal apps or customer portals using our React and JS SDKs. ⚡ One of the most common questions I get asked from customers is “Why can I only use Superblocks…
Introducing 𝐀𝐋𝐎𝐇𝐀 𝐔𝐧𝐥𝐞𝐚𝐬𝐡𝐞𝐝 🌋 - Pushing the boundaries of dexterity with low-cost robots and AI. @GoogleDeepMind Finally got to share some videos after a few months. Robots are fully autonomous filmed in one continuous shot. Enjoy!
Excited to introduce a new project I've been working on called Payman! Payman is an AI Agent tool that gives Agents the ability to pay people for tasks they cannot do themselves. While many people imagine a future where humans pay AI agents for services they want completed,…
It's been a wild ride. Just 20 of us, burning through thousands of H100s over the past months, we're glad to finally share this with the world! 💪 One of the goals we’ve had when starting Reka was to build cool innovative models at the frontier. Reaching GPT-4/Opus level was a…
Meet Reka Core, our best and most capable multimodal language model yet. 🔮 It’s been a busy few months training this model and we are glad to finally ship it! 💪 Core has a lot of capabilities, and one of them is understanding video --- let’s see what Core thinks of the 3 body…
at least once a week, i get a text like this from a different 10X engineer. headless browsers are hard!
🎨Spent some time refactoring the 2021 post on diffusion model with new content: lilianweng.github.io/posts/2021-07-… ⬇️ ⬇️ ⬇️ 🎬Then another short piece on diffusion video models: lilianweng.github.io/posts/2024-04-… (Yes, I had an intensive weekend🥹)
@kchonyc @LightningAI Not exactly solving your issue, but if you're looking for free GPUs, Colab has free 65 TFLOPs Tesla T4 GPUs. Also have a Colab for Gemma 2b which makes HF inference natively 2X faster. 64 tokens ~4.8s. Finetuning is also 2x faster and uses 70% less VRAM. colab.research.google.com/drive/15gGm7x_…
How do LLMs scale to million token context window? Ring Attention is a nice trick to parallelize long sequence across devices and rotate them in a ring with zero overhead scaling. In our new blog, we cover the tricks behind this magic. It looks like this (1/5🧵)
Colab now offers TPU VM runtimes! The TPU runs locally rather than across the network. This improves reliability, debuggability, and enables support for JAX 0.4.x on TPU! Try it out by selecting the "TPU v2" accelerator or try this Gemma + TPU notebook! colab.research.google.com/github/googlec…
Why does Kahneman-Tversky Optimization (KTO) achieve successful alignment, even though it uses only binary signals? Check our Binary Classifier Optimization (BCO), which uncovers the connection between Direct Preference Optimization (DPO) and alignment from binary signal.
🎮 Introducing the new and improved Policy-Guided Diffusion! Vastly more accurate trajectory generation than autoregressive models, with strong gains in offline RL performance! Plus a ton of new theory and results since our NeurIPS workshop paper... Check it out ⤵️
Trends for United States
You might like
