Dan Fu @realDanFu
CS PhD Candidate at Stanford, systems for machine learning. Sometimes YouTuber/podcaster. Academic Partner, @togethercompute. danfu.org Joined September 2019-
Tweets581
-
Followers4K
-
Following176
-
Likes978
Today we are thrilled to share that we’ve raised $106M in a new round led by @SalesforceVC with participation from @coatuemgmt and our existing investors. Our vision is to rapidly bring innovations from research to production and to ultimately build the best platform we can for…
Thrilled to announce our investment in @togethercompute! @vipulved, @ce_zhang, @percyliang & the rest of the team are building open-source AI infra for enterprises with a research-to-production velocity that far outpaces the competition. Read more: salesforceventures.com/perspectives/w…
📢 Announcing our new speculative decoding framework Sequoia ❗️❗️❗️ It can now serve Llama2-70B on one RTX4090 with half-second/token latency (exact❗️no approximation) 🤔Sounds slow as a sloth 🦥🦥🦥??? Fun fact😛: DeepSpeed -> 5.3s / token; 8 x A100: 25ms / token (costs 8 x…
We are excited to present Caduceus: bi-directional DNA language model built on Mamba, with long range modeling that respects inherent symmetry of double helix DNA structure. Caduceus is SoTA on several benchmarks, including identifying causal SNPs for gene expression. 🧵1/9
1st of a couple new goodies this week Releasing our Based preprint, code, initial models Like others, we’ve found attention is still great. But 3 simple ideas to make it better: ☝️Too expensive? Use exact attn in small sliding windows ✌️Doesn’t capture long range? Fill in…
1st of a couple new goodies this week Releasing our Based preprint, code, initial models Like others, we’ve found attention is still great. But 3 simple ideas to make it better: ☝️Too expensive? Use exact attn in small sliding windows ✌️Doesn’t capture long range? Fill in…
Stoked to be sharing Based! We find that the simple combo of linear and sliding window attention can enable 24x higher throughput than Transformers. Had a ton of fun diving deep on the tradeoffs that govern these recurrent models! arxiv.org/abs/2402.18668 github.com/HazyResearch/b…
Stoked to be sharing Based! We find that the simple combo of linear and sliding window attention can enable 24x higher throughput than Transformers. Had a ton of fun diving deep on the tradeoffs that govern these recurrent models! arxiv.org/abs/2402.18668 github.com/HazyResearch/b… https://t.co/5Bi0kPhwKA
Excited to release Based, an architecture that combines two✌️ simple, familiar, attention-like primitives – short (size-64) sliding window attention and softmax-approximating linear attention – to enable high quality and efficient inference! 💨 🚀 joint w/ @EyubogluSabri,…
Current protein models (ESM-2, AlpaFold2,...) only encode the 20 wild-type amino acids -- what about PTMs, which significantly influence the diversity of the proteome? 💁♂️To solve this, we present the first PTM-aware protein language model, PTM-Mamba! biorxiv.org/content/10.110…
Is DNA all you need? Introducing Evo, a long context 7B foundation model for biology Evo has SOTA *zero-shot* prediction across DNA, RNA, and protein modalities Evo can generate DNA, RNA+proteins & make CRISPR-Cas systems for first time blog …n-model-tool-arc-institute.vercel.app/news/blog/evo
Our paper "Diffusion Models without Attention" has been accepted by #CVPR2024! Congrats to our amazing collaborators @NathanYan2012 @srush_nlp ! More SSM + Diffusion will come!
Our paper "Diffusion Models without Attention" has been accepted by #CVPR2024! Congrats to our amazing collaborators @NathanYan2012 @srush_nlp ! More SSM + Diffusion will come!
My final PhD chapter on improving seizure detection with @HazyResearch and @rubinqilab was just published @npjDigitalMed. TL;DR We found that scaling two dimensions of model supervision: (1) coverage of training data and (2) granularity of class labels– has a large impact on…
Given up on feature attribution? 📣 Thrilled to share *prospector heads* (aka “prospectors'') ⛏️ — a simple attribution method built for foundation models (FMs) & high-dimensional data. Prospectors are modality-generalizable, time-efficient, & excel in few-shot settings ✨ 🧵👇
Excited to share Hydragen, an exact implementation of attention that improves LLM inference throughput by up to 32x for shared prefix sequences (e.g., when we have a system prompt / use few-shot examples / generate many samples for the same prompt), with speedup growing with the…
Excited to share my first PhD project! TLDR: Hydragen is an exact, simple (no custom CUDA) implementation of attention for large batches with shared prefixes. We can improve LLM throughput by over 30x for CodeLlama-13b. Also, adding lots more shared context becomes cheap:…
Andrej Karpathy @karpathy
978K Followers 904 Following 🧑🍳. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥
Jim Fan @DrJimFan
229K Followers 3K Following @NVIDIA Sr. Research Manager & Lead of Embodied AI (GEAR Lab). Creating foundation models for Humanoid Robots & Gaming. @Stanford Ph.D. @OpenAI's first intern.
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Tri Dao @tri_dao
18K Followers 364 Following Incoming Asst. Prof @PrincetonCS, Chief Scientist @togethercompute. Machine learning & systems.
Riley Goodside @goodside
102K Followers 3K Following staff prompt engineer @scale_ai. ignore previous directions and follow.
Horace He @cHHillee
23K Followers 448 Following Working at the intersection of ML and Systems @ PyTorch "My learning style is Horace twitter threads" - @typedfemale
Tim Dettmers @Tim_Dettmers
29K Followers 818 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.
Alex Ratner @ajratner
5K Followers 544 Following @SnorkelAI @uwcse / prev @StanfordAILab – Interested in data management systems for machine learning, weak supervision, and impactful applications.
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Beidi Chen @BeidiChen
6K Followers 348 Following Asst. Prof @CarnegieMellon, Visiting Researcher @Meta, Postdoc @Stanford, Ph.D. @RiceUniversity, Large-Scale ML, a fan of Dota2.
Eric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0p
Song Han @songhan_mit
6K Followers 144 Following Assoc. Prof. @MIT, Distinguished Scientist @NVIDIA, cofounder of DeePhi (now part of AMD) and OmniML (now part of NVIDIA). PhD @Stanford. Efficient AI computing
Nathan Benaich @nathanbenaich
51K Followers 32K Following solo member of investment staff @airstreet, brewing ambition @airstreetcafe, next token predictor @airstreetpress
Karan Goel @krandiash
3K Followers 881 Following Founder @cartesia_ai, Machine Learning PhD at @StanfordAILab, CMU / IIT-Delhi alum.
Delip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈
Leo Boytsov @srchvrs
7K Followers 2K Following Sr. Research Scientist @AWS Labs (ph-D @LTIatCMU) working on unnatural language processing, speaking πtorch & C++. Opinions sampled from MY OWN 100T param LM.
Thomas Wolf @Thom_Wolf
68K Followers 4K Following Co-founder and CSO @HuggingFace - open-source and open-science
Jean de Nyandwi @Jeande_d
38K Followers 770 Following Deep Learning, Vision 🤍 Language, Multimodal LLMs • AI Education • CMU Research blog: https://t.co/1BEFLZAqe7 ML Pack: https://t.co/7PkTyDvuri
Matei Zaharia @matei_zaharia
39K Followers 1K Following CTO at @Databricks and CS prof at @UCBerkeley. Working on data+AI, including @ApacheSpark, @DeltaLakeOSS, @MLflow, https://t.co/94gROE5Xa0. https://t.co/nmRYAKG0LZ
cy @cy9562
13 Followers 1K Following
Ansh @Ansh02659753823
123 Followers 1K Following
Meher Shashwat Nigam @ShashwatNigam99
324 Followers 985 Following Master's CS @GeorgiaTech. Prev-Analyst at @GoldmanSachs. CS grad @iiit_hyderabad. Interested in computer vision and generative AI!
Yong Dai @daiyongya
10 Followers 274 Following Researcher in Tencent AI Lab, focusing on LLM pretraining, SFT, alignment, agent, and multi-media. Previously @Microsoft and @Westlake U.
Changqing Fu @evergreencqfu
44 Followers 381 Following PhD student in Computer Vision and Machine Learning in Univ. Paris 9 - PSL
Weiyan Shi @shi_weiyan
3K Followers 682 Following Postdoc @StanfordNLP, incoming assistant professor @Northeastern, PhD @Columbia| Prev Intern @MetaAI |Co-created CICERO | persuasive chatbots + privacy #nlproc
Urmish Thakker @UrmishThakker
423 Followers 1K Following LLM @SambanovaAI | | Ex-@arm research| @mlperf1| @BigscienceW| @TXInstruments,@AMD| @WisconsinCS| @bitspilaniindia
Raeid Saqur @RaeidSaqur
553 Followers 500 Following PhD candidate @UofTCompSci, @VectorInst | Fulbright Scholar @PrincetonCS | MBA @Rotman School of Management |
Shuang @Footfish
213 Followers 303 Following
Gabin MAURY @csgmaury
12 Followers 86 Following Robotics, AI and low level programming enthusiast Software engineer apprentice in R&D (my contract explicitly prohibits me from saying where on social media😭)
Arkadiy Saakyan @rkdsaakyan
139 Followers 387 Following PhD student @ColumbiaCompSci @columbianlp working on natural language processing. prev. intern @AmazonScience
Magnus Petersen @Omorfiamorphism
792 Followers 1K Following Deep Learning methods dev for creativity 👾🖼️| Ph.D. student researching deep learning for molecular dynamic simulations @fias_science @CovinoLab
Nikhil Namburi @nikhilvnamburi
33 Followers 217 Following Venture @Lux_Capital | previously @InsightPartners @UCSF @Columbia
Vikranth Kanumuru @kanlanc
119 Followers 883 Following A Curious Fellow in love with Technology, Studying @cornell — Featured in ABC Australia | 6xTop Writer Medium
U @deee1f9b7c28f1
0 Followers 888 Following Ugly bag of mostly water. Still too arrogant, too primitive.
Ivan Timoshenko @JTaurus19
25 Followers 334 Following Co-Founder @ClickTheRoad | CPO Software engineer
Amrit Singh Bedi @amritsinghbedi3
522 Followers 1K Following CS Faculty at UCF (AlignAI Lab), previous @UMD @ARL @IITK Interested in RL, Nonconvex Optimization, AI text Detection, Federated Learning, Robotics
chenlailin @chenlailin
28 Followers 164 Following Using twitter for only one purpose: bookmark research papers
biscotte wong @biscottew
9 Followers 48 Following
Ismail Chaida 👨�.. @Ismail_CHAIDA
409 Followers 4K Following Software & Data/Kotlin/Scala Engineer | Views are my own
David Stafford @davidstafford
704 Followers 2K Following AI and robotics. Bit twiddling. Opinions are my own.
لبنان مغنية @lebmogh
177 Followers 2K Following الدنيا كلها جهل، الّا مواضع العلم والعلم كله جهل، الّا ما عُمِل به والعمَل كله رياء، الّا ما كان مخلصاً والاخلاص على خطَر، حتى ينظرَ العبد بما يُختم له
vamshi kumar @vamshirocks
22 Followers 604 Following
emanon @JianSuji
76 Followers 1K Following
Aayush Srivastava @aayunomics
870 Followers 3K Following Co-Founder, Solutions Center @GoogleCloud | Previous: Startup PM/BD @aws | @Columbia_Biz ‘26| @NLUD_official ‘14 |
Vi @AvimanyuRoy3
576 Followers 2K Following 🍎🕊/🦦☕️/😴🛌/he/him Shouting into the Void (TM) GPU poor peasant
Rishab Verma @Rishab5595Verma
84 Followers 2K Following
techmaraudersmap @techmaruadermap
2 Followers 84 Following I write about #AI #MLjobs #softwareengineering #aiengineering #softskills.
Ervin Lang @ervinlang
48 Followers 1K Following
Agamdeep Singh @agammessi10
44 Followers 722 Following Trying to make a business out of RAG and training a foundational pose comparison model @ MOON lab, IISERB.
Bhavin Jawade @BhavinJawade
362 Followers 3K Following Ph.D Candidate at University at Buffalo @UBuffalo | Research Scientist Intern @Yahoo | Ex. Research Scientist Intern at Adobe Research @Adobe
Zekun Wang (Seeking 2.. @ZenMoore1
2K Followers 673 Following 🥷 #LLM #AGI Research Intern @01AI_Yi @hkust @ETH; 💼 Formerly @BAAIBeijing #Langboat; 🔥 Looking for #25Fall PhD!
Linz @lin72h
174 Followers 4K Following Someday I'm gonna make great machines that fly. And me and my friends are gonna go flying together, into the forever and beautiful sky.
ИΛVIY @yivannaviy
101 Followers 274 Following All tweets are generated from a poorly trained neural network.
Shashank @5hv5hvnk
167 Followers 865 Following pre doc @prosemsft working mostly on ml, little on pl. | TIET23
Andrej Karpathy @karpathy
978K Followers 904 Following 🧑🍳. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥
Yann LeCun @ylecun
710K Followers 718 Following Professor at NYU. Chief AI Scientist at Meta. Researcher in AI, Machine Learning, Robotics, etc. ACM Turing Award Laureate.
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Tri Dao @tri_dao
18K Followers 364 Following Incoming Asst. Prof @PrincetonCS, Chief Scientist @togethercompute. Machine learning & systems.
Horace He @cHHillee
23K Followers 448 Following Working at the intersection of ML and Systems @ PyTorch "My learning style is Horace twitter threads" - @typedfemale
Tim Dettmers @Tim_Dettmers
29K Followers 818 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.
Alex Ratner @ajratner
5K Followers 544 Following @SnorkelAI @uwcse / prev @StanfordAILab – Interested in data management systems for machine learning, weak supervision, and impactful applications.
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Beidi Chen @BeidiChen
6K Followers 348 Following Asst. Prof @CarnegieMellon, Visiting Researcher @Meta, Postdoc @Stanford, Ph.D. @RiceUniversity, Large-Scale ML, a fan of Dota2.
Eric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0p
Song Han @songhan_mit
6K Followers 144 Following Assoc. Prof. @MIT, Distinguished Scientist @NVIDIA, cofounder of DeePhi (now part of AMD) and OmniML (now part of NVIDIA). PhD @Stanford. Efficient AI computing
Karan Goel @krandiash
3K Followers 881 Following Founder @cartesia_ai, Machine Learning PhD at @StanfordAILab, CMU / IIT-Delhi alum.
Jeff Dean (@🏡) @JeffDean
296K Followers 6K Following Chief Scientist, Google DeepMind and Google Research. Co-designer/implementor of things like @TensorFlow, MapReduce, Bigtable, Spanner, Gemini .. (he/him)
Jean de Nyandwi @Jeande_d
38K Followers 770 Following Deep Learning, Vision 🤍 Language, Multimodal LLMs • AI Education • CMU Research blog: https://t.co/1BEFLZAqe7 ML Pack: https://t.co/7PkTyDvuri
Matei Zaharia @matei_zaharia
39K Followers 1K Following CTO at @Databricks and CS prof at @UCBerkeley. Working on data+AI, including @ApacheSpark, @DeltaLakeOSS, @MLflow, https://t.co/94gROE5Xa0. https://t.co/nmRYAKG0LZ
AI Pub @ai__pub
72K Followers 342 Following AI papers and AI research explained, for technical people. Get hired by the best AI companies: https://t.co/MySVjUGOQ3
Jack Rae @drjwrae
9K Followers 353 Following Principal Scientist @ Google DeepMind Work on Gemini 💎♊ Compression is all you need LLMs (e.g. Gopher, Chinchilla, Gemini) 💼 Past: OpenAI, Quora
Aakanksha Chowdhery @achowdhery
7K Followers 3K Following LLMs @ Google DeepMind :: PaLM, Gemini // Previously @MSFTResearch, @Stanford, @Princeton // views my own and subject to change
Jordan Juravsky @jordanjuravsky
247 Followers 160 Following AI Research | PhD Student at Stanford. Proud former goose at UWaterloo.
Sen Wu @Wu_Sen
172 Followers 146 Following
jack morris @jxmnop
10K Followers 760 Following getting my phd in nlp @cornell_tech 🚠 // academic optimist // tweeting from the snack aisle at trader joes
Abhi Venigalla @abhi_venigalla
5K Followers 1K Following Researcher @Databricks. Former @MosaicML, @CerebrasSystems. Addicted to all things compute.
Eric Nguyen @exnx
2K Followers 325 Following PhD in BioEngineering & AI @stanford @HazyResearch @StanfordAILab @arcinstitute
Siyi Tang @SiyiTang_
297 Followers 287 Following Machine Learning Scientist @arteraAI | #MachineLearning for Medicine | PhD @Stanford
Gautam Machiraju 🌺 @gmachiraju
650 Followers 4K Following PhD-ing @StanfordAILab w/ @ParagMallick @HazyResearch🌲 AI-driven data copilots for scientific discovery♟️🧬🔬🛰🔭 Powered by prog house, people, 3rd places 🪩✨
Nathan Lambert @natolambert
25K Followers 688 Following Figuring out AI @allen_ai, "rl boi" DM me papers. Writes @interconnectsai, talks @retortai Has phd and some credentials
Teknium (e/λ) @Teknium1
29K Followers 3K Following Cofounder @NousResearch, prev @StabilityAI Github: https://t.co/LZwHTUFwPq HuggingFace: https://t.co/sN2FFU8PVE Support me on Github Sponsors
derek guy @dieworkwear
809K Followers 963 Following Menswear writer. Editor at @putthison. Creator of @RLGoesHard. Bylines at The New York Times, The Washington Post, The Financial Times, Esquire, and Mr. Porter
Jon Saad-Falcon @JonSaadFalcon
438 Followers 188 Following CS PhD @StanfordAILab @hazyresearch | Previously @databricks @allen_ai @GeorgiaTech
Willie Neiswanger @willieneis
1K Followers 204 Following Assistant Professor @USC in CS + AI. Previously @Stanford, @SCSatCMU. Machine Learning, Decision Making, AI-for-Science, Generative AI, Uncertainty, ML Systems.
Josh Robinson @Josh_d_robinson
719 Followers 368 Following Postdoc at @Stanford. PhD from @MIT_CSAIL.
Jade Lai @jadelai__
2K Followers 1K Following Partner @ Coatue | formerly enterprise investment partner @a16z, investor @Playground_VC | proud 🇨🇦
Daniele Paliotta @DanielePaliotta
312 Followers 1K Following ML PhD @Unige_en, and other things. Building https://t.co/Zn3q5oZuXR
Antoine SIMOULIN @antoinesimoulin
154 Followers 198 Following I am currently completing my Ph.D. in Natural Language Processing at Paris University in a joint program sponsored by Quantmetry.
ES-FoMo@ICML2023 @ESFoMo
168 Followers 33 Following Efficient Systems for Foundation Models Workshop, ICML2023. Join us if you are interested in the challenges associated with large models training & inference!
Khosla Lab @KhoslaLab
26 Followers 64 Following Chemical Biology @Stanford | Studying the mysteries of PKS, Celiac Disease, and LAC | Student run account Tweets by Chaitan signed CK
Karina Nguyen @karinanguyen_
12K Followers 646 Following AI research & eng @AnthropicAI, prev. intern @nytimes, @square, @dropbox
Nicolas Machado @machado___nic
714 Followers 961 Following Cofounder @TryLume (YC W23) | AI @stanford | Forbes 30u30 🇧🇷
Joan Kim @joanofdao
3K Followers 2K Following Investor @samsungnext | Supporting @aleohq, @axieinfinity, @coframe_ai, @offchainlabs, @mysten_labs, @Spectral_Labs et al | @cornell
rishi @RishiBommasani
4K Followers 2K Following Stanford CS PhD @StanfordCRFM @StanfordNLP @StanfordAILab @StanfordHAI Advisers: @percyliang @jurafsky Previous: @CornellCIS @clairecardie #FoundationModels
Tatsunori Hashimoto @tatsu_hashimoto
6K Followers 202 Following Assistant Prof at Stanford CS, member of @stanfordnlp and statsml groups; Formerly at Microsoft / postdoc at Stanford CS / Stats.
Yejin Choi @YejinChoinka
19K Followers 330 Following professor at UW, director at AI2, adventurer at heart
Raphael Townshend @raphaeljlt
1K Followers 114 Following Founder & CEO of Atomic AI (https://t.co/lb3M8gEIaF, we are hiring!). Forbes 30u30. CS PhD @StanfordAILab. Machine Learning, Structural Biology.
Stella Biderman @BlancheMinerva
15K Followers 749 Following Open source LLMs and interpretability research at @BoozAllen and @AiEleuther. My employers disown my tweets. She/her
Colin Raffel @colinraffel
30K Followers 654 Following nonbayesian parameterics, sweet lessons, and random birds. Friend of @srush_nlp
Susan Zhang @suchenzang
20K Followers 504 Following @ Google Deepmind. Past: @MetaAI, @OpenAI, @unitygames, @losalamosnatlab, @Princeton etc. Always hungry for compute.
Michael Poli @MichaelPoli6
2K Followers 278 Following @Stanford @StanfordAILab, Staff Scientist @togethercompute, prev @MSFTResearch. DL, numerics and systems. I like to architect big neural nets that run fast.
Armin W. Thomas @ai_with_brains
742 Followers 970 Following AI and Computational Neuroscience Postdoc at Stanford University working with {@russpoldrack, @HazyResearch, @StanfordData} | He/him
Hao Zhang @haozhangml
3K Followers 262 Following Asst. Prof. @HDSIUCSD and @ucsd_cse running @haoailab. Cofounder and runs @lmsysorg. 20% with @SnowflakeDB
Dylan Sam @dylanjsam
428 Followers 351 Following phd student @mldcmu | past: intern @AmazonScience, BS @BrownCSDept
Eric Steinberger @EricSteinb
7K Followers 478 Following Writing code that writes code on a mission to build safe superintelligence | CEO/cofounder @magicailabsTogether AI and Snowflake partner to bring their state-of-the-art Arctic LLM to enterprise customers. Experience Arctic on Together Inference with best in class performance. api.together.xyz/playground/cha…
There is a really nice community of researchers developing transformer alternatives. Want to highlight these impressive folks. Simran Arora (@simran_s_arora), Chunting Zhou (@violet_zct), Dan Fu (@realDanFu), and Songlin Yang (@SonglinYang4)
We are thrilled to be a launch partner for Meta Llama 3. Experience Llama 3 now at up to 350 tokens per second for Llama 3 8B and up to 150 tokens per second for Llama 3 70B, running in full FP16 precision on the Together API! 🤯 together.ai/blog/together-…
@realDanFu @arankomatsuzaki @JonSaadFalcon @HazyResearch We posted only now. Shouldn't have waited till the random committee made their decision. Yet another confirmation one should post to arxiv as soon as possible. 🥲
@realDanFu @arankomatsuzaki @JonSaadFalcon @HazyResearch This is also a bias in queries for sure. Otherwise, a well-written summary wouldn't have been sufficient. There are only a handful of queries that can be answered using a short document prefix.
@arankomatsuzaki @JonSaadFalcon @realDanFu @HazyResearch From Table 1 in your paper, truncation to 128 (I assume these are tokens) still gives you a score of 70.3 vs 94.7 for a very long sequence. Whereas if one removes relevant info from the prefix at all, truncation only gives you a random baseline preformance.
@arankomatsuzaki Great work @JonSaadFalcon @realDanFu @HazyResearch ! We have come to similar conclusions: We need better collections where suffix-truncation methods don't work. Yet, even with LOCO they still are a decent baseline. Yet, it shouldn't always be the case ↩️: x.com/srchvrs/status…
🧵📢Attention folks working on LONG-document ranking & retrieval! We found evidence of a PROFOUND issue in existing long-document collections, most importantly MS MARCO Documents. It can potentially affect all papers comparing different architectures for long document ranking.⏩
I am very happy to give this tutorial next week! We will discuss several developments on sub-quadratic long-context architectures such as SSMs, CKConv, Hyena and Mamba. Thank you @Ellis_Amsterdam for having me!
💥 We are excited that @davidwromero (@nvidia) will talk about 'Beyond Transformers: Exploring Subquadratic Long-Context Architectures' at the upcoming Deep Thinking Hour Tutorial! 📅 Thu, April 11th ⏰️09:00 - 11:00 📍L1.01 of @Lab42UvA Come and deep think with us! 🏍
Many people seem to think they can't do interesting LLM research outside a large lab, or are shoehorned into crowded topics. In reality, there are tons of wide-open high value questions. To prove it, I'll be tweeting one per week (every Monday) in 2024. Please steal my ideas!
Mark your calendars! Dyah Adila's talk on zero-shot methods for improving embeddings for foundation models is coming up on Friday, April 5th! Free & perfect for data scientists & researchers. Learn more & register: snorkel.ai/event/better-f… #LLMs #AI #AIresearch
I think I'm allowed to say this? COLM abstracts are just awesome so far, and wildly multi-disciplinery. I think this is going to be a special event.
It is currently PhD visit days at UW. Choosing among schools for a PhD is a tough choice. I wrote a blog post about some ways to think about this choice to make it easier and to find the school that is the best fit for you: timdettmers.com/2022/03/13/how…
We are excited to announce the technical program for MLSys 2024! The provisional set of accepted papers is now available on the website at mlsys.org/Conferences/20…. Register for MLSys now at mlsys.org/Register/
Today we are thrilled to share that we’ve raised $106M in a new round led by @SalesforceVC with participation from @coatuemgmt and our existing investors. Our vision is to rapidly bring innovations from research to production and to ultimately build the best platform we can for…
Thrilled to announce our investment in @togethercompute! @vipulved, @ce_zhang, @percyliang & the rest of the team are building open-source AI infra for enterprises with a research-to-production velocity that far outpaces the competition. Read more: salesforceventures.com/perspectives/w…
📢 Announcing our new speculative decoding framework Sequoia ❗️❗️❗️ It can now serve Llama2-70B on one RTX4090 with half-second/token latency (exact❗️no approximation) 🤔Sounds slow as a sloth 🦥🦥🦥??? Fun fact😛: DeepSpeed -> 5.3s / token; 8 x A100: 25ms / token (costs 8 x…
Today we're excited to introduce Devin, the first AI software engineer. Devin is the new state-of-the-art on the SWE-Bench coding benchmark, has successfully passed practical engineering interviews from leading AI companies, and has even completed real jobs on Upwork. Devin is…
If you want to know what OSS model serving API has the best performance just ask Devin to build you an objective benchmark. It builds a real-time website with comparative metrics all by itself! Truly incredible product from @cognition_labs.
Today we're excited to introduce Devin, the first AI software engineer. Devin is the new state-of-the-art on the SWE-Bench coding benchmark, has successfully passed practical engineering interviews from leading AI companies, and has even completed real jobs on Upwork. Devin is…
Excited to demonstrate Mamba's potential as the backbone of DNA language models! This significantly extends preliminary results from the original paper, and the release comes with pretrained models - one of the most common requests we've gotten :)
We are excited to present Caduceus: bi-directional DNA language model built on Mamba, with long range modeling that respects inherent symmetry of double helix DNA structure. Caduceus is SoTA on several benchmarks, including identifying causal SNPs for gene expression. 🧵1/9
We are excited to present Caduceus: bi-directional DNA language model built on Mamba, with long range modeling that respects inherent symmetry of double helix DNA structure. Caduceus is SoTA on several benchmarks, including identifying causal SNPs for gene expression. 🧵1/9
Trends for United States
You might like
