Tengyu Ma @tengyuma
Assistant professor at Stanford; Co-founder of Voyage AI (https://t.co/wpIITHLgF0) ; Working on ML, DL, RL, LLMs, and their theory. ai.stanford.edu/~tengyuma Palo Alto, CA Joined June 2011-
Tweets400
-
Followers25K
-
Following512
-
Likes197
Correction: I did the math wrong (not considering log/log scales). Sophia is ~1.6x times more efficient than Adam (thanks for pointing out @tengyuma).
Correction: I did the math wrong (not considering log/log scales). Sophia is ~1.6x times more efficient than Adam (thanks for pointing out @tengyuma).
Very cool project that not enough people talk about! Have tried this in production, voyage ai embedding provides a few percentage of final performance improvement, despite being a small segment in the entire ML pipeline! Bullish! 🫡
Very cool project that not enough people talk about! Have tried this in production, voyage ai embedding provides a few percentage of final performance improvement, despite being a small segment in the entire ML pipeline! Bullish! 🫡
Final Update: One more magnitude of testing Sophia. We're talking model sizes in the B's, tokens in the T's. Sophia once again wins out. For me at least this is clear evidence that Sophia may be a replacement for Adam even in large scale runs.
Final Update: One more magnitude of testing Sophia. We're talking model sizes in the B's, tokens in the T's. Sophia once again wins out. For me at least this is clear evidence that Sophia may be a replacement for Adam even in large scale runs. https://t.co/1l8XKBswaU
Anthropic does not offer its own embedding model. One embeddings provider that has a wide variety of options and capabilities encompassing all four of the above considerations is Voyage AI. Voyage AI makes state of the art embedding models and offers customized models for…
Anthropic does not offer its own embedding model. One embeddings provider that has a wide variety of options and capabilities encompassing all four of the above considerations is Voyage AI. Voyage AI makes state of the art embedding models and offers customized models for…
⛵ @Voyage_AI_ Embedding Integration Package ↗️ Use the same custom embeddings that power Chat LangChain via the new langchain-voyageai package! Recommended by @AnthropicAI as their preferred embedding provider, Voyage AI builds custom embedding models for your company or…
Thanks for trying our optimizer! hope that Sophia can save some compute for FAIR and others :)
Thanks for trying our optimizer! hope that Sophia can save some compute for FAIR and others :)
In increasing difficulty, 1. train artificial neural nets 2. train one’s own biological neural net 3. train others’ neural nets Level 1.5: train others’ neural nets when others are also willing to train their own— that’s why profs can mentor even though they may fail at 2.
Found a new giant in the Embedding and Reranking 👍 Embedding Models: Model Tokens Dimension voyage-large-2 16000 1536 voyage-code-2 16000 1536 voyage-2 4000 1024…
Found a new giant in the Embedding and Reranking 👍 Embedding Models: Model Tokens Dimension voyage-large-2 16000 1536 voyage-code-2 16000 1536 voyage-2 4000 1024… https://t.co/vmV1pXkiEq
Voyage AI (@Voyage_AI_) is the newest giant in the embedding, reranking, and search model game! 🔥 I am SUPER excited to publish our latest Weaviate podcast with Tengyu Ma (@tengyuma), Co-Founder of Voyage AI and Assistant Professor at Stanford University! 🎙️ We began the…
Very excited to announce @Voyage_AI_'s SOTA reranker!
Very excited to announce @Voyage_AI_'s SOTA reranker!
Last summer we announced the Sophia optimizer, a successor to Adam that can achieve up to 2x gains over Adam. We’ve now merged mainline support into Levanter! Check out @tengyuma’s original thread for how Sophia works: x.com/tengyuma/statu… @HongLiu9903 github.com/stanford-crfm/…
Last summer we announced the Sophia optimizer, a successor to Adam that can achieve up to 2x gains over Adam. We’ve now merged mainline support into Levanter! Check out @tengyuma’s original thread for how Sophia works: x.com/tengyuma/statu… @HongLiu9903 github.com/stanford-crfm/…
Q&A with code and documentation is a strongly demanded yet challenging RAG task - the embedding model must grasp tech terms and code deeply. We evaluated latest models from @Voyage_AI_ and saw exceptional quality on code tasks. Evaluation setup: - Retrieval system: Zilliz Cloud…
Typical feedback for grant proposals, blog posts, theses, etc.: 2019: please polish the language 2024: please do more prompt engineering
Gautam Kamath @thegautamkamath
44K Followers 505 Following Assistant Prof of CS @UWaterloo, Faculty @VectorInst, Canada @CIFAR_News AI Chair. Co-EiC @TmlrOrg. I lead @TheSalonML. Privacy, robustness, machine learning.
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Dan Roy @roydanroy
45K Followers 2K Following ML / AI researcher, emphasis on theory. Research Director and Canada CIFAR AI Chair, @VectorInst Professor, @UofT (Statistics/CS)
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Behnam Neyshabur @bneyshabur
18K Followers 690 Following Senior Staff Research Scientist @GoogleDeepMind, Interested in reasoning w. LLMs, traveling & backpacking
Sergey Levine @svlevine
79K Followers 122 Following Associate Professor at UC Berkeley Co-founder, Physical Intelligence
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Csaba Szepesvari @CsabaSzepesvari
8K Followers 704 Following "If there is not folly in the world, then the world itself is folly. You must understand that mistakes are not always regrets." - Paul Tobin, Bandette🤠
Prof. Anima Anandkuma.. @AnimaAnandkumar
25K Followers 2K Following Bren Professor @caltech, Fmr Sr Director of #AI research @nvidia, Fmr Principal Scientist @awscloud, AI+Science, PDE, Neural operators. Views my own.
Natasha Jaques @natashajaques
25K Followers 1K Following Senior Research Scientist at @GoogleAI and Assistant Professor @uwcse. Social Reinforcement Learning in multi-agent and human-AI interactions. PhD from @MIT.
Zachary Lipton @zacharylipton
59K Followers 2K Following Professor: CMU/@acmi_lab, CTO / CSO: @AbridgeHQ, Creator: @d2l_ai & https://t.co/QQt98VNLUp, Relapsing 🎷
Yi Tay @YiTayML
29K Followers 97 Following chief scientist / cofounder @RekaAILabs 🫠 past: research scientist @google brain 🤯 currently learning to be a dad 🍼
Animesh Garg @animesh_garg
21K Followers 1K Following Foundation Models for Generalizable Autonomy. Assistant Professor in AI Robotics @GeorgiaTech + @NvidiaAI. prev @Stanford @berkeley_ai @UofTCompSci
Yuandong Tian @tydsh
16K Followers 801 Following Research Scientist and Senior Manager in Meta AI (FAIR). AI-guided Optimization and Representation Learning. Novelist in spare time. PhD in @CMU_Robotics.
Eugene Vinitsky @EugeneVinitsky
13K Followers 2K Following Anti-cynic. Artificial narrow intelligence. Autonomous vehicles, multi-agent learning, and transportation. RS at Apple, Asst. Prof at @nyutandon. He/him.
Amin Karbasi @aminkarbasi
8K Followers 2K Following Associate Professor at Yale University, staff research scientist at Google.
Shane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)
Pirattorrent @Pirattorrents
744 Followers 3K Following
ywh @ywhyster
15 Followers 113 Following
sh lin @shlin12647437
18 Followers 82 Following
Wayne Painters 🌻 �.. @PaintersWayne
1K Followers 2K Following Let’s do this! Follow me in the fight against sedition, hypocrisy and anti-democracy in an already great country. Don’t give in to the magat gaslighting! 🌊
Aurick Qiao @AurickQ
253 Followers 199 Following @SnowflakeDB AI Research | @LLM360 | Previously @PetuumInc | PhD @SCSatCMU | CS @UWaterloo
Adrian @Adri_154T
1 Followers 23 Following
Amir Saeidi @sahsaeedi
0 Followers 27 Following CS PhD @ASU | Vision and Language Researcher | Alignment | GenAI
T @JengRoong
5 Followers 245 Following
ssteevens @Steevens43
152 Followers 4K Following
Rakibul @raakibul_
225 Followers 831 Following CS Student || ML, Data Science and NLP || https://t.co/jn48vp1QsW
Florian Huo @Florian36864958
8 Followers 17 Following
Sebastián Uría @SebastinUra1
101 Followers 547 Following
Anusheel Bhushan @sheel_ai
203 Followers 1K Following Engineer, hacker, entrepreneur working on code generators #compilers #LLM #stub
Winston Iskandar @WinstonIsk
82 Followers 267 Following TEDx Speaker | Emergent Ventures | Concert Pianist
Jack Reacher @JackReach516
76 Followers 985 Following
Billy Schwartz @billy_schwartzR
0 Followers 433 Following
Carl Grafe @CarlGrafe
930 Followers 1K Following Data analyst / consultant / problem solver @byuidaho | informatics PhD | epidemiology MS | sims | machine learning | math.
Harsh Pathak @HARSH_306
76 Followers 1K Following PhD Student | Data Scientist at Expedia | Deep and Machine Learning | Optimization| Bifurcations
Sahil @iSahilSingh
135 Followers 454 Following Prime NPC at @xpanse_gg // Making video games immersive with intelligent NPCs
Vidhya @whaats_that
3 Followers 61 Following
Trunkboy PeeZ @P10895Peez
12 Followers 204 Following
Swarup Dwivedy @swarup5662
9 Followers 43 Following
JC Zhu @JCZhu143293
1 Followers 86 Following
Charles Vaske @CharlesVaske
762 Followers 938 Following I work on genomics, but love all of biology and any means to investigate it with math, probability, and computation. He/him/his/they/their.
Aynaz @aynazjavani
0 Followers 239 Following
Vitor Zucher | ויט.. @vmzucher
269 Followers 786 Following 2x Founder (1x Bootstrapped, 1x Seed $5M) - Acquired by IC 23' I do sales, marketing, code, data, product & growth. Zionist. Tech-Optimist. e/acc.
paligonshik @Paligonshik
13 Followers 65 Following
Sang Bin Moon @SangBinMoon1
0 Followers 10 Following
Mira Kwak (Irma Snow) @aleph0
1K Followers 5K Following Head, Art Chaosmos/ Phaidalos, Mystral, Mystral Andel/ mind, transhumanism, cybernetics, semiotics, truth, freedom, metaverse/ Velvet Goldmine, Remy Martin
Lucas Antonio II @LucasAntonioII
111 Followers 674 Following
Grig Vardanyan @grig_vardanyan
0 Followers 3 Following
Jabulani Chibaya @Jabulanichibaya
1K Followers 5K Following Snr. Software Engineer I Apache Spark, Pulsar & Kafka I #DataScience I Big Data Engineer I Emerging Technologies Consultant I DataOps I #BI I #OSS I @mises
Sunny Sun @JialiangLiu666
0 Followers 43 Following
trang pham @pnghtrang
7 Followers 644 Following
weichao tian @wecot23
8 Followers 106 Following
Yi Ma @YiMaTweets
71K Followers 123 Following Chair Professor in AI, Director of IDS, Head of CS, HKU; Professor of EECS, Berkeley; Author of Book: High-Dim Data Analysis, https://t.co/gwaqMJp8av.
Clément Canonne @ccanonne_
31K Followers 928 Following Senior Lecturer @Sydney_Uni. Postdocs @IBMResearch, @Stanford; PhD @Columbia. Converts ☕ into puns: sometimes theorems. He/him. @[email protected]
Gautam Kamath @thegautamkamath
44K Followers 505 Following Assistant Prof of CS @UWaterloo, Faculty @VectorInst, Canada @CIFAR_News AI Chair. Co-EiC @TmlrOrg. I lead @TheSalonML. Privacy, robustness, machine learning.
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Dan Roy @roydanroy
45K Followers 2K Following ML / AI researcher, emphasis on theory. Research Director and Canada CIFAR AI Chair, @VectorInst Professor, @UofT (Statistics/CS)
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Ben Recht @beenwrekt
26K Followers 365 Following optimization. machine learning. uc berkeley. I blog at https://t.co/fkJujOPsJb The world won't end.
Behnam Neyshabur @bneyshabur
18K Followers 690 Following Senior Staff Research Scientist @GoogleDeepMind, Interested in reasoning w. LLMs, traveling & backpacking
Sergey Levine @svlevine
79K Followers 122 Following Associate Professor at UC Berkeley Co-founder, Physical Intelligence
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Csaba Szepesvari @CsabaSzepesvari
8K Followers 704 Following "If there is not folly in the world, then the world itself is folly. You must understand that mistakes are not always regrets." - Paul Tobin, Bandette🤠
Boaz Barak @boazbaraktcs
17K Followers 419 Following Computer Scientist. See also https://t.co/EXWR5k634w, https://t.co/SEVX6it6z3 ( @[email protected] , boaz.barak in threads ). Opinions my own.
Natasha Jaques @natashajaques
25K Followers 1K Following Senior Research Scientist at @GoogleAI and Assistant Professor @uwcse. Social Reinforcement Learning in multi-agent and human-AI interactions. PhD from @MIT.
Zachary Lipton @zacharylipton
59K Followers 2K Following Professor: CMU/@acmi_lab, CTO / CSO: @AbridgeHQ, Creator: @d2l_ai & https://t.co/QQt98VNLUp, Relapsing 🎷
Yi Tay @YiTayML
29K Followers 97 Following chief scientist / cofounder @RekaAILabs 🫠 past: research scientist @google brain 🤯 currently learning to be a dad 🍼
Amin Karbasi @aminkarbasi
8K Followers 2K Following Associate Professor at Yale University, staff research scientist at Google.
Shane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)
Dimitris Papailiopoul.. @DimitrisPapail
11K Followers 974 Following prof @ wisconsin; thinking about transformers; learning in context; babas of Inez Lily
Andrew Gao @itsandrewgao
24K Followers 2K Following techno optimist! currently: @nomic_ai @stanford; prev @LangChainAI; Z Fellow 🇺🇸
Daniel Han @danielhanchen
7K Followers 934 Following Building @UnslothAI. Finetune LLMs 30x faster https://t.co/aRyAAgKOR7. Prev ML at NVIDIA. Hyperlearn used by NASA. I like maths, making code go fast
Jason Hu @onjas_buidl
729 Followers 572 Following founding eng @withmartian | https://t.co/rFBheEZsa9
Nathan Labenz @labenz
14K Followers 2K Following AI Scout, building text-2-video @Waymark, host of The Cognitive Revolution podcast
Yujie Qian @Yujie_Qian
262 Followers 193 Following Founding Research Scientist @ Voyage AI; PhD @ MIT NLP Group
Jay Hack @mathemagic1an
37K Followers 3K Following Founder/CEO @codegen. Tweets about AI, computing, and their impacts on society. Previously did startups, @palantir, @stanford. Not a pseudonym.
Armen Aghajanyan @ArmenAgha
6K Followers 263 Following Research Scientist @ Meta AI (FAIR) https://t.co/8XF2vtiIVy Opinions are my own.
Alfred Lin @Alfred_Lin
47K Followers 286 Following Partner @sequoia. Working w/ founders from idea to IPO & beyond: @airbnb @diaandco @dollskill @doordash @foundforbiz @houzz @kalshi @truework @zipline
Brian Sam-Bodden @bsbodden
1K Followers 1K Following Senior Applied AI Engineer at Redis. @Java_Champions Data Science @Harvard 🟧🟦
Benjamin Clavié @bclavie
2K Followers 732 Following regressing linearly on a daily basis @answerdotai | cooking some late interaction RAGatouille | 日本語NLPを通じて日本語を学んでいます。
meng shao @shao__meng
2K Followers 1K Following Developer | Exploring Gen AI 👨💻 Passionate about LLM and T2I 🧠 Share images generated by 👇🏻 Freepik, Ideogram, Stylar and others
Kasper Green Larsen @kasperglarsen
1K Followers 234 Following Professor and Head of Algorithms, Data Structures and Foundations of Machine Learning at Computer Science, Aarhus University
Yangqing Jia @jiayq
12K Followers 263 Following Founder @leptonai. @UCBerkeley alumni. ex @google & @facebook. ex vp @AlibabaGroup. Open source work on caffe, @pytorch, @tensorflow, & @onnxai.
Amr Awadallah 🤖 @awadallah
36K Followers 14K Following Founder/CEO of @Vectara (Trusted GenAI for the Enterprise). Founder/ex-CTO @Cloudera, ex-VP at Yahoo & Google. PhD EE Stanford. IG, FB, LI: @awadallah.
Jiang Chen @jiangc1010
74 Followers 92 Following Head of AI Platform & Ecosystem @ Zilliz; Prev: PM & TL @ Google Search Indexing
Dylan Patel @dylan522p
39K Followers 683 Following SemiAnalysis Boutique AI & Semiconductor Research and Consulting DMs are open for consulting, quotes, or to talk shop
Connor Shorten @CShorten30
16K Followers 15K Following Research Scientist @weaviate_io! Mostly working on Generative Feedback Loops with DSPy and Filtered ANN. Host of the Weaviate podcast! DSPy playlist below!
Rohan Taori @rtaori13
2K Followers 1K Following phd student @StanfordAILab🌲| proud @Cal alum 🐻 | prev taught w @BerkeleyML
Epsilla, Inc @epsilla_inc
81 Followers 26 Following Ship Production-Ready RAG on Day 1 Discord: https://t.co/x9hLYR23P8 LinkedIn: https://t.co/fdP1xNvDAQ
Maithra Raghu @maithra_raghu
17K Followers 476 Following Cofounder and CEO @Samaya_AI. Formerly Research Scientist Google Brain (@GoogleAI), PhD in ML @Cornell.
Omar Khattab @lateinteraction
11K Followers 2K Following CS PhD candidate @StanfordNLP. 2022 Apple Scholar in AI/ML. Author of ColBERT (https://t.co/2ZtgXoa1np), DSPy (https://t.co/BH7WmMKDXR), & various retrieval & LM systems.
Anthropic @AnthropicAI
262K Followers 26 Following We're an AI safety and research company that builds reliable, interpretable, and steerable AI systems. Talk to our AI assistant Claude at https://t.co/aRbQ97uk4d.
Elliott Robinson @TheValuesVC
20K Followers 101 Following Partner @BessemerVP Growth 👨🏽💻 Values 1st @render @trydatabook @hyperscienceai @hingehealth @forterglobal @implydata @canva @netlify @aimlab @coactiveAI
Chip Huyen @chipro
92K Followers 443 Following Data processing on GPUs @VoltronData Designing ML Systems: https://t.co/G81hL2dWmr @designmlsys #AI x #GPU
Hugging Face @huggingface
343K Followers 189 Following The AI community building the future. https://t.co/VkRPD0VKaZ #BlackLivesMatter #stopasianhate
Lenny Rachitsky @lennysan
206K Followers 2K Following Newsletter: https://t.co/LF0tRFpCeT 💥 Podcast: https://t.co/ZQWNT0iXvJ 💥 Jobs: https://t.co/h5qbpegVYd 💥 Lennybot: https://t.co/TqkCh3hVCa 💥 Swag and Book: https://t.co/qgtUH5DM4S
Teknium (e/λ) @Teknium1
29K Followers 3K Following Cofounder @NousResearch, prev @StabilityAI Github: https://t.co/LZwHTUFwPq HuggingFace: https://t.co/sN2FFU8PVE Support me on Github Sponsors
Founders You Should K.. @foundersysk
454 Followers 15 Following We host a monthly IRL startup showcase in SF to help the best engineers connect with breakout startups
Waseem @waseemhnyc
865 Followers 517 Following I post about projects I'm working on and what I've learned • Engineer • AI, LLMs & Robotics • BJJ @digitsu_bjj • Ask a self defense, mma or jiu jitsu question👇
Parker Rex ∆ @ParkerRex
2K Followers 455 Following Making life goals easy @Map_Coach. ex: https://t.co/4uRxbAXtHf, used by . Cofounder Venu marketplace. founding team @deliverydudes acq '21.
Joshua Achiam ⚗️ @jachiam0
14K Followers 947 Following Human. Trying to make safe alchemy machines. Thinking about humanist alchemism (h/alc ⚗️, maybe). Main author of https://t.co/cKuSh210l1
Tarik Remila @tarikremila
1K Followers 2K Following Co-founder of a #realestate company in #France. #startup Investor @tryduplo @Kiln_finance @sosimplified LP @Soma_Capital https://t.co/XBWAtlGA0s
Alex Graveley @alexgraveley
31K Followers 931 Following I’m Alex Graveley, creator of GitHub Copilot, AI Tinkerers, Dropbox Paper, MobileCoin, and Hackpad. Building @ai_minion Hiring https://t.co/nsHar8OLPC
Richard Ngo @RichardMCNgo
35K Followers 1K Following What would we need to understand in order to design an amazing future? Figuring that out @openai
Geoffrey Irving @geoffreyirving
8K Followers 258 Following Research Director at the UK AI Safety Institute (AISI). Previously DeepMind, OpenAI, Google Brain, etc. @[email protected]
Ann Bordetsky @annbordetsky
10K Followers 2K Following Partner @NEA, early stage AI, Consumer, SaaS | BoD @perplexity_AI @contra | I like building with founders 🤖🛠️ | views expressed here are my own
Michael Lampē @Michae1_Lampe
728 Followers 373 Following listen to my playlist https://t.co/iD3hJFILtj
Mason Meyer @masonmeyer_
234 Followers 125 Following research @openai. but ever with the eternal goal of the true, the beautiful, and the good.
Correction: I did the math wrong (not considering log/log scales). Sophia is ~1.6x times more efficient than Adam (thanks for pointing out @tengyuma).
Putting together all the experiments, scaling looks very healthy. We're slightly more than 1.2x more efficient with Sophia vs. AdamW at scale. Doesn't get close to 2x the original paper stated but also original paper used a lot less compute. Seems like free lunch!
Someone just dropped a dataset of 15 trillion tokens (as many as were used to train Llama 3)!!! Download this now before it gets taken down for “copyright reasons” Breakdown in thread 🧵 👇👇
Massive Text Embeddings Benchmark 海量文本嵌入基准评测，共收取 269个模型。为大家选择 RAG 场景的嵌入模型作参考。 我们看到前十名中，@Voyage_AI_ @tengyuma 以很小的参数量和内存占用，排到了第三名。 gte 模型是前十名中参数量最小的。 Mted HF: huggingface.co/mteb
Fascinating development suggesting a new "algorithm overhang" in AI @tengyuma – a Stanford professor – published the Sophia optimizer in May, 2023, showing 50% compute savings! Yet, with research going exponential, it's taken almost a year for folks to validate it at scale! 🤔
Final Update: One more magnitude of testing Sophia. We're talking model sizes in the B's, tokens in the T's. Sophia once again wins out. For me at least this is clear evidence that Sophia may be a replacement for Adam even in large scale runs.
Theorist and builder very very rare
🆕📢 @Voyage_AI_'s new embedding model for legal and long-context retrieval and RAG: voyage-law-2! 1.🥇 # 1 on MTEB legal retrieval benchmark with a large margin 2.📜 Best quality for long-context (16K) 3.✨ Improved quality across domains 4.🛒 On AWS Marketplace #RAG #LLMs
Way to go @HongLiu9903, @zhiyuanli_, @dlwh, @percyliang & @tengyuma !
Update: As promised, one order of magnitude more compute testing AdamW vs. Sophia. This time applied to two different transformer architectures. Sophia is clearly the winner again. Will run one more ablation with another order of magnitude more compute to see if trend holds.
@ArmenAgha @tengyuma You’ve given me the motivation to go try and tune Sophia’s lr, so I’ll report back after that 🫡
@ArmenAgha Cudos for doing the hard work! I’m sure @tengyuma will find this interesting. Sophia hasn’t worked on my tasks but clearly you’re doing something right.
There's a new exciting reranking API from @Voyage_AI_! It's already supported in `rerankers` v0.1.2, try it out in your pipelines! `pip install --upgrade rerankers`
Rerankers refine the retrieval in RAG. 🆕📢 Excited to announce our first reranker, rerank-lite-1: state-of-the-art in retrieval accuracy on 27 datasets across domains (law, finance, tech, long docs, etc.), enhancing various search methods, vector-based or lexical. 🧵
@tengyuma @Voyage_AI_ Congrats on the launch. @lyzrai agent framework now has an additional reranker option.
pseudo-hessian. nice.
Sophia pre-conditions the gradient with a lightweight estimate of the diagonal Hessian, followed by an element-wise clipping (pseudo-code in first figure), and is easily implementable with the PyTorch code below.
Looking forward to giving the Voyage AI embeddings a spin on wandbot once we get access :)
OpenAI’s embedding v3 were out 🎉! Curious about its quality? We tested on 11 code retrieval datasets & 9 industry-domain datasets: 1. @OpenAI v3 > ada-002 & cohere (except v3-small on code) 2. voyage-code-2 is the best with + 14% margin on code & + 3% on industry docs 🚀
@tengyuma @Voyage_AI_ @OpenAI This is so impressive! 17% gain is amazing.
@tengyuma @Voyage_AI_ @OpenAI Congrats to @Voyage_AI_ on the launch of their new embedding models! The increased accuracy, top performance, production-ready latency, and availability on AWS Marketplace make them a great choice. Excited to see how these models perform in real-world applications!
@tengyuma @Voyage_AI_ @OpenAI This is a big deal congrats!
@tengyuma @Voyage_AI_ @OpenAI Congrats Ma, don't forget list your product on my directory at affordhunt dot com
@tengyuma @Voyage_AI_ @OpenAI Super exciting!! Congrats Tengyu! 🔥🔥🔥
@tengyuma A neutral followup like that is my default method with GPT-4. Arguing with it amounts to trying to bias it in one direction or another.
Trends for United States
You might like
