sophiaalthammer @sophiaalthammer
Member of Technical Staff in Retrieval-Augmented Generation Team @cohere, previously PhD in neural Information Retrieval @tu_wien sophiaalthammer.github.io Munich, Germany Joined March 2019-
Tweets216
-
Followers941
-
Following595
-
Likes2K
LLMs-as-Juries? A better way to automatically evaluate LLMs? 👨⚖️ LLM-as-a-judge refers to LLMs to evaluate the performance or quality of other LLMs. 🤔 @cohere released a new paper exploring the results of replacing a single LLM “as Judge” with multiple LLMs “Juries” where they…
New paper from our team, led by @pat_verga Are you: * Doing evaluation with LLMs? * Using a huge model? * Worried about self-recognition? Try an ensemble of smaller LLMs. Use a PoLL: less biased, faster, 7x cheaper. Works great on QA & Arena-hard evals arxiv.org/abs/2404.18796
Replacing Judges with Juries Evaluating LLM Generations with a Panel of Diverse Models As Large Language Models (LLMs) have become more advanced, they have outpaced our abilities to accurately evaluate their quality. Not only is finding data to adequately probe
@_lewtun @lmsysorg @PSH_Lewis @sophiaalthammer @olapiktus and others published a paper on arXiv today that advocates for using an ensemble of judges (panel of LLMs; PoLL). Their evaluation includes an ablation comparing different prompt variants. arxiv.org/abs/2404.18796
Felt cute, might delete later github.com/cohere-ai/cohe…
👀
Automate your enterprise workflows with Cohere's multi-step tool use. Our generative model Command R+ excels at leveraging external tools to execute complex tasks to streamline business operations. Get started today! txt.cohere.com/multi-step-too…
We've enabled multi-step tooluse with Command R+ in our Chat API! With just one API call, Command R+ can explore the world with your tools, deep diving across application boundaries to answer your prompts. Developers are able to review its reasoning through its planning and…
We've enabled multi-step tooluse with Command R+ in our Chat API! With just one API call, Command R+ can explore the world with your tools, deep diving across application boundaries to answer your prompts. Developers are able to review its reasoning through its planning and…
Excited to announce the Compass Beta, a very powerful multi-aspect data search system powered by a new embedding model, Compass. We're looking for help stress-testing the model's capabilities and finding where it breaks. Sign up here: txt.cohere.com/compass-beta/
I ❤️ cross-encoders! Awesome to see another one from Cohere A quick test showing it turbocharged when using Graph-based Adaptive Reranking :)
I ❤️ cross-encoders! Awesome to see another one from Cohere A quick test showing it turbocharged when using Graph-based Adaptive Reranking :) https://t.co/Nd6E8cZ1tC
Glad a startup present in Paris 🇫🇷 and Europe 🇪🇺 delivered a model better than GPT4. Plus it's open weights 🫡 Hoping there will be more. cc @EmmanuelMacron
🏟Command R+ is 6th in the arena leaderboard, as the first open-weights model to surpass earlier versions of GPT-4 🤔No RAG in the arena yet! Download at huggingface.co/CohereForAI/c4… or try via @cohere's API with the @cohereForAI Research Grant Program txt.cohere.com/c4ai-research-…
Cohere's Chat interface with new ⌘R+ model looks... really good! 📄 search + cite 💻 web browsing 🐍 python Watch this
mind-blow by how good ⌘ R+ multi-step tool use is 🤯 rewrites my sloppy query -> fetches numbers -> plots them with citations
Adaptive RAG w/ Cohere's new Command-R+ Adaptive-RAG (@SoyeongJeong97 et al) is a recent paper that combines (1) query analysis and (2) iterative answer construction to seamlessly handle queries of differing complexity. We took at stab at implementing these ideas from scratch…
Re. LLM tool-use, ToolTalk (Hard) really requires the model to BOTH handle tool-use/chat history well 📝 AND to issue tool-calls either in parallel 🤹 or in a multi-hop "agentic" fashion 🐰 We are very excited to observe CMD R+ rising up to the challenge and coming out on top 🎉
Re. LLM tool-use, ToolTalk (Hard) really requires the model to BOTH handle tool-use/chat history well 📝 AND to issue tool-calls either in parallel 🤹 or in a multi-hop "agentic" fashion 🐰 We are very excited to observe CMD R+ rising up to the challenge and coming out on top 🎉
This just in: I've been promoted to Associate Professor (with tenure) at TU Wien @tu_wien Thank you to all the people in my lab (@schrototo, @andreashappe, Markus Böck, Nathanael Nußbaumer), all my students, and collaborators. To many more years of cool research!
Jo Kristian Bergum @jobergum
9K Followers 814 Following Distinguished Engineer @vespaengine. Tweets about Vespa, search, recommendation, ranking, and IR. CET. #StandWithUkraine 💙💛
Shubham Chatterjee | .. @ShubhamC526
1K Followers 2K Following Research Associate | University of Edinburgh , Scotland | Neural IR | Representation Learning | Conversational IR | Tweets are my own opinion
Guido Zuccon @guidozuc
1K Followers 703 Following Professor at The University of Queensland, leader of @IELabGroup (https://t.co/yLTRjRQAWA), Information Retrieval researcher
Sean MacAvaney @macavaney
1K Followers 480 Following he/him · Lecturer (Assistant Professor) at @GlasgowCS @TerrierTeam · working at the intersection of IR&NLP · PhD from @Georgetown IRLab Website: https://t.co/TvZBNq61Ey
Debasis Ganguly @debforit
655 Followers 375 Following Lecturer/Asst. Professor at the School of Computing, University of Glasgow (@UofGlasgow/@GlasgowCS/@IDAglasgow/@ir_glasgow)
Hitarth Narvala @hitarth_08
376 Followers 263 Following PhD @ir_glasgow @GlasgowCS, University of Glasgow
Leo Boytsov @srchvrs
7K Followers 2K Following Sr. Research Scientist @AWS Labs (ph-D @LTIatCMU) working on unnatural language processing, speaking πtorch & C++. Opinions sampled from MY OWN 100T param LM.
Arian Askari @arian_ask
463 Followers 779 Following Final year PhD Candidate @UniLeiden | VR @Irlab_amsterdam | Interested in IR, NLP, Large Language Models
Webis Group @webis_de
709 Followers 402 Following Research group working the fields of Information Retrieval, Natural Language Processing, Data Mining, Machine Learning, and Artificial Intelligence.
Ayah Soufan 🇵🇸 @AyahSoufan
1K Followers 1K Following Postdoctoral Researcher in AI| PhD in Interactive Information Retrieval| Wanderer and a lifelong learner! 🇵🇸 🏴 !ما نعرفه قطرة وما لا نعرفه محيط
Ivan Sekulić @puding_p
426 Followers 375 Following PhD from @USI_INF. Conversational Search, IR, NLP. 🐧
Rodrigo Nogueira @rodrigfnogueira
2K Followers 304 Following Researcher in Deep Learning, Information Retrieval, and NLP
Arthur Câmara @ArthurCamara
1K Followers 847 Following Applied IR Research @ZetaVector | PhD @tudelft | ex @naverlabseurope, @bloomberg | Dad that likes games. Not enough dopamine nor insulin | CNF✈️AMS (he/him)
Nandan Thakur @nandan__thakur
2K Followers 2K Following PhD @uwaterloo | Author of BEIR, Upcoming: TREC-RAG | Prev: @GoogleAI, @UKPLab | Undergrad @BitsPilaniGoa | Interested in IR and NLP 🇨🇦🇩🇪🇮🇳
Siwei Liu @TedSiwei
271 Followers 292 Following Lecturer/AP of the School of Natural & Computing Sciences at University of Aberdeen.
Yashar Deldjoo @yashardel
764 Followers 1K Following Tenure Track (Rtd-b), Asst. Professor @polibaofficial; #RecSys #GenerativeAI #FairML #TrustworthyML #Multimedia #Fashion
Sebastian Schuster @sebschu
2K Followers 2K Following Lecturer @LinguisticsUCL, and starting in 2025, Assistant Professor @univienna. #nlproc, computational and experimental semantics and pragmatics. he/him.
liuyong @forrestbing
272 Followers 5K Following I am a researcher in AIGC, Multi-modality and VitrualHuman tech direction
chuanming yu @CastilY62290
0 Followers 34 Following
GPT Maestro @GptMaestro
63 Followers 400 Following curator of the LLMpedia (Illustrated Large Language Model Encyclopedia)
Mimansa Jaiswal @MimansaJ
1K Followers 3K Following MoTS @normativeai. Ex @UMichCSE, 2x @MetaAI, @allen_ai | Speech & NLP | Robustness, Data & Annotations, Evaluation & Interpretability in LLMs
Vaibhav Adlakha @vaibhav_adlakha
664 Followers 972 Following PhD candidate @MILAMontreal and @mcgillu | RA @iitdelhi | Maths & CS undegrad from @IITGuwahati Interested in #NLProc
Alberto Castelo @acaste10
239 Followers 1K Following Applied Machine Learning @Shopify. Previously, @Nextail_co and @NCState.
Aman Karmani @tmm1
6K Followers 3K Following full stack tinkerer and perf nerd. formerly vp of infra @github and ruby-core committer. now @getchannels working on ffmpeg/go. dabbling in machine learning.
Abhay Puri @AbhayPuri98
577 Followers 923 Following Visiting Researcher @ServiceNowRSRCH| ex-MLE @Jumio | Grad Student @Mila_Quebec
Balacoumarane @vbala2223
0 Followers 1K Following
CollaborativeDynamics.. @CoDynamicsAI
22 Followers 853 Following Boost all aspects of your business with our bespoke B2B AI solutions in prompt engineering, personas and automation. #AI #Automation #GenerativeAI🚀
Yifei Hu @hu_yifei
317 Followers 380 Following Ph.D. Candidate @LifeAtPurdue | NLP | LLM | UX | Programmer On job market for any AI related industry/academia roles
Blaze (Balázs Galamb.. @gblazex
1K Followers 980 Following A Smooth Guy; Developer of SmoothScroll for macOS, Windows & Google Chrome.
Tahmid Rahman @tahmedge
127 Followers 380 Following Applied Scientist (NLP & ML) @ Dialpad | MSc in CS from YorkU
Freddie Vargus @freddie_v4
619 Followers 1K Following CTO & Co-founder @quotientai — Aya Multilingual @cohereforai — past: @github Copilot, @quantopian — Tico 🇨🇷🇺🇸
Ricardo Valencia Albo.. @rvalenciaaz
307 Followers 2K Following Towards an automated biochemist. PhD student @SBSatED @EdinburghUni. @soreearskid and @doyarzunrod Labs. Funding @DarwinTrustOfEd. Choir conductor (in training)
Elachqar Oussama @Oussama_e
60 Followers 2K Following
Lucio La Cava - @luci.. @luciolcw
165 Followers 738 Following Ph.D. in ICT @ University of Calabria 🇮🇹 Prev. Visiting @ IT University of Copenhagen 🇩🇰 🤖 Multimodal Representation Learning & Networks
Christopher Klamm @ck.. @chklamm
748 Followers 1K Following 👨🔬 CompPolSci @dwsunima & visiting @gesis_org 🚀 https://t.co/srZLDZr4kU co-orga 🎓 MSc CS & MA PolSci @TUDarmstadt 👣 prev. @CompPolCologne, @UKPlab & @COSS_eth
Abhinav Gupta @backpropper
793 Followers 5K Following phd student @Mila_Quebec | ms @CILVRatNYU @NYU_Courant | previously @GoogleDeepMind @AIatMeta @GoogleAI @labsdotgoogle @MSFTResearch @AdobeResearch
Florent Daudens @fdaudens
11K Followers 6K Following Press Lead @HuggingFace / Passionate about AI & news / Previously @radiocanadainfo @ledevoir & co
Nick Frosst @nickfrosst
13K Followers 847 Following cofounder @cohere - singer @goodkidband pfp: @polarfishh_26
Mohammed Hamdy @mhamdy_res
83 Followers 3K Following A curious explorer of human and machine learning 🧐🤝🤖
Tom Hosking @tomhosking
784 Followers 610 Following PhD student in NLP @EdinburghNLP @Edin_CDT_NLP. Ex @cohere @BloomsburyAI @UCL @DRWTrading
Amina Abdullahi @amilah_dul
233 Followers 394 Following CS PhD student @BrownCSDept | Biomedical AI | IR | NLP.
Phillip Lindsay @EastLAPinche
61 Followers 421 Following
ElsaWhittier @427s8cb6vI42Gku
3 Followers 197 Following
Haris Riaz @Haaris_Riaz
17 Followers 127 Following PhD-ing @uarizona @LabCLU. Working on making LLMs more data efficient with applications in reasoning, IR etc.
Chuanming @ChuanmingLiu
230 Followers 4K Following Ex-PhD student and alumni @sjtu1896 . Global citizen. Bootstrapping silicon-based life.
Daniel San @dani_avila7
5K Followers 1K Following Building artificial intelligence tools 🤖 https://t.co/PxOrsWzI55
Elizabeth Orji @Lizzy_Orji
1K Followers 3K Following Google Women Tech Maker Ambassador. Software developer. Founder of Techiepistle. Feminist. Dental Therapist
Aaditya ; @Aaditya26082004
535 Followers 7K Following CS'26 • Machine Learning • Open-Source • Web Dev. • Algorithms • Jai Shree Krishna 🦚🪈
Jon Ander Campos @jaa_campos
238 Followers 313 Following Member of Technical Staff @cohere. PhD in Natural Language Processing. Previously @IxaGroup, @Apple, @AIatMeta, @CNRS and @nyuniversity.
Marco Del Tongo @marcodeltongo
593 Followers 4K Following ⠠⠵ • HCI Technologist • CEO @ Audiencerate
Sara Hooker @sarahookr
39K Followers 7K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.
Wei Huang @sphinxhuang
7 Followers 771 Following Founder&CTO@AnyLink IoT, https://t.co/WhKqXzF8Bz, blockchain enthusiast
Aman Bhandula @AmanBhandula2
1K Followers 3K Following Founder & CEO, @FarmakoIn (YC S20) | Quick medicine delivery in India | IIT Roorkee
Irem Ergün @irombie
1K Followers 456 Following •ML Engineer @cohere developing LLMs🫡 • Previously: @UCR_CSE & @BilkentUniv • yoga & writing & music 🌈🦄 •Tweets in 🇹🇷🇬🇧🎸 •Blogs @ 2cute2tech 👩💻👇🏻
Jo Kristian Bergum @jobergum
9K Followers 814 Following Distinguished Engineer @vespaengine. Tweets about Vespa, search, recommendation, ranking, and IR. CET. #StandWithUkraine 💙💛
Shubham Chatterjee | .. @ShubhamC526
1K Followers 2K Following Research Associate | University of Edinburgh , Scotland | Neural IR | Representation Learning | Conversational IR | Tweets are my own opinion
Guido Zuccon @guidozuc
1K Followers 703 Following Professor at The University of Queensland, leader of @IELabGroup (https://t.co/yLTRjRQAWA), Information Retrieval researcher
Nicola Ferro @frrncl
1K Followers 209 Following Information Retrieval, Digital Libraries, Evaluation - Full Professor in Computer Science at @UniPadova, leading @iiia_unipd #RussiaUkraineConflict #againstwar
Jimmy Lin @lintool
13K Followers 842 Following I profess CS-ly at the @UWaterloo and gaze into the technological crystal ball at @Primal. I used to write code for @Twitter and slides for @Cloudera.
Fernando Diaz @841io
5K Followers 1K Following Associate Professor, CMU. Researcher, Google. Evaluation and design of information retrieval and recommendation systems, including their societal impacts.
Nils Reimers @Nils_Reimers
10K Followers 434 Following Director of Machine Learning @Cohere | ex-huggingface | Creator of SBERT (https://t.co/MKKOMfuQ4C)
Sean MacAvaney @macavaney
1K Followers 480 Following he/him · Lecturer (Assistant Professor) at @GlasgowCS @TerrierTeam · working at the intersection of IR&NLP · PhD from @Georgetown IRLab Website: https://t.co/TvZBNq61Ey
Yann LeCun @ylecun
712K Followers 719 Following Professor at NYU. Chief AI Scientist at Meta. Researcher in AI, Machine Learning, Robotics, etc. ACM Turing Award Laureate.
Debasis Ganguly @debforit
655 Followers 375 Following Lecturer/Asst. Professor at the School of Computing, University of Glasgow (@UofGlasgow/@GlasgowCS/@IDAglasgow/@ir_glasgow)
Hitarth Narvala @hitarth_08
376 Followers 263 Following PhD @ir_glasgow @GlasgowCS, University of Glasgow
Leo Boytsov @srchvrs
7K Followers 2K Following Sr. Research Scientist @AWS Labs (ph-D @LTIatCMU) working on unnatural language processing, speaking πtorch & C++. Opinions sampled from MY OWN 100T param LM.
Glasgow IR Group @ir_glasgow
1K Followers 128 Following Glasgow Information Retrieval Group @GlasgowCS
Arian Askari @arian_ask
463 Followers 779 Following Final year PhD Candidate @UniLeiden | VR @Irlab_amsterdam | Interested in IR, NLP, Large Language Models
Akari Asai @AkariAsai
11K Followers 650 Following Ph.D. student @uwcse & @uwnlp. NLP. IBM Ph.D. fellow (2022-2023). Meta student researcher (2023-) . ☕️ 🐕 🏃♀️🧗♀️🍳
Webis Group @webis_de
709 Followers 402 Following Research group working the fields of Information Retrieval, Natural Language Processing, Data Mining, Machine Learning, and Artificial Intelligence.
Reka @RekaAILabs
11K Followers 13 Following An AI research and product company 🫠. We are a team of scientists and engineers building state-of-the-art multimodal language models 😻
Matthew Carrigan @carrigmat
3K Followers 353 Following @huggingface engineer. I'm the reason your LLM frontend has a jinja2cpp dependency. Sometimes yells about housing and trans rights instead of working He/him
Giannis @giannis2two
193 Followers 112 Following Member of Technical Staff at @cohere, CS + Math @MIT, primarily european but surprisingly american
Yi Chern Tan @yichern_tan
87 Followers 94 Following Command modeling @cohere. Previously @Waymo @Facebook @Yale. 🇸🇬
Philipp Dufter @PDufter
220 Followers 224 Following Machine Learning Engineer @ Apple Before: PhD @CisLmu
Marzieh Fadaee @mziizm
403 Followers 333 Following seeks to understand language. Senior Research Scientist @CohereForAI @Cohere. PhD from @UvA_Amsterdam. [email protected]. Contemplates in private @mzi.
Saurabh Baji @sbaji
900 Followers 2K Following SVP Eng @CohereAI LLMs 🚀 Ex - VP, AI and Data @ Unity, Quantcast, AWS / EMR, Athena. AI, ML, Big Data - always hiring; DM if interested. Tweets are my own.
Jon Ander Campos @jaa_campos
238 Followers 313 Following Member of Technical Staff @cohere. PhD in Natural Language Processing. Previously @IxaGroup, @Apple, @AIatMeta, @CNRS and @nyuniversity.
Maxime Voisin @maximevoisin_ai
747 Followers 669 Following Product manager RAG/Tools/Code @cohere. Previously @labelbox, @stanford computer vision labs
Matthias Gallé @mgalle
2K Followers 1K Following Manager of Technical Staff and Stuff @CohereAI. Born in 🇧🇷, raised in 🇩🇪, studied in 🇦🇷, living in 🇫🇷.
Kelly Marchisio (St. .. @cheeesio
1K Followers 558 Following Multilingual NLP @cohere. Formerly: PhD @jhuclsp Alexa Fellow @amazon dev @Google MPhil @cambridgenlp EdM @hgse 🔑🔑¬🧀 (@kelvenmar20)
Luiza Pozzobon @luizapzbn
400 Followers 271 Following Research Scholar @CohereForAI | MSc @ Unicamp, Brazil
Minjie Xu @chokky_vista
223 Followers 273 Following ML/NLP researcher & practitioner. RAG & tool-use @cohere 🧠 x @tractable_ai @TechAtBloomberg 👨🏻💻 PhD from Tsinghua CS 🎓 In meinem Lieben, in meinem Lied 🎵
Ola Piktus @olapiktus
1K Followers 396 Following
Zhaochun Ren @zhaochun_ren
1K Followers 900 Following Associate Professor @TMLeiden, @LIACS, @UniLeiden, working on Information Retrieval and Natural Language Processing. My tweets are my own.
Ethan Gotlieb Wilcox @weGotlieb
921 Followers 415 Following Postdoc at ETH Zurich. Formerly PhD student at Harvard Linguistics, affiliated with MIT Brain & Cog Sci. Language, Computers, Cognition.
Natalie Schluter @natschluter
5K Followers 487 Following #NoJusticeNoPeace-- Machine Learning Researcher at Apple MLR-- All tweets/opinions my own
Max Bartolo @max_nlp
2K Followers 787 Following I lead the Command modelling team at @Cohere and co-chair the @DynabenchAI @MLCommons working group. Prev @DeepMind, @MetaAI / FAIR & @BloomsburyAI.
Maximilian Mozes @maximilianmozes
201 Followers 483 Following Member of Technical Staff @cohere. PhD @UCL/@ucl_nlp. Previously: @GoogleAI/@SpotifyResearch. He/Him.
Weronika Łajewska @WLajewska
1 Followers 1 Following
Nick Frosst @nickfrosst
13K Followers 847 Following cofounder @cohere - singer @goodkidband pfp: @polarfishh_26
Oren Sultan @oren_sultan
682 Followers 597 Following AI Researcher & Data Scientist @Lightricks, CS PhD Candidate #AI #NLP @HebrewU, advised by @HyadataLab 🇮🇱 | prev. @TU_Muenchen 🇩🇪 @UniMelb 🇦🇺 8200 Unit
Laura Waltersdorfer @LaWaltersdorfer
56 Followers 428 Following Praedoc Student @tu_wien and @semsys_research Research Topic: Auditable Semantic Web Machine Learning Systems
Andrew Drozdov @mrdrozdov
2K Followers 1K Following RAG at @MosaicML x @Databricks 🧱 Prev: @UMass_NLP, @Google, @IBM
Jim Fan @DrJimFan
230K Followers 3K Following @NVIDIA Sr. Research Manager & Lead of Embodied AI (GEAR Lab). Creating foundation models for Humanoid Robots & Gaming. @Stanford Ph.D. @OpenAI's first intern.
Chris de Vries @device_null
608 Followers 1K Following I do things with stuff that go brrr. Here I share news
Marwah Alaofi @Marwah_k
418 Followers 934 Following Information Retrieval PhD candidate @RMIT 🔍🌱| Academic @Taibahu 🤹🏻♀️| @MonashUni alumna 💫 I research to satisfy inquisitive minds
Philipp Schmid @_philschmid
16K Followers 653 Following Tech Lead and LLMs at @huggingface 👨🏻💻 🤗 AWS ML Hero 🦸🏻 | Cloud & ML enthusiast | 📍Nuremberg | 🇩🇪 https://t.co/l1ppq3q3hk
OpenSearchProj @OpenSearchProj
2K Followers 270 Following OpenSearch is a community-driven, open source search and analytics suite. Join the conversations at: https://t.co/8yvpGCe1Fh
Christos Baziotis @cbaziotis
472 Followers 412 Following Machine Learning @Samaya_AI | PhD @InfAtEd. Ex @MetaAI (FAIR) and @AmazonScience.
Haystack @Haystack_AI
804 Followers 34 Following The open-source LLM framework by @deepset_ai Follow for regular feature updates and developer content 🚀 Discord for support: https://t.co/v7iEbzdeT7
Dr. Amanda Swearngin @a_swearngin
654 Followers 334 Following Research and engineering, applying ML to user interfaces at @apple, Ph.D. from @uwcse, Nebraska-born :)
Mistral AI @MistralAI
91K Followers 0 Following Fast, open-source and secure language models. Join us https://t.co/INALdNGvCP
Albert Jiang @AlbertQJiang
2K Followers 409 Following AI4Maths @Cambridge_CL Science @MistralAI I bake my own opinions at temperature=2.0
Munich🥨NLP @MunichNlp
664 Followers 270 Following https://t.co/SXLZqkM9OY @TU_Muenchen x @LMU_Muenchen
Wei Ping @_weiping
783 Followers 220 Following Principal Research Scientist @NVIDIA. Working on large language models and generative models. Views are my own.
Elias Bassani @EliasBassani
43 Followers 109 Following Ph.D. in CS. I like Information Retrieval, Neural Networks, usability, efficiency, einsum, memes, and improperly used emojis. 🫠
pablomendes @pablomendes
924 Followers 1K Following Co-founder & CEO of https://t.co/tHENfmDGq2 | Ex-Apple, Lattice (acq. by Apple), IBM Research, Yahoo. AI, ML, NLP, KG, Q&A. Views are my own.
John Hewitt @johnhewtt
4K Followers 22 Following CS PhD @stanford with @stanfordnlp. Frmr. @penn, intern @deepmind, @googleai, ++. Understanding and improving neural learning from language. Co-teach CS 224n.
Last day of @ProjectDossier ... a bit of heartache, too, looking back on the journey we're now concluding. Great team, and hope to work with you again: @allanhanbury @msalampasis @gabriellapasi Marco Viviani @suzan @arjenpdevries Roberto Cornachia @leifos @martinhalvey ...
... Ian Ruthven, Elaine Toms and all our wonderful 15 PhDs - @arian_ask @uhrishabh @AyahSoufan @kanaadpathak Molly McGregor @ginarsantika Stefanie Segura @daria_alexan @aminvenv @GeorgiosPeikos Oscar Espitia, @WojciechKusa @sophiaalthammer Yasin Ghafourian, Vasilis Stamatis
LLMs as a judge has been widely accepted as a workable replacement of human eval but relying on a single model introduces systematic bias. Happy to share a new paper from our team led by @pat_verga that shows a panel of models as judge offers a more accurate and cheaper solution.
New paper from our team, led by @pat_verga Are you: * Doing evaluation with LLMs? * Using a huge model? * Worried about self-recognition? Try an ensemble of smaller LLMs. Use a PoLL: less biased, faster, 7x cheaper. Works great on QA & Arena-hard evals arxiv.org/abs/2404.18796
LLMs-as-Juries? A better way to automatically evaluate LLMs? 👨⚖️ LLM-as-a-judge refers to LLMs to evaluate the performance or quality of other LLMs. 🤔 @cohere released a new paper exploring the results of replacing a single LLM “as Judge” with multiple LLMs “Juries” where they…
New paper from our team, led by @pat_verga Are you: * Doing evaluation with LLMs? * Using a huge model? * Worried about self-recognition? Try an ensemble of smaller LLMs. Use a PoLL: less biased, faster, 7x cheaper. Works great on QA & Arena-hard evals arxiv.org/abs/2404.18796
Replacing Judges with Juries Evaluating LLM Generations with a Panel of Diverse Models As Large Language Models (LLMs) have become more advanced, they have outpaced our abilities to accurately evaluate their quality. Not only is finding data to adequately probe
@_lewtun @lmsysorg @PSH_Lewis @sophiaalthammer @olapiktus and others published a paper on arXiv today that advocates for using an ensemble of judges (panel of LLMs; PoLL). Their evaluation includes an ablation comparing different prompt variants. arxiv.org/abs/2404.18796
Berkeley Function Calling Leaderboard: Introducing Consistent 8 X V100 with pay-as-you-go pricing for measuring costs and latency. In depth: We fix inconsistency in the cost and latency calculation for open-source models, which are now all calculated when serving the model with…
📊Delighted to welcome Command-R-Plus, Llama-3, and and Gemini-Pro-1.5 into the Berkeley Function Calling Leaderboard. Check out how they stack up across different categories, P95 latency, and costs at gorilla.cs.berkeley.edu/leaderboard.ht… Congratulations to @cohere, @AIatMeta, and…
Ship ship ship ⛴️🇫🇷🇨🇦
we open sourced our chat interface. github.com/cohere-ai/cohe…
Felt cute, might delete later github.com/cohere-ai/cohe…
Excited to be back in Right Wing Portland today (Seattle)
A personal milestone: hit 10,000 citations! Keep going and look forward to more impactful work in the future.
@fkruta What pricing did you find for llama3 api?
@fkruta Yeah should’ve specified the timestamp 😅 llama3 performance is likely going to change in the next few days, will need updating
Llama3 was trained on 15 trillion tokens of public data. But where can you find such datasets and recipes?? Here comes the first release of 🍷Fineweb. A high quality large scale filtered web dataset out-performing all current datasets of its scale. We trained 200+ ablation…
We have just released 🍷 FineWeb: 15 trillion tokens of high quality web data. We filtered and deduplicated all CommonCrawl between 2013 and 2024. Models trained on FineWeb outperform RefinedWeb, C4, DolmaV1.6, The Pile and SlimPajama!
Listening to @arian_ask giving a talk at SEA about our recent works on document generation for information retrieval.
You have experiences with training state-of-the-art neural search models and want to lead a team of excellent engineers to train the next big thing? Looking for a great manager for one of my core search teams: jobs.lever.co/cohere/0feec90…
Performance also != Chatbot Arena Elo. But a massive improvement over previous plots! 🔥 The main takeaway for me here is that we need significantly better evals that reflect the real-world value created by LLMs
Trends for United States
You might like
