Dan Deutsch @_danieldeutsch
Research Scientist at Google Translate working on text generation evaluation danieldeutsch.github.io San Francisco Joined September 2012-
Tweets60
-
Followers267
-
Following78
-
Likes94
New paper alert! Designing reliable human evaluation is both crucial and difficult. Human raters can exhibit different behaviors when rating NLG outputs. These differences are not generally due to a rater performing the task incorrectly, but rather due to differences in…
Exciting News! Introducing our new Quality-Aware Machine Translation model! 🚀 100x faster MBR decoding 🔝 Improved translation quality 🧠 Self-evaluation and guidance for top-notch translations Take a look: arxiv.org/abs/2310.06707
We're looking for a final-year PhD intern passionate about working on automatic metrics for machine translation and NLP in Mountain View. If interested, please send an email to me and @_danieldeutsch. The ideal candidate should have experience working on automatic evaluation and…
Can we not criticize LLM but pinpoint errors it makes and automatically guide it with fine-grained actionable feedback? Can we formulate iterative refinement into a local search problem, simulated annealing? My cool summer intern work @Google @_danieldeutsch @markuseful @ucsbNLP
Our accepted papers and program are now online: eval4nlp.github.io/2023/program.h… eval4nlp.github.io/2023/accepted-… Moreover, we're excited to have @alexfabbri4 as invited speaker on the topic of "Re-Evaluating Summarization Evaluation in the Era of LLMs" See u tomorrow 9am (UTC+8), online only!
Happy to share that our paper “Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translation” has been accepted at @emnlpmeeting. This work further pushes the SOTA in inference methods for machine translation and NLG in…
Happy to share that our paper “Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translation” has been accepted at @emnlpmeeting. This work further pushes the SOTA in inference methods for machine translation and NLG in…
Mara did a fantastic job distilling MBR translations from LLMs into small enc-dec models, further pushing sota of MT, outperforming not only LLMs, but enc-dec models fine-tuned on human labelled data! Help me share this super impactful work of a rising researcher!
Mara did a fantastic job distilling MBR translations from LLMs into small enc-dec models, further pushing sota of MT, outperforming not only LLMs, but enc-dec models fine-tuned on human labelled data! Help me share this super impactful work of a rising researcher!
📢📢To accommodate the recent ARR author response period, Eval4NLP @aaclmeeting extends the deadline for pre-reviewed papers until September 30th. Pre-reviewed papers must include: the paper along with its original reviews and scores. More details: eval4nlp.github.io
It has been shown that MBR decoding significantly outperforms MAP decoding, but its high cost makes it impractical for most applications. Can we harvest the quality gains of MBR at train time w/o sacrificing inference-time efficiency? TLDR: yes! arxiv.org/abs/2309.10966…
Christoph Leiter @ChrLeiter
29 Followers 49 Following
Diego Guerra @diegoguerror
76 Followers 556 Following Consultant @McKinsey by day. Bag packer on the weekends. @Wharton alumnus. AI and VC. 🇲🇽 living the dream 🗽. He/him. Views my own
Dean Carignan @DeanCarignan
923 Followers 1K Following Chief of Staff for @Microsoft's Chief Scientific Officer; exploring responsible practices in AI, Data Science, ML Ops. Ex: @MSFTReseach @Mckinsey, @Worldbank
Mohammad Rifat Arefin @mo_rifat
63 Followers 238 Following CS PhD Student at @utarlington | Software Engineering | Program Analysis
Yixiao Song @yixiao_song
159 Followers 216 Following I am a PhD student at UMass Linguistics @LinguistsUMass and @UMass_NLP working with Prof. Rajesh Bhatt and Prof. Mohit Iyyer. Love hiking and fungi.
Shulin Zhang @shulin_zh
74 Followers 378 Following Ph.D. Candidate at the University of Georgia | Computational Linguistics
Valerie Wendenburg @Vwendenburg
305 Followers 702 Following Redakteurin bei Bajour und dem jüdischen Wochenmagazin tachles
Shruti Singh ⇾ @shr.. @shruti_rsingh
225 Followers 2K Following Representation Learning for Scientific Literature | #NLProc | Fulbright fellow @yale | CS Ph.D. Student @iitgn | Past @daiictofficial
Alexandra DeLucia @Alexir563
547 Followers 742 Following Sony intern Fall 2023. Computer Science PhD Student at @jhuclsp in the @mdredze group. @rollinscollege alum.
Amir Kargaran @amir_nlp
607 Followers 2K Following PhD student in Computer Science #NLProc at University of Munich @CisLmu / Woman, Life, Freedom 🕊
Hizkiel Mitiku @Hizclick
35 Followers 136 Following
Ananya Mukherjee @CuriousAnanya
13 Followers 62 Following Curious Learner. Passionate abt Kuchipudi. Born to Eat & Travel. In an extramarital affair wd Mountains & Forest.
Ali Athar @AliAthar1401
71 Followers 367 Following 🌟 AI PhD student in South Korea | Researching AI, NLP, and healthcare applications 💻 | MS degree from NUST 🎓 | Travel lover.
Mojtaba Vàlipour @ValipourMojtaba
388 Followers 3K Following CS PhD at @UWaterloo, Founding Engineer at Coastal Carbon and part time Researcher at @huawei Noah’s Arc Lab, prev enjoyed my time at @oraclelabs, & @CVC_UAB
Tian Yun @tianyunnn
158 Followers 334 Following Current PhD student at @BrownUniversity, co-advised by Ellie Pavlick @Brown_NLP and Chen Sun @jesu9. NLP & Multimodal Learning & Interpretability.
Pamela Bustamante @pambusf
323 Followers 774 Following PhD(c) @ucatolica 👩🏻🎓| MSc @ubbchile | @pythonchiledev 🇨🇱 | @juliainclusive | @pyladiesSCL 🐍 . #MathOptimization #GameTheory #ML.
Ganesh @buzzganesh
167 Followers 639 Following
Mehar Bhatia @bhatia_mehar
985 Followers 2K Following NLP || Grad CS Student at @UBC Vancouver 👩🎓|| @UBC_NLP @VectorInst || Studying culture, reasoning, alignment, fairness and biases
David Stap @davidstap
288 Followers 727 Following PhD candidate in Artificial Intelligence and Natural Language Processing @UvA_Amsterdam | Previously intern @Amazon | MSc AI from @UvA_Amsterdam
Chunyuan Deng @ChunyuanDeng
88 Followers 164 Following Incoming CS PhD Student @RiceCompSci, MS @GeorgiaTech. Working on #NLProc. 🪙
Dayeon (Zoey) Ki 🍀 @zoeykii
102 Followers 159 Following 📝 Incoming research intern at @AdobeResearch | CS PhD at @umdclip Interested in Aligning LLMs with Multilingual users 🗣️🌐
Elizabeth Salesky @esalesk
1K Followers 657 Following PhD student @jhuclsp more commonly known as Liz ☀️ Friend of @NLPwithFriends ☀️ I like bubbles, bicycles, and language variation
Ricardo Gonzalez @ricardo_agzz
89 Followers 317 Following Building @neutrinoai, prev CS & Wharton @penn, quant @jpmorgan
Isaac R Caswell @iseeaswell
519 Followers 132 Following low resource MT, plants, insects, music+sangeetham
Fei Wang @fwang_nlp
919 Followers 2K Following PhD candidate @USC. PhD Fellow @Amazon. Responsible LLM.
Pedro Martins @PedroHenMartins
75 Followers 587 Following Research Scientist at Unbabel | PhD in Machine learning and NLP | Liberal
Jerry Wu @JerryWu27705751
2 Followers 33 Following
Barbadrafteh @Barbadraft44842
0 Followers 377 Following
Manos Zaranis @ManosZaranis
40 Followers 268 Following
🇰🇷송혜은🇰.. @AshleighHunt137
664 Followers 807 Following 결국 내 마음은 너무 오랫동안 공허했고 그 누구도 내 마음의 공허함을 채워줄 수 없으니까 { Nice to meet you }
Poornima Devi @Poornim14
84 Followers 2K Following Machine Learning Engineer | Deep learning | Natural Language Processing
Ashutosh Mehra @ashutoshmehra
1K Followers 5K Following Senior Principal Scientist at Adobe. Working on Acrobat AI Assistant, LLMs, and document ML.
Vilém Zouhar @zouharvi
2K Followers 2K Following PhD student @ ETH Zürich | all aspects of #NLProc but mostly HCI, evaluation and MT | go #vegan
thebes @voooooogel
4K Followers 525 Following ꙮ programming & LLM & SFF enjoyer @ https://t.co/aykxqKippW ꙮ games @ https://t.co/3Pz19vHOwd ꙮ 💞💍📝 @holotopian ꙮ she/they 🏳️⚧️
Luca Soldaini 🎀 @soldni
6K Followers 1K Following I like tokens! Lead for OLMo data team at @allen_ai (Dolma 🍇), OSS is fun, @QueerInAI organizer 🤖☕️🍕they/them (views mine, not my employer’s)
GodGPT @GodGPT88
252 Followers 3K Following Founder and CEO of GodGPT,AGI,Robot,e/acc,super-LOVE-alignment
nick nassuphis @NNassuphis
120 Followers 5K Following
Robert Scoble @Scobleizer
504K Followers 68K Following Follow me on my new podcast with AI startups, Unaligned. Tech industry color commentator since 1993. Author/Blogger. Former strategist @Microsoft.
Olioli @Oliolilyx
122 Followers 2K Following
Arman Adibi @AdibiArman
490 Followers 2K Following Postdoc @Princeton | Ph.D. from @Penn, @WarrenCntrPenn | Studying machine learning and optimization.
Elizabeth Salesky @esalesk
1K Followers 657 Following PhD student @jhuclsp more commonly known as Liz ☀️ Friend of @NLPwithFriends ☀️ I like bubbles, bicycles, and language variation
Vilém Zouhar @zouharvi
2K Followers 2K Following PhD student @ ETH Zürich | all aspects of #NLProc but mostly HCI, evaluation and MT | go #vegan
Luca Soldaini 🎀 @soldni
6K Followers 1K Following I like tokens! Lead for OLMo data team at @allen_ai (Dolma 🍇), OSS is fun, @QueerInAI organizer 🤖☕️🍕they/them (views mine, not my employer’s)
Shreya Havaldar @shreyahavaldar
232 Followers 268 Following PhD student @cis_penn | multilingual NLP + cultural psychology | prev @usc @microsoft | she/her 🌸
Marine Carpuat @MarineCarpuat
2K Followers 389 Following Associate Professor, Computer Science, University of Maryland. I go by she/her.
Isaac R Caswell @iseeaswell
519 Followers 132 Following low resource MT, plants, insects, music+sangeetham
David Vilar @davvil_dvt
34 Followers 50 Following
Dipanjan Das @dipanjand
4K Followers 308 Following Senior Director of Research at @GoogleDeepmind. Working on improving the factuality of LLM generated content.
Wei Xu @cocoweixu
9K Followers 1K Following CS professor @GeorgiaTech @gtcomputing @ICatGT @mlatgt. Natural language processing, machine learning, social media research.
Manaal Faruqui @manaalfar
3K Followers 646 Following Senior Staff Research Scientist @Google Bard. Love eating, movies, travel and politics. Spread love, not war.
Sam Bowman @sleepinyourhat
35K Followers 3K Following AI alignment + LLMs at NYU & Anthropic. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Jacob Andreas @jacobandreas
14K Followers 958 Following Teaching computers to read. Assoc. prof @MITEECS / @MIT_CSAIL (he/him). https://t.co/5kCnXHjtlY https://t.co/2A3qF5vdJw
Luke Zettlemoyer @LukeZettlemoyer
8K Followers 2K Following
Mohit Iyyer @MohitIyyer
6K Followers 1K Following assoc. prof at @umasscs, member of @UMass_NLP. i work on natural language processing and deep learning
Andre Martins @andre_t_martins
2K Followers 397 Following NLP/ML researcher in Lisbon ([email protected])
Violet Peng @VioletNPeng
5K Followers 398 Following NLP researcher, Assistant Professor @ UCLA-CS. (she/her/hers)
Kai-Wei Chang @kaiwei_chang
6K Followers 711 Following Associate Professor @UCLAengineering/@UCLA. Area: #NLProc/#ML/#AI https://t.co/zj1ssZj9ox
Dan Roth @DanRothNLP
2K Followers 54 Following VP/Distinguished Scientist, AWS AI Labs and the Eduardo D. Glandt Distinguished Professor, CIS, University of Pennsylvania
Jonathan Clark @JonClarkSeattle
3K Followers 2K Following Research Scientist @ Google: Multilingual NLP, Machine Learning, C++. Previously MT@Microsoft and CMU. Opinions are my own.
Salvatore Giorgi @sal_giorgi
356 Followers 893 Following Data at NIDA and @WWBProject, PhD student @Penn, aspiring #nlproc researcher, young #dungeonmaster, failed noiser. Views my own.
Tanya Goyal @tanyaagoyal
1K Followers 339 Following Incoming Asst Professor @Cornell_CS in Fall 2024, Post-doc @princeton_nlp. she/her
Patrick Fernandes @psanfernandes
534 Followers 237 Following PhD Student @LTIatCMU & @istecnico Previously research @Google, @Microsoft & @Unbabel
Viresh Ratnakar (Very.. @vireshratnakar
351 Followers 2K Following Anyone living in an anyhow town. Cryptic crossword setter Gussalufz for #TheHinduCrossword and elsewhere. @[email protected] & @viresh.bsky.social
Joel Tetreault @Tetreault_NLP
1K Followers 422 Following VP of Research at Dataminr co-Program Chair of ACL2020 https://t.co/Bj7uRRRf09
Nitika Mathur @probablyNitika
177 Followers 172 Following
Matt Post @mjpost
2K Followers 2K Following Machine translation research for big tech and big academia and director of the @aclanthology. Tweets here are mostly personal.
Yash Kumar Lal @lal_yash
273 Followers 508 Following PhD candidate at @stonybrooku in @stonybrooknlp; Fall 2023 at Google Research; Summer 2023 @ai2_aristo @allen_ai; Prev: @sfresearch; MS from @jhuclsp
Pierre Colombo @PierreColombo6
448 Followers 1K Following Associate Professor at Université Paris Saclay - CentraleSupelec - NLP - GenAI
Tom Kocmi @KocmiTom
603 Followers 178 Following Senior researcher at Microsoft Translator (he/him) | AI Evaluation (LLMs, MT, Multilingulity)
Wenda Xu @WendaXu2
679 Followers 319 Following PHD candidate at UCSB’s NLP lab coadvised by William Wang and Lei Li
Juan Cervino @juancervinouy
146 Followers 469 Following Ph.D. student at @Penn, became an engineer at @IIE_FIng_UdelaR. A stereotypical Uruguayan, if this provides no information, visit Uruguay.
Thomas Scialom @ThomasScialom
6K Followers 232 Following AGI Researcher @MetaAI -- Lead Llama 2 and Postraining Llama 3. Also CodeLlama, Galactica, Toolformer, Bloom, Nougat, GAIA, ..
Maxime Peyrard @peyrardMax
215 Followers 279 Following Junior Professor @CNRS (previously @EPFL, @TUDarmstadt) -- AI Interpretability, causality, and interaction flows between LLM, humans, and tools
Steffen Eger @egere14
339 Followers 223 Following NLP researcher, hobby mathemagician, also interested in social science. Head of NLLG (https://t.co/Bsf7UVJwB4). Also Heisenberg Grant 2022 recipient of the DFG. they/we.
Haoyu Wang @Haoyu_Wang_97
130 Followers 409 Following PhD student @Penn, intern @GoogleAI. Ex-intern @tiktok_us @allen_ai @TencentGlobal @GoldmanSachs, undergrad @sjtu1896.
Veronica Qing Lyu @veronica3207
726 Followers 341 Following PhD student @upennnlp | NLP, Linguistics, Explainable AI | Intern @tencent, @allenai
Eval4NLP @nlp_evaluation
316 Followers 35 Following Workshop on Evaluation and Comparison of NLP Systems, co-located with #AACL2023.
Leonardo F. R. Ribeir.. @leonardoribeiro
847 Followers 734 Following Applied Scientist at @Amazon, #NLProc PhD @TUDarmstadt - https://t.co/pbIb2RB5Bd
Elizabeth Clark @eaclark07
1K Followers 252 Following Doing NLP research at @GoogleAI. PhD from @uwcse.
Excited to release IndicGenBench: A suite of evaluation datasets for multiple tasks in 29 Indic languages! Evaluations over many LLMs reveal huge room for improvement. IndicGenBench is multi-way parallel opening doors for interesting research! Great work by @Harman26Singh!
New work on evaluating LLMs for generation in Indic Languages: IndicGenBench 👉5 diverse tasks, 29 Indic languages, >100k examples. 👉Curated using human translations ensuring high quality. 👉Multi-way parallel dataset. arxiv.org/abs/2404.16816 github.com/google-researc… (1/n)
📢 Happy to share that our paper "On the Role of Summary Content Units in Text Summarization Evaluation" has been accepted at the #NAACL2024 main conference! Link to the camera-ready version: arxiv.org/abs/2404.01701
It's rare to find people who are passionate about evaluation. We need more research into quality estimation to allow evaluation of fresh unannotated data. Because even blind tests are spoiled once you release source files.
I don't think people have fully internalized just how broken public evaluation of models is.
Last week we spent the week in Washington DC speaking to US Senators, staffers for the house and senate, and a number of government organizations. Overall, I was pleasantly surprised! They wanted to focus on real issues today, not hypothetical future issues. Main points 🧵:
We're proud to have co-hosted this important, landmark discussion on AI & Climate in DC last week with @BezosEarthFund. Thank you to @politico;s @StevenOverly, Andre Perkins, @JesseDodge, @LifeAtPurdue's Bruce Erickson and @MITEECS's @priyald17 for sharing your expertise. And…
Navigating the world of MT metrics is daunting, given that each metric behaves differently. Our new paper aims to clarify what metric gains signify and how trustworthy they are. For instance, we empirically show that a +1 increase in a COMET-22 score has 90% accuracy with humans.
Rare moment of vulnerability here, just wanted to say that if you have any grievances against me at all, reply to this tweet. This thread is the place for them. I'm here to listen, get it off your chest.
Excited to share the work we have been doing on TowerLLM, a 7B model supporting 10 languages that excels at multilingual tasks such as translation! We release both the pretrained model (TowerBase) and the instruction tuned version (TowerInstruct). Both available in @huggingface
Introducing Tower our cutting-edge multilingual #LLM for translation-related tasks! 🚀 With 7B parameters and support for 10 languages, Tower dominates in pre-translation tasks and machine translation. 🌎 Explore the future of #NLP now 👉 hubs.li/Q02g7_9B0
Super happy to share something we have been working on lately: TowerLLM, a multilingual model geared towards cross-lingual and translation-related tasks. It has very good performance on translation benchmarks and supports 10 languages.
Introducing Tower our cutting-edge multilingual #LLM for translation-related tasks! 🚀 With 7B parameters and support for 10 languages, Tower dominates in pre-translation tasks and machine translation. 🌎 Explore the future of #NLP now 👉 hubs.li/Q02g7_9B0
6.1mpl, #cappuccino. Mediter-run #running in Barcelona, cheers!
@voooooogel if interested in learning more about reference based evaluation, i can’t recommend @_danieldeutsch’s work enough! his dissertations on challenges of evaluating summarization models is a great starting point repository.upenn.edu/server/api/cor…
TOMORROW! I am organizing a virtual session at #CMStatistics2023 about the statistical challenges in model-based data science. Excited for the great talks by @yuvalbenj @ch_hardmeier and Stefan Riezler!🧮📊
Congratulations, it's a well deserved award! I especially love how quickly you reacted to the emergence of this issue and came up with a solution.
Excited to receive an Outstanding Paper award for this work at @emnlpmeeting! Thanks to my co-authors George Foster and @markuseful! Updated version available here: aclanthology.org/2023.emnlp-mai…
@_danieldeutsch @emnlpmeeting @markuseful I knew it! This is a great paper and it's well-deserved!
@_danieldeutsch @emnlpmeeting @markuseful Congrats Dan!
Wow I really cursed myself with this tweet. Flight turned around halfway across the Atlantic due to... snowstorms 😂 Will be making another attempt today! But I've learned my lesson about tempting the snow gods ❄️❄️
Snowstorms notwithstanding, I'm on my way to Singapore for @emnlpmeeting 🤞🛫 Reach out if you will be there and want to chat about semantic parsing, model calibration, ambiguity, or pretty much anything else! Or if you have any tips on where to go/what to do in Singapore!
Announcing the Conference of LLM-Generated Reviews. It's like all the other venues, but we're honest about it.
Can we not criticize LLM but pinpoint errors it makes and automatically guide it with fine-grained actionable feedback? Can we formulate iterative refinement into a local search problem, simulated annealing? My cool summer intern work @Google @_danieldeutsch @markuseful @ucsbNLP
🚨 New Dataset Alert 🚨 I'm extremely excited to announce Universal NER v1, available now. It is gold-standard human annotations of 18 datasets covering 12 languages, based on Universal Dependencies texts. This is the first data release of the UNER project. 1/3
Eval4NLP23 has concluded. We thank everyone + congratulate our shared task winners on inducing high-quality metrics for MT+summ. using prompting and efficient models: "HIT-MI&T Lab" (even beating GEMBA + COMET🚀) & "DSBA". Shared task overview paper: arxiv.org/pdf/2310.19792…
Trends for United States
You might like
