Greg Durrett @gregd_nlp
CS professor at UT Austin. I do NLP most of the time. he/him Joined December 2017-
Tweets1K
-
Followers6K
-
Following752
-
Likes3K
🚨New NAACL 2024 Paper 🚨 We trained four vision-language models on 23 source tasks and evaluated on 29 target tasks in order to look for patterns and latent factors in vision-language evaluation benchmarks. arxiv.org/abs/2404.02415
Great lineup of speakers at our second Disinformation Day at UT Austin! Registration open to all!
Great lineup of speakers at our second Disinformation Day at UT Austin! Registration open to all!
Can LMs correctly distinguish🔎 confusing entity mentions in multiple documents? We study how current LMs perform QA task when provided ambiguous questions and a document set📚 that requires challenging entity disambiguation. Work done at @UTCompSci✨ w/ @xiye_nlp, @eunsolc
LLMs can mimic human curiosity by generating open-ended inquisitive questions given some context, similar to how humans wonder when they read. But which ones are more important to be answered?🤔 We predict the salience of questions, substantially outperforming GPT-4.🌟 🧵1/5
Check out Liyan's system + benchmark! Strong LLM fact-checking models like MiniCheck will allow response refinement and training for better factuality (work in progress!). LLM-AggreFact collects 10 high-quality labeled datasets of LLM errors in the literature to evaluate them!
Check out Liyan's system + benchmark! Strong LLM fact-checking models like MiniCheck will allow response refinement and training for better factuality (work in progress!). LLM-AggreFact collects 10 high-quality labeled datasets of LLM errors in the literature to evaluate them!
📢 New Preprint! Can LLMs detect mistakes in LLM responses? We introduce ReaLMistake, error detection benchmark with errors by GPT-4 & Llama 2. Evaluated 12 LLMs and showed LLM-based error detectors are unreliable! @ruizhang_nlp @Wenpeng_Yin @armancohan + arxiv.org/abs/2404.03602
🥱Tired of LLM’s generic “hope you feel better” responses? 🧠Can we dive much deeper and instill cognitive capabilities in them? Under the right instructions, LLMs (zero-shot) score very high per expert psychologist evaluators! 📢arxiv.org/abs/2404.01288 1/🧵
Summarizing long documents (>100K tokens) is a popular use case for LLMs, but how faithful are these summaries? We present FABLES, a dataset of human annotations of faithfulness & content selection in LLM-generated summaries of books. arxiv.org/abs/2404.01261 🧵below:
@gregd_nlp @cmalaviya11 Abhika @mishrabhika also led a project where we annotated 1k LLM responses (llama2 7b&70b chat and ChatGPT) to diverse instruction following prompts with span level hallucinations and types. The data is publicly available!
@gregd_nlp @cmalaviya11 Abhika @mishrabhika also led a project where we annotated 1k LLM responses (llama2 7b&70b chat and ChatGPT) to diverse instruction following prompts with span level hallucinations and types. The data is publicly available!
Last fall, along with Joydeep Biswas and Don Fussell, I created a 1-credit-hour course on "The Essentials of AI for Life and Society". The lecture videos are now all online: hr.utexas.edu/events/cs-109-…
What roles do emotion triggers play in language models’ emotion predictions? 📢 Our #NAACL2024 paper examines this with the help of our dataset, EmoTrigger! Our study leverages explainability tools & elicits natural explanations from LLMs. 📑 tinyurl.com/5taz5y7y w/@jessyjli
Excited to share something that we've needed since the early open RLHF days: RewardBench, the first benchmark for reward models. 1. We evaluated 30+ of the currently available RMs (w/ DPO too). 2. We created new datasets covering chat, safety, code, math, etc. We learned a lot.…
New preprint “The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models” w/ @danfriedman0 & @danqi_chen! We use structured pruning to find surprising phenomena and new insights on how a pretrained LM generalizes! arxiv.org/abs/2403.03942 1/8
(((ل()(ل() 'yoav))).. @yoavgo
46K Followers 2K Following
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Yoav Artzi @yoavartzi
13K Followers 163 Following Research/prof @cs_cornell + @cornell_tech🚡 / https://t.co/9YnWry7yHs / https://t.co/3VmRSyYm2d / asso. faculty director @arxiv / building https://t.co/f9QkzO5kaC
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Graham Neubig @gneubig
31K Followers 588 Following Associate professor at CMU, studying natural language processing and machine learning.
Akari Asai @AkariAsai
11K Followers 650 Following Ph.D. student @uwcse & @uwnlp. NLP. IBM Ph.D. fellow (2022-2023). Meta student researcher (2023-) . ☕️ 🐕 🏃♀️🧗♀️🍳
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Delip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈
Luca Soldaini 🎀 @soldni
6K Followers 1K Following I like tokens! Lead for OLMo data team at @allen_ai (Dolma 🍇), OSS is fun, @QueerInAI organizer 🤖☕️🍕they/them (views mine, not my employer’s)
Jacob Andreas @jacobandreas
14K Followers 958 Following Teaching computers to read. Assoc. prof @MITEECS / @MIT_CSAIL (he/him). https://t.co/5kCnXHjtlY https://t.co/2A3qF5vdJw
William Wang @WilliamWangNLP
14K Followers 718 Following UCSB NLP Lab + ML Center. https://t.co/6TOnqbk6YT https://t.co/KJYhnav3Et Mellichamp Chair Prof. at UCSB CS. PhD @ CMU SCS. Areas: #NLProc, Machine Learning, AI.
Kayo Yin @kayo_yin
8K Followers 560 Following PhD student @berkeley_ai @berkeleynlp working on interpretability and signed languages. Former @msftresearch @deepmind @carnegiemellon @polytechnique. 🇫🇷🇯🇵
Sam Bowman @sleepinyourhat
35K Followers 3K Following AI alignment + LLMs at NYU & Anthropic. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Danish Pruthi @danish037
7K Followers 628 Following Faculty at Indian Institute of Science, Bangalore. PhD from @LTIatCMU.
Tal Linzen @tallinzen
16K Followers 893 Following Professor @nyuling and @NYUDataScience, research scientist @GoogleAI
Naomi Saphra @nsaphra
7K Followers 1K Following Waiting on a robot body. ML/NLP. All opinions are universal and held by both employers and family. Same username on every lifeboat off this sinking ship.
Mark Dredze @mdredze
4K Followers 786 Following John C Malone Professor at @JohnsHopkins @JHUCompSci @jhuclsp @jhumceh; Part time @techatbloomberg (tweets my own) Mastodon @[email protected]
Shruti Rijhwani @shrutirij
4K Followers 499 Following * Research Scientist @GoogleDeepMind * #NLProc research * PhD from @LTIatCMU * Amateur woodworker, scuba diver, foosball player
Yiqing Xie @YiqingXXX
67 Followers 89 Following ✨ NLP for Code & Code for NLP 🎓 PhD student @LTIatCMU; MSCS @dmguiuc. 👩💻 Intern (incoming) @meta; (previously) @MSFTResearch; @AlibabaDAMO.
Emre Yavuz @Emre_Yavuz_21
2K Followers 6K Following PhD student in Cognitive Neuroscience at UCL | Neuroscience MSc graduate - Imperial College | 🎼🏔✈️🍲📸 | MedTech | Founders of the Future Fellow
Vikram Dutt @vd_
836 Followers 7K Following
精神病狗婊子杂.. @frkglp
0 Followers 3K Following 神病狗婊子杂种邓小平，刘少奇就是整个世界的敌人，它那套歪把戏不除，世界战乱不断。Cgkl精神病狗婊子杂种习近平被凌迟处死。Cgk凌迟处死精神病狗婊子杂种中共狗屁家族邓小平，习近平，陈云，刘少奇，陈一新，张又侠，何卫东，刘振立，苗华，董军。锸s你跟踪本人的精神病狗婊子杂种全部中共空军、警察、台湾间谍
Vivek Baghel @vivkba
134 Followers 807 Following fine tuning_ also help Creators and Podcasts in videos🤝
Seren Scott @Serenaughty
28 Followers 192 Following
郑晓琼（Audrey Zh.. @Audrey_802
95 Followers 545 Following CEO of Beijing Open Space Technology 🌛 Global Publisher of JieTeng（China) 🌞ETH Zurich+HSG @embaX_swiss 👸 “Wave Rider” Translator
simpletrading @simpletrad17722
411 Followers 7K Following
ssteevens @Steevens43
159 Followers 5K Following
Viviana @Viviana75842443
2 Followers 161 Following
Robert Zhang @0xrobertzhang
201 Followers 285 Following Incoming PhD @UTCompSci | CS undergrad & Masson Fellow @JohnsHopkins | Formerly @Reactjs @MetaOpenSource
Jerrin John Thomas @jr_john_
14 Followers 251 Following
kimi @kimi59835793
0 Followers 265 Following
Anh Trần Hoàng @Anh_Tran_Hoang
0 Followers 39 Following
Wenzhao Qiu @WenzhaoQiu
1 Followers 138 Following
Dino CarloS @d1n0CS
0 Followers 972 Following
Eric Ringger @eringger
300 Followers 988 Following Computer scientist: machine learning, NLP, text mining. Aspiring disciple.
Tanmoy Chakraborty @Tanmoy_Chak
2K Followers 823 Following Associate Professor @iitdelhi; ACM Distinguished Speaker; Lab @lcs2lab; Previously @IIITDelhi @UofMaryland @iitkgp; #NLP #SocialComputing #GraphNeuralNetworks
Mohammed Amine BEN CH.. @AmineLehocine
35 Followers 1K Following
Florian Laurent @MasterScrat
2K Followers 5K Following Building ⚡️ @dreamlookai: Finetune Stable Diffusion in *Minutes* on TPUs
Tunazzina Islam @Tunaz_Islam
180 Followers 632 Following Ph.D. candidate @PurdueCS Travelholic🌎Yogi🧘♀️Mom of 2 👧👦 first-generation Ph.D. student.
Trevor Loy @trevorloy
17K Followers 2K Following VC investor emerging ecosystems @FlywheelVC. Lecturer entrepreneurship & VC @Stanford. Prev: BoD @NVCA; Mentor @KauffmanFellows; 3x founder; Chip design @Intel.
Mai Hiền @HienHMai
6 Followers 99 Following
Edmar Miyake @emiyake
38 Followers 473 Following
Ravi Shankar @JustAnotherRavi
353 Followers 3K Following IIT Madras Alum, Civil Engineer, Ex Consultant at Ernst and Young, Currently pursuing PhD in Project Finance at NUS.
Mahesh Sathiamoorthy @madiator
9K Followers 933 Following LLMs and Data. Discuss about data for LLMs: https://t.co/x4iAft5cHV Ex-GoogleDeepMind
Gilbert Mizrahi @GilbertMizrahi
1K Followers 2K Following Entrepreneur, technologist, experimenting with generative AI and published author.
Atrey Desai @atreydesai
0 Followers 29 Following
EO @EO84494235
58 Followers 1K Following
Jenna Russell @jennajrussell
1 Followers 76 Following Incoming Cs PhD Student @umass advised by @MohitIyyer, currently @BankofAmerica NLP, formerly @CornellCIS
Eva Louise Marie Gabr.. @e681554349
9 Followers 3K Following
Kelly W. Zhang @kewzha
138 Followers 174 Following
Mingrui Liu @mingruiliuCS
325 Followers 838 Following Assistant Professor @GMUCompSci, Postdoc @BU_Computing, PhD in Computer Science at @uiowa
Janhavee Shinde @SJanhavee
58 Followers 2K Following
Carter Leffen @carterleffen
1K Followers 870 Following We are here to learn, make a difference, and have fun. - Deming
Michael Johnson @onemoremichael
461 Followers 5K Following Co-Founder of Ref | Leaving the resume behind. Al-native platform that surfaces relevant & authentic context on candidates, validated by Al-assisted referrals.
Pensé FFun @inftyCategory
100 Followers 6K Following
David Nikson @SamuelD76488206
73 Followers 111 Following
Imad Khwaja @flyingblackswan
145 Followers 2K Following SaaS Growth || SEO Marketing Agency || Entrepreneur
Robonomous @realpolity101
2K Followers 2K Following controls,robotics & UAVs//Algos,CV//usual sh!tposter XD //
(((ل()(ل() 'yoav))).. @yoavgo
46K Followers 2K Following
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Yoav Artzi @yoavartzi
13K Followers 163 Following Research/prof @cs_cornell + @cornell_tech🚡 / https://t.co/9YnWry7yHs / https://t.co/3VmRSyYm2d / asso. faculty director @arxiv / building https://t.co/f9QkzO5kaC
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Graham Neubig @gneubig
31K Followers 588 Following Associate professor at CMU, studying natural language processing and machine learning.
Akari Asai @AkariAsai
11K Followers 650 Following Ph.D. student @uwcse & @uwnlp. NLP. IBM Ph.D. fellow (2022-2023). Meta student researcher (2023-) . ☕️ 🐕 🏃♀️🧗♀️🍳
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Delip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈
Luca Soldaini 🎀 @soldni
6K Followers 1K Following I like tokens! Lead for OLMo data team at @allen_ai (Dolma 🍇), OSS is fun, @QueerInAI organizer 🤖☕️🍕they/them (views mine, not my employer’s)
Christopher Manning @chrmanning
127K Followers 116 Following Director, @StanfordAILab. Assoc. Director, @StanfordHAI. Founder, @stanfordnlp. Prof. CS & Linguistics, @Stanford. IP @aixventureshq. 🇦🇺 Do #NLProc & #AI. 👋
Jacob Andreas @jacobandreas
14K Followers 958 Following Teaching computers to read. Assoc. prof @MITEECS / @MIT_CSAIL (he/him). https://t.co/5kCnXHjtlY https://t.co/2A3qF5vdJw
William Wang @WilliamWangNLP
14K Followers 718 Following UCSB NLP Lab + ML Center. https://t.co/6TOnqbk6YT https://t.co/KJYhnav3Et Mellichamp Chair Prof. at UCSB CS. PhD @ CMU SCS. Areas: #NLProc, Machine Learning, AI.
Kayo Yin @kayo_yin
8K Followers 560 Following PhD student @berkeley_ai @berkeleynlp working on interpretability and signed languages. Former @msftresearch @deepmind @carnegiemellon @polytechnique. 🇫🇷🇯🇵
Sam Bowman @sleepinyourhat
35K Followers 3K Following AI alignment + LLMs at NYU & Anthropic. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Tal Linzen @tallinzen
16K Followers 893 Following Professor @nyuling and @NYUDataScience, research scientist @GoogleAI
Naomi Saphra @nsaphra
7K Followers 1K Following Waiting on a robot body. ML/NLP. All opinions are universal and held by both employers and family. Same username on every lifeboat off this sinking ship.
Mark Dredze @mdredze
4K Followers 786 Following John C Malone Professor at @JohnsHopkins @JHUCompSci @jhuclsp @jhumceh; Part time @techatbloomberg (tweets my own) Mastodon @[email protected]
Tanmoy Chakraborty @Tanmoy_Chak
2K Followers 823 Following Associate Professor @iitdelhi; ACM Distinguished Speaker; Lab @lcs2lab; Previously @IIITDelhi @UofMaryland @iitkgp; #NLP #SocialComputing #GraphNeuralNetworks
Fangcong Yin @fangcong_y10593
14 Followers 11 Following
Anirudh Khatry @AnirudhKhatry
427 Followers 746 Following Incoming CS PhD @UTAustin | Research Fellow at @ProseMsft, @Microsoft | AI4Code | Guitarist | VJTI ‘21
Ruiyi Wang @RuiyiWang153
127 Followers 177 Following Incoming PhD @ucsd_cse | MS @LTIatCMU | BS @UMichCSE and @sjtu1896 | NLP & HCI research
Adithya Bhaskar @AdithyaNLP
58 Followers 51 Following First Year CS Ph.D. student at Princeton University (@princeton_nlp), previously CS undergrad at IIT Bombay
Jason Weston @jaseweston
9K Followers 569 Following Research @MetaAI+NYU. Pretrain+FT: NLP from Scratch (2011). Multilayer attention+position embed+LLM: MemNets (2015). Recent (2023+):Sys 2 Attn, Self-Rewarding..
Shreya Shankar @sh_reya
39K Followers 589 Following I study ML & AI engineers and try to make their lives a little better. PhD-ing in databases & HCI @Berkeley_EECS @UCBEPIC and MLOps-ing around town. She/they.
*SEM 2024 @_starsem
374 Followers 91 Following The 13th Joint Conference on Lexical and Computational Semantics. 16 June 2024
Ofir Press @OfirPress
10K Followers 3K Following I build tough benchmarks for LMs and then I get the LMs to solve them. Postdoc @Princeton. PhD from @nlpnoah @UW. Ex-visiting researcher @MetaAI & @MosaicML.
Harsh Trivedi @harsh3vedi
263 Followers 487 Following #NLProc PhD candidate in @stonybrooku. Past intern @allen_ai & student research visitor @CILVRatNYU
main @main_horse
8K Followers 477 Following AGI Believer. Haven't applied @OpenAI. Likes are not always endorsement.
Tim Dettmers @Tim_Dettmers
29K Followers 821 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.
Nadia Polikarpova @polikarn
4K Followers 307 Following Associate prof @ucsd_cse. Building tools for program verification and synthesis.
Nathan Lambert @natolambert
25K Followers 690 Following Figuring out AI @allen_ai, "rl boi" DM me papers. Writes @interconnectsai, talks @retortai Has phd and some credentials
Jiatao Gu @thoma_gu
3K Followers 2K Following Machine Learning Researcher at @Apple ML Research (MLR) based in NYC | ex-FAIRer | PhD from HKU | Research on Generative AI for multimodalities. また日本語もできます。
Sanket Vaibhav Mehta,.. @sanketvmehta
685 Followers 1K Following Research Scientist @GoogleAI | Ph.D. @LTIatCMU @SCSatCMU @CarnegieMellon | Past @AdobeResearch, @IITRoorkee
Ritika Mangla @ritikarmangla
22 Followers 56 Following CS Graduate Student at University of Texas at Austin
Michi Yasunaga @michiyasunaga
3K Followers 867 Following CS PhD @Stanford working on language models and multimodal models. Previously @Meta @GoogleDeepMind @Yale
Pavel Izmailov @Pavel_Izmailov
6K Followers 1K Following Researcher @xai Incoming Assistant Professor @nyuniversity 🏙️ Previously @OpenAI #StopWar 🇺🇦
Prithviraj (Raj) Amma.. @rajammanabrolu
5K Followers 520 Following Interactive & grounded AI, RL, NLP. Assistant Prof @UCSanDiego. Research Scientist @DbrxMosaicAI. Prev: @allen_ai, @GeorgiaTech
Jakob Uszkoreit @kyosu
4K Followers 276 Following
AK @_akhaliq
310K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo follow on Hugging Face: https://t.co/q2Qoey80Gx
Ziyu Yao @ZiyuYao
1K Followers 544 Following Asst Prof @GeorgeMasonU CS interested in #NLProc #AI. Alum @OhioState. Prev intern @LTIatCMU @MSFTResearch @FujitsuAmerica @Tsinghua_Uni.
Dylan HadfieldMenell @dhadfieldmenell
2K Followers 2K Following Assistant Prof @MITEECS working on value (mis)alignment in AI systems; @[email protected] @[email protected] he/him
ICLR 2024 @iclr_conf
41K Followers 40 Following International Conference on Learning Representations #ICLR2024. SPC is @yisongyue and GC is @_beenkim OpenReview:https://t.co/OD1sg0r7F8
Tom McCoy @RTomMcCoy
3K Followers 483 Following Assistant professor @YaleLinguistics. Studying computational linguistics, cognitive science, and AI. He/him.
Griffiths Computation.. @cocosci_lab
4K Followers 129 Following Tom Griffiths' Computational Cognitive Science Lab. Studying the computational problems human minds have to solve.
Dixin Tang @DixinTang
569 Followers 350 Following Assistant Professor at UT Austin; Previously postdoc at UC Berkeley, Ph.D. at UChicago CS; Database Researcher
Princeton PLI @PrincetonPLI
1K Followers 19 Following Princeton University initiative enhancing fundamental understanding of AI, enabling its use in academic disciplines, and examining AI's societal implications.
Chaitanya Malaviya @cmalaviya11
99 Followers 121 Following PhD student at UPenn | currently @GoogleDeepMind
Cornell Bowers Comput.. @CornellCIS
6K Followers 384 Following The @Cornell Ann S. Bowers College of Computing and Information Science develops computing and information technologies & explores societal and human impact.
Shunyu Yao @ShunyuYao12
7K Followers 857 Following Language agents (ReAct, Reflexion, Tree of Thoughts) for digital automation (WebShop, SWE-bench, SWE-agent)
Sebastian Schuster @sebschu
2K Followers 2K Following Lecturer @LinguisticsUCL, and starting in 2025, Assistant Professor @univienna. #nlproc, computational and experimental semantics and pragmatics. he/him.I guess the answer is just that the summarization metrics always were really bad.
@srush_nlp I feel this is really captured by @tanyaagoyal and colleagues: arxiv.org/abs/2209.12356 If you start looking at the *human* preferences of summaries, things start looking different very quickly. I'd expect the same tendency holds for GPT-2 (minus instruction tuning)?
For some reason GPT2 never really cracked summarization. Always curious why that one was hard.
(1/7) Do you want to test code generation models on the domains you care about? Struggling to find existing benchmarks that suit your needs? Our new work *CodeBenchGen* helps you build execution-based benchmarks based on your selected code fragments! (arxiv.org/abs/2404.00566)
Just wrapped up another edition of DL for NLP! Managed to have covered both statistical and advanced NLP. Big shoutout to 78 students for active participation. Page: sites.google.com/view/ell881 Course content is inspired by @chrmanning @MohitIyyer @gregd_nlp @gneubig @danqi_chen 🙏
I wanted to take the opportunity to say a special thanks to my two PhD advisors, @IsilDillig and @gregd_nlp, without whose wisdom, mentorship, and kindness I couldn't possibly be where I am today. For anyone considering a PhD in AI for Code, I highly recommend their groups!
The length bias may be quite prevalent. @prasann_singhal @gregd_nlp et al. made a similar finding regarding RLHF of LLMs. arxiv.org/abs/2310.03716
🚨New NAACL 2024 Paper 🚨 We trained four vision-language models on 23 source tasks and evaluated on 29 target tasks in order to look for patterns and latent factors in vision-language evaluation benchmarks. arxiv.org/abs/2404.02415
Today was this semester's last session of the ✨Social Applications and Impact of NLP✨ seminar that I organized. Huge thanks to the first author presenters who joined us! What an amazing lineup of papers, check them out: jessyli.com/courses/lin393
Some personal updates: I joined OpenAI a few months ago, working on all things robustness/safety/privacy. Also, we are working to publish more of our safety work. See my first project here below, where we make initial progress on prompt injections and other attacks!
Introducing the Instruction Hierarchy, our latest safety research to advance robustness for prompt injections and other ways of tricking LLMs into executing unsafe actions. More details: arxiv.org/abs/2404.13208
Join us on May 2 for Disinformation Day 2024! This virtual event brings together researchers and thought leaders from a variety of disciplines and sectors to discuss approaches to curbing the spread of digital disinformation. Learn more and register: disinfoday.github.io
Can LLM comprehensively capture information spread across multiple documents? Can LLM distinguish confusing entity mentions? Please check out our preprint on multi-document reasoning for LLM, focusing on entity disambiguation!
Can LMs correctly distinguish🔎 confusing entity mentions in multiple documents? We study how current LMs perform QA task when provided ambiguous questions and a document set📚 that requires challenging entity disambiguation. Work done at @UTCompSci✨ w/ @xiye_nlp, @eunsolc
Can LMs correctly distinguish🔎 confusing entity mentions in multiple documents? We study how current LMs perform QA task when provided ambiguous questions and a document set📚 that requires challenging entity disambiguation. Work done at @UTCompSci✨ w/ @xiye_nlp, @eunsolc
Unlike any sane person who gets a PhD in NLP right now, afterwards I made a game. I just released it in early access talktomehuman.com Talk to NPCs who talk back at you, try to persuade your way out of sticky situations
LLMs can mimic human curiosity by generating open-ended inquisitive questions given some context, similar to how humans wonder when they read. But which ones are more important to be answered?🤔 We predict the salience of questions, substantially outperforming GPT-4.🌟 🧵1/5
MiniCheck 用于高效地对 LLM 生成的文本进行事实核查 论文作者 @LiyanTang4 @PhilippeLaban, @gregd_nlp 这项工作的核心是创建了一个合成训练数据集，该数据集通过结构化的过程生成具有挑战性的事实错误实例，以此来训练小型模型。这些模型在性能上达到了 GPT-4 的水平，但成本仅为 GPT-4 的…
🔎📄New model & benchmark to check LLMs’ output against docs (e.g., fact-check RAG) 🕵️ MiniCheck: a model w/GPT-4 accuracy @ 400x cheaper 📚LLM-AggreFact: collects 10 human-labeled datasets of errors in model outputs arxiv.org/abs/2404.10774 w/ @PhilippeLaban, @gregd_nlp 🧵
Excited & honored to give the Distinguished Lecture at @CSatUSC today! Looking forward to meeting the awesome @nlp_usc group 🙂 -- thanks @swabhz + everyone for the kind invitation 🙏 PS. Details in the link below, feel free to stop by!
Looking forward to welcome @mohitban47 as a distinguished lecturer to @CSatUSC tomorrow and learn about his latest work on Multimodal LLMs: viterbi.usc.edu/calendar/?even… It's going to be a good day at @nlp_usc
Excited to announce that "NELLIE: A Neuro-Symbolic Inference Engine for Grounded, Compositional, and Explainable Reasoning" has been accepted to IJCAI2024! Check out our work on an interpretable, systematic hypothesis proving engine: arxiv.org/abs/2209.07662
🚨 New preprint! 🚨 @ben_vandurme and I built a neuro-symbolic expert system using language models! ArXiv: arxiv.org/abs/2209.07662 (1/N)
(1/N) New paper! Dataset Reset Policy Optimization for RLHF (arxiv.org/pdf/2404.08495…) RLHF is a popular paradigm for fine-tuning generative models. But the question is, can we design algorithms that take advantage of additional properties of the RLHF framework?
Check out @LiyanTang4's great work! Using very clever synthetic data generation schemes, he trained a very strong fact-checking model, which can get GPT4-level accuracies, while being 400x cheaper. The model which is on HF will be very useful in RAG/summarization settings.
🔎📄New model & benchmark to check LLMs’ output against docs (e.g., fact-check RAG) 🕵️ MiniCheck: a model w/GPT-4 accuracy @ 400x cheaper 📚LLM-AggreFact: collects 10 human-labeled datasets of errors in model outputs arxiv.org/abs/2404.10774 w/ @PhilippeLaban, @gregd_nlp 🧵
Trends for United States
You might like
