Yanchen Liu @_yanchenliu

🚨 You can bypass ALL safety guardrails of GPT-OSS-120B 🚨❗🤯 How? By detecting behavior-associated experts and switching them on/off. 📄 Steering MoE LLMs via Expert (De)Activation 🔗 arxiv.org/abs/2509.09660 🧵👇

Soon, AI agents will act for us—collaborating, negotiating, and sharing data. But can they truly protect our privacy? We simulate privacy-critical scenarios, using alternating search to evolve attacks and defenses, uncovering severe vulnerabilities and building protections.

🚨 Incredibly excited to share that I'm starting my research group focusing on AI safety and alignment at the ELLIS Institute Tübingen and Max Planck Institute for Intelligent Systems in September 2025! 🚨 Hiring. I'm looking for multiple PhD students: both those able to start…

Want to talk to an expert on AI x Cyber security? Well, unfortunately @StevenyzZhang isn't here due to visa issues... So instead you'll have to chat with me about his amazing work at poster 311 in Hall X4!

💥New Paper💥 #LLMs encode harmfulness and refusal separately! 1️⃣We found a harmfulness direction 2️⃣The model internally knows a prompt is harmless, but still refuses it🤯 3️⃣Implication for #AI #safety & #alignment? Let’s analyze the harmfulness direction and use Latent Guard 🛡️

Jiachen Zhao @jcz12856876

2 months ago

💥New Paper💥 #LLMs encode harmfulness and refusal separately! 1️⃣We found a harmfulness direction 2️⃣The model internally knows a prompt is harmless, but still refuses it🤯 3️⃣Implication for #AI #safety & #alignment? Let’s analyze the harmfulness direction and use Latent Guard 🛡️ https://t.co/APFnjEtjPS

6 16 70 27K 36

Download Gif

We analyzed different #jailbreaking methods. - They suppress the refusal but did NOT change the models' judgements on harmfulness - (except for some cases in our persuasive jailbreaker) 🤯The model **knows** internally that a prompt is harmful, yet still accepts it🤯

Jiachen Zhao @jcz12856876

2 months ago

We analyzed different #jailbreaking methods. - They suppress the refusal but did NOT change the models' judgements on harmfulness - (except for some cases in our persuasive jailbreaker) 🤯The model **knows** internally that a prompt is harmful, yet still accepts it🤯 https://t.co/TZlMvHuCh0

1 0 1 2K 0

Download Image

🚨 70 million US workers are about to face their biggest workplace transmission due to AI agents. But nobody asks them what they want. While AI races to automate everything, we took a different approach: auditing what workers want vs. what AI can do across the US workforce.🧵

🧵There is a lot of conjecture about whether LLMs need to be trained with examples of harmful data in order to be more robust to exhibiting that harmful behavior. I think it probably depends. 🧵

Super excited to participate in @OpenAI Security Research Conference to talk about our PrivacyLens project and some recent exploration. I will be around from 5/1 to 5/2. DMs are open if you want to chat about agents/human-in-the-loop/sandboxing! events.openai.com/oaisecurity

New Anthropic Alignment Science blog post: Modifying LLM Beliefs with Synthetic Document Finetuning We study a technique for systematically modifying what AIs believe. If possible, this would be a powerful new affordance for AI safety research.

Thrilled to know that our paper, `Safety Alignment Should be Made More Than Just a Few Tokens Deep`, received the ICLR 2025 Outstanding Paper Award. We sincerely thank the ICLR committee for awarding one of this year's Outstanding Paper Awards to AI Safety / Adversarial ML.…

ICLR 2026 @iclr_conf

5 months ago

Thrilled to know that our paper, `Safety Alignment Should be Made More Than Just a Few Tokens Deep`, received the ICLR 2025 Outstanding Paper Award. We sincerely thank the ICLR committee for awarding one of this year's Outstanding Paper Awards to AI Safety / Adversarial ML.…

4 26 155 107K 88

Very excited that our work, "Safety Alignment Should be Made More Than Just a Few Tokens Deep" was recognized for an Outstanding Paper Award at #ICLR2025! We hope this is a step forward in improving and understanding robustness of language model alignment. It was great working…

ICLR 2026 @iclr_conf

5 months ago

Very excited that our work, "Safety Alignment Should be Made More Than Just a Few Tokens Deep" was recognized for an Outstanding Paper Award at #ICLR2025! We hope this is a step forward in improving and understanding robustness of language model alignment. It was great working…

4 26 155 107K 88

Delighted to share that two papers from our group @EPrinceton got recognized by the @iclr_conf award committee. Our paper, "Safety Alignment Should be Made More Than Just a Few Tokens Deep", received the ICLR 2025 Outstanding Paper Award. This paper showcases that many AI…

ICLR 2026 @iclr_conf

5 months ago

Delighted to share that two papers from our group @EPrinceton got recognized by the @iclr_conf award committee. Our paper, "Safety Alignment Should be Made More Than Just a Few Tokens Deep", received the ICLR 2025 Outstanding Paper Award. This paper showcases that many AI…

4 26 155 107K 88

New Anthropic research: AI values in the wild. We want AI models to have well-aligned values. But how do we know what values they’re expressing in real-life conversations? We studied hundreds of thousands of anonymized conversations to find out.

Congrats, your jailbreak bypassed an LLM’s safety by making it pretend to be your grandma! But did the model actually give a useful answer? In our new paper we introduce the jailbreak tax — a metric to measure the utility drop due to jailbreaks.

What if LLMs are sometimes capable of doing a task but don't try hard enough to do it? In a new paper, we use subtasks to assess capabilities. Perhaps surprisingly, LLMs often fail to fully employ their capabilities, i.e. they are not fully *goal-directed* 🧵

New Anthropic research: Tracing the thoughts of a large language model. We built a "microscope" to inspect what happens inside AI models and use it to understand Claude’s (often complex and surprising) internal mechanisms.

🔒 Can we build LLMs that are truly safe—without falling into an endless cycle of jailbreaks and patches? In this two-part thread, I dive into: 1️⃣ Adaptive defenses that respond in real time 2️⃣ Why some defenses may backfire and create new risks 👇

Wrote a blog post with some personal thoughts on AI safety. windowsontheory.org/2025/01/24/six…

🚀 DeepSeek-R1 is here! ⚡ Performance on par with OpenAI-o1 📖 Fully open-source model & technical report 🏆 MIT licensed: Distill & commercialize freely! 🌐 Website & API are live now! Try DeepThink at chat.deepseek.com today! 🐋 1/n