Kulin Shah @shahkulin98

We’re releasing Open-dCoder — the first fully open diffusion large language model for code: 🏋️ Pretraining pipeline 📂 Training data 📊 Evaluation suite ⚡ Inference code 📦 Checkpoints 🔗 Repo: github.com/pengzhangzhi/O… 📖 Blog: bit.ly/oDLLM-blog (1/n)

Wrote up some notes providing an introduction to discrete diffusion models, going into the theory of time-inhomogeneous CTMCs via generators/time-evolution systems. What motivated me was the sheer difficulty of finding a useful reference which laid out the theory (e.g.…

Excited to announce the Reliable ML workshop at @NeurIPSConf 2025! 📣 How do we build trustworthy models under distribution shift, adversarial attacks, strategic behavior, and missing data? Fantastic lineup of speakers/panelists: Ahmad Beirami (@abeirami), Surbhi Goel…

I am excited to announce that our AI institute (Institute for Foundations of Machine Learning, IFML) has been renewed. IFML was part of the first cohort of AI Institutes announced in 2020. Led by UT Austin, the new award will build on the trajectory of the past five years and…

congrats on the award!! great to see more work designing insightful tasks that bring out the role of token ordering & difficulty (should i say "indecipherability" ;-) ). I think the idea of learning token-level subproblems is broken both for diffusion (and next-token learning)

Sitan Chen @sitanch

7 months ago

congrats on the award!! great to see more work designing insightful tasks that bring out the role of token ordering & difficulty (should i say "indecipherability" ;-) ). I think the idea of learning token-level subproblems is broken both for diffusion (and next-token learning)

5 23 156 37K 101

Download Image

Today @ChenHenryWu and I will be presenting our #ICML work on creativity in the Oral 3A Reasoning session (West Exhibition Hall C) 10 - 11 am PT Or please stop by our poster right after @ East Exhibition Hall A-B #E-2505 11am-1:30pm. (Hope you enjoy some silly human drawings!)

Congrats to @UTAustin students Kulin Shah and Vasilis Kontonis who won an Outstanding Paper Award at #ICML2025! Their work pushes the boundaries of how AI models learn and understand the world. We’re proud to see @UTCompSci students leading at the frontier of machine learning.

Congrats to the Kim, Shah, et al. team from Harvard on winning an outstanding paper award at ICML for their work on understanding generation orderings in masked diffusion! And thank you for the shout-out to @InceptionAILabs!

📢 Thrilled to share our new paper, LASER: Attention with Exponential Transformation, accepted at ICML2025, work done at Google. Come by our poster presentation! 🗓️ Thurs, July 17th, 4:30-7pm 📍 West Exhibition Hall B2-B3, # W-915 Read the full paper here: arxiv.org/abs/2411.03493

A team from #KempnerInstitute, @hseas & @UTCompSci has won a best paper award at #ICML2025 for work unlocking the potential of masked diffusion models. Congrats to @Jaeyeon_Kim_0, @shahkulin98, Vasilis Kontonis, @ShamKakade6 and @sitanch. kempnerinstitute.harvard.edu/news/kempner-i… #AI

Thrilled to share that our work received the Outstanding Paper Award at ICML! I will be giving the oral presentation on Tuesday at 4:15 PM. @Jaeyeon_Kim_0 and I both will be at the poster session shortly after the oral presentation. Please attend if possible!

Sitan Chen @sitanch

7 months ago

Thrilled to share that our work received the Outstanding Paper Award at ICML! I will be giving the oral presentation on Tuesday at 4:15 PM. @Jaeyeon_Kim_0 and I both will be at the poster session shortly after the oral presentation. Please attend if possible!

5 23 156 37K 101

Download Image

Incredible to see how fast the field moves since we worked on masked diffusion arxiv.org/abs/2406.04329. Huge congratulations to @Jaeyeon_Kim_0 @sitanch for the award!

Kosta Derpanis @CSProfKGD

2 months ago

Incredible to see how fast the field moves since we worked on masked diffusion arxiv.org/abs/2406.04329. Huge congratulations to @Jaeyeon_Kim_0 @sitanch for the award!

keyboard_arrow_left Previous keyboard_arrow_right Next

1 13 81 19K 45

Download Image

Announcing Ambient Protein Diffusion, a state-of-the-art 17M-params generative model for protein structures. Diversity improves by 91% and designability by 26% over previous 200M SOTA model for long proteins. The trick? Treat low pLDDT AlphaFold predictions as low-quality data

Standard attention is softmax(QKᵀ)V. We explore 2-simplicial attention using additional keys K' for richer interactions: softmax(Q(K⊗K')ᵀ). 🧵

Aurko Roy @aurko79

2 months ago

Standard attention is softmax(QKᵀ)V. We explore 2-simplicial attention using additional keys K' for richer interactions: softmax(Q(K⊗K')ᵀ). 🧵

26 97 813 139K 519

Download Image

Excited to share that I’ll be presenting two oral papers in this ICML—see u guys in Vancouver!!🇨🇦 1️⃣ arxiv.org/abs/2502.06768 Understanding Masked Diffusion Models theoretically/scientifically 2️⃣ arxiv.org/abs/2502.09376 Theoretical analysis on LoRA training

Nice thread by Aayush on our new work on diffusion reward guidance! Was quite surprised how well this worked and how simple the algorithm is. Also happy that we finally managed to prove some rigorous guarantees for DPS (diffusion posterior sampling)

Aayush Karan @aakaran31

3 months ago

Nice thread by Aayush on our new work on diffusion reward guidance! Was quite surprised how well this worked and how simple the algorithm is. Also happy that we finally managed to prove some rigorous guarantees for DPS (diffusion posterior sampling)

7 27 159 16K 114

Download Image

Excited about this work, where we show that a simple algorithm of inverting candidate samples boosts the performance for reward guidance in diffusion, both in experiments and in theory! Check out the thread by Aayush for more details.

Aayush Karan @aakaran31

3 months ago

Excited about this work, where we show that a simple algorithm of inverting candidate samples boosts the performance for reward guidance in diffusion, both in experiments and in theory! Check out the thread by Aayush for more details.

7 27 159 16K 114

Download Image

Steering diffusion models with external rewards has recently led to exciting results, but what happens when the reward is inherently difficult? Introducing ReGuidance: a simple algorithm to (provably!) boost your favorite guidance method on hard problems! 🚀🚀🚀 A thread: (1/n)

We’ve developed Gemini Diffusion: our state-of-the-art text diffusion model. Instead of predicting text directly, it learns to generate outputs by refining noise, step-by-step. This helps it excel at coding and math, where it can iterate over solutions quickly. #GoogleIO

Introducing d1🚀 — the first framework that applies reinforcement learning to improve reasoning in masked diffusion LLMs (dLLMs). Combining masked SFT with a novel form of policy gradient algorithm, d1 significantly boosts the performance of pretrained dLLMs like LLaDA.