Michael Pearce @_MichaelPearce

Mechanistic Interpretability @ Goodfire | Physics | Evolution Joined September 2015

Tweets

84
Followers

162
Following

609
Likes

600

Goodfire @GoodfireAI

3 weeks ago

We're excited to announce a collaboration with @MayoClinic! We're working to improve personalized patient outcomes by extracting richer, more reliable signals from genomic & digital pathology models. That could mean novel biomarkers, personalized diagnostics, & more.

3 9 70 8K 22

Download Image

Goodfire @GoodfireAI

3 weeks ago

Does making an SAE bigger let you explain more of your model's features? New research from @ericjmichaud_ models SAE scaling dynamics, and explores whether SAEs will pack increasingly many latents onto a few multidimensional features, rather than learning more features.

3 20 151 25K 86

Download Image

Goodfire @GoodfireAI

a month ago

Adversarial examples - a vulnerability of every AI model, and a “mystery” of deep learning - may simply come from models cramming many features into the same neurons! Less feature interference → more robust models. New research from @livgorton 🧵 (1/4)

4 24 246 27K 136

Download Image

Goodfire @GoodfireAI

a month ago

New research! Post-training often causes weird, unwanted behaviors that are hard to catch before deployment because they only crop up rarely - then are found by bewildered users. How can we find these efficiently? (1/7)

10 43 378 45K 200

Jack Merullo @jack_merullo_

2 months ago

Could we tell if gpt-oss was memorizing its training data? I.e., points where it’s reasoning vs reciting? We took a quick look at the curvature of the loss landscape of the 20B model to understand memorization and what’s happening internally during reasoning

14 51 509 43K 397

Download Image

Goodfire @GoodfireAI

2 months ago

New research with coauthors at @Anthropic, @GoogleDeepMind, @AiEleuther, and @decode_research! We expand on and open-source Anthropic’s foundational circuit-tracing work. Brief highlights in thread: (1/7)

3 22 248 18K 132

Eric Ho @ericho_goodfire

2 months ago

Just wrote a piece on why I believe interpretability is AI’s most important frontier - we're building the most powerful technology in history, but still can't reliably engineer or understand our models. With rapidly improving model capabilities, interpretability is more urgent,…

1 17 132 18K 53

Download Image

Goodfire @GoodfireAI

3 months ago

(1/7) New research: how can we understand how an AI model actually works? Our method, SPD, decomposes the *parameters* of neural networks, rather than their activations - akin to understanding a program by reverse-engineering the source code vs. inspecting runtime behavior.

14 86 786 101K 599

Download Image

Goodfire @GoodfireAI

4 months ago

New research update! We replicated @AnthropicAI's circuit tracing methods to test if they can recover a known, simple transformer mechanism.

2 53 499 52K 243

Download Image

Curt Jaimungal @TOEwithCurt

8 months ago

“There is no wave function...” This claim by Jacob Barandes sounds outlandish, but allow me to justify it with a blend of intuition regarding physics and rigor regarding math. We'll dispel some quantum woo myths along the way. (1/13)

60 147 1K 147K 990

Download Image

Goodfire @GoodfireAI

4 months ago

We created a canvas that plugs into an image model’s brain. You can use it to generate images in real-time by painting with the latent concepts the model has learned. Try out Paint with Ember for yourself 👇

39 98 923 175K 585

Download Video

Adam Karvonen @a_karvonen

7 months ago

We're excited to announce the release of SAE Bench 1.0, our suite of Sparse Autoencoder (SAE) evaluations! We have also trained / evaluated a suite of open-source SAEs across 7 architectures. This has led to exciting new qualitative findings! Our findings in the 🧵 below 👇

4 35 193 36K 112

Download Image

Andy Keller @t_andy_keller

7 months ago

In the physical world, almost all information is transmitted through traveling waves -- why should it be any different in your neural network? Super excited to share recent work with the brilliant @mozesjacobs: "Traveling Waves Integrate Spatial Information Through Time" 1/14

148 925 7K 757K 5K

Download Gif

Bart Bussmann @BartBussmann

8 months ago

Do SAEs find the ‘true’ features in LLMs? In our ICLR paper w/ @NeelNanda5 we argue no The issue: we must choose the number of concepts learned. Small SAEs miss low-level concepts, but large SAEs miss high-level concepts - it’s sparser to compose them into low-level concepts

3 38 270 40K 221

Download Image

Nora Belrose @norabelrose

8 months ago

MLPs and GLUs are hard to interpret, but they make up most transformer parameters. Linear and quadratic functions are easier to interpret. We show how to convert MLPs & GLUs into polynomials in closed form, allowing you to use SVD and direct inspection for interpretability 🧵

5 32 300 31K 200

Download Image

David D. Baek @dbaek__

8 months ago

1/9 🚨 New Paper Alert: Cross-Entropy Loss is NOT What You Need! 🚨 We introduce harmonic loss as alternative to the standard CE loss for training neural networks and LLMs! Harmonic loss achieves 🛠️significantly better interpretability, ⚡faster convergence, and ⏳less grokking!

76 530 4K 1.2M 4K

Download Gif

tdooms @thomasdooms

8 months ago

Can we understand neural networks from their weights? Often, the answer is no. An MLP's activation function obscures the relationship between inputs, outputs, and weights. In our new ICLR'25 paper, we study "bilinear MLPs", a special MLP that's performant AND interpretable! 🧵

3 44 395 43K 369

Ekdeep Singh @EkdeepL

11 months ago

Paper alert—accepted as a NeurIPS *Spotlight*!🧵👇 We build on our past work relating emergence to task compositionality and analyze the *learning dynamics* of such tasks: we find there exist latent interventions that can elicit them much before input prompting works! 🤯