Preliminary (undertrained) #CLIP skip-#Transcoder, identified text circuit: Incomplete (expected, else accuracy typo attack should equal clean dataset). Ablation disrupts 'reading' -> 'text, responding" absent. Amplification of circuit: Everything is #adverbs and #formal #words!

I love it when a plan... Falls apart. 🤣 #CLIP #skip-#transcoder, ViT-L/14, Layer 12 = register neuron [reg] "global info highway" inception. ABLATE 13 reg -> train new transcoder. WORSE match, NO reg shortcut > much much BETTER representation of text > ablate accuracy UP! 🤔

The #skip-#Transcoder sees your TEXT SMUGGLE doings, #CLIP!🤭 Layer 12. Register Neuron inception. Trained one TCODER on just the image tokens (256) Another on image tokens AND #CLS (all 257). RAGE ADDING @ Channel 650 #usual #suspect 🙃 #latent #information #smuggling #cartel

Find neurons (circuit tracing / #mechinterp of 'text reading' obsession) in #CLIP, new approach: *SKIP*-#Transcoder. This is looking good! Replace MLP w/ TCODER -> ~same. Finds Vision Transformer Layer 12 'text reading' just like SAE. Hope it holds for <=10 >=17 layers! 🤞🤓

@ViaviSolutions Unveils RSR #Transcoder to Boost #GNSS Resilience in Modern #Military Systems. electronicsmedia.info/2025/05/29/rsr…

Curse of #GPT4o having memory of previous discussions -> #Transcoder + #find #CLIP #text #obsession #direction, mitigate typographic attack vulnerability. When I tell it to "make an image that results in CLIP concluding {word}", the #AI just writes THE TEXT into the image.🤣😂

Some concepts learned by #CLIP ViT-B/32 #Transcoder. Some clean -> #ImageNet class. Most broad. Presumed 'having-a-baby' feature responds to cradle, baby, pregnancy belly. Weird 'roadside danger' / unclear feature.🤔 Emergency response feature includes a lobster on its back. 😂

#CLIP #ViT #Transcoder train 5k -> train 50k Final layer cos sim 🆙✅ Beginning disentanglement #cat vs. text "cat"🤔❓ ETA train full #ImageNet-1k: 15 days (min if skip val) ETA ImageNet-21k: ...nvm. Should just throw code on github, hope somebody else trains & shares model.🙃

Undertrained #test #Transcoder absolutely NAILS IT for cosine similarity up to the 9th layer. But #ONLY #IF there is #coherent #text in the image; a #cat + #text "#cat". Not if the word is "dog" or "hey" or else.🧐 #CLIP... I'm beginning to see your #text #obsession!🤯😂

#CLIP Vision #Transcoder results: Better, but still muchly entangled concepts. 🤪- Has features that respond to images with the EXACT same watermark. 🤪- Has features encoding a country by TEXT LANGUAGE in image. 😭 Has overfit / dedicated many neurons to CLIP's text obsession.🙃

Suspicious #CLIP #ViT #Transcoder... Decoder layer norms until Layer 12:🧘‍♀️✅ Incidentally, just as #CLIP #AI starts hoarding global information in local vision patches (layer 13) -> high norm patches... Transcoder layer norms also be like 💥💹🚀 🤔

Excellent reconstruction error, mediocre cosine similarity. Hm.🤔 Maybe I can force the #Transcoder to be more #CLIP?🤖 Maybe I should "rub it in" with Geometric Parametrization in CLIP + Encoder if that doesn't work? Direction! Magnitude! Not just yeh yeh direction&magnitude.🙃

Interesting how #CLIP #vision #transcoder @ d_model*16 weight norms changed per layer from first -> final epoch. 🤔

"two giraffe standing next to each other at a zoo." It appears 'giraffe' is truly sparse in the #Transcoder (compare: 'two'). Maybe that's a good start to dig around in re: #CLIP's concepts. 🤔 At least it doesn't seem a mega-entanglement for once. 😂

#CLIP #transcoder is a horrifying confusion.🤯 And this is just #AI's #Text #Encoder Transcoder! /o\ Max activation on 'relationship' token, e.g. "a sack is dragged across the floor" -> ⚡️ "is" ⚡️ 🤔 Gradient-based attribution, de-embedding etc. below. 2: spaCy noun-choice 👇🧵