themagicianscode.studio @codeofmagicians

on track

Masa’s Son @masayoshisson

3 months ago

AI predictions for next 6-9 months -> Anthropic/ OAI fail to ban OSS in the US -> Anthropic/ OAI continue to innovate product but stall on fundamental tech breakthroughs -> SOTA Gemini 4 launches at 10x lower cost -> Elon open sources Grok -> Anthropic/ OAI ARR growth stalls as

228 640 8K 2.6M 10K

Wow. I woke up to an impressive number of upset people who apparently don’t understand how LLMs work and can’t cope well with me pointing it out. An LLM is a statistical next-token prediction model. That’s how it works. It’s an extraordinarily sophisticated calculator performing math you could theoretically do by hand (if you were sufficiently masochistic). Math doesn’t know what it’s calculating. Pi doesn’t spend its evenings contemplating the nature of circles, and c doesn’t ponder the relationship between matter and energy. That doesn’t make the results any less accurate, impressive, or useful. Between inference requests, an LLM isn’t sitting around thinking. It isn’t reconsidering its answers, wondering whether it made a mistake, contemplating its existence, or questioning whether it’s conscious. Its weights remain fixed (it doesn’t learn), and computation resumes when it receives input. We know this because we designed these systems. The architectures, mathematics, implementations, and research are extensively documented. You can study them yourself. Of course, some people immediately lean into comparisons of biological intelligence or philosophical debates over consciousness because that is what they know. But biological intelligence and artificial neural networks are fundamentally different systems. Similar outputs don’t establish equivalent internal experiences. I experience consciousness directly. I infer that other humans do too because we share biological structures and processes. We don’t have equivalent evidence for an LLM, and nothing about its known operation gives evidence of conscious experience. Which brings me back to my original point. To me, understanding is conscious comprehension. It means being aware of information, recognizing its meaning, and experiencing that comprehension. If I wasn’t conscious I could not understand. We can perform actions unconsciously. We can produce correct results without knowing why they’re correct. But I distinguish those capabilities from consciously understanding what we’re doing. An LLM can solve complex problems, explain difficult concepts, and produce remarkably accurate information. I give it full credit for those capabilities. But producing an answer that we rationalize as understanding isn’t the same as experiencing understanding. You don’t have to agree with my definitions. We can debate functional understanding versus conscious comprehension all day, but that won’t establish whether an LLM actually experiences anything. The right debate is this… Could machines exhibit something different from human consciousness, something measurable, meaningful, and worth studying? I think so, absolutely! I’m entirely open to discussing “machine consciousness” in terms of observable, testable computational properties. We built these systems, we can instrument them, examine their internal mechanisms, and investigate what they’re capable of. That seems far more productive than endlessly arguing over whether a machine experiences life the way humans do. So here’s my question: How would you define machine consciousness, and more importantly, how would you measure it?

Gregory M. Kurtzer @gmkurtzer

a week ago

Not sure who needs to hear this but.... An LLM doesn't "understand" anything. It is a statistical model that spews out tokens based on probabilities calculated through massive matrix operations and context, not an understanding of the subject matter.

1K 1K 9K 1.1M 2K

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com

i love writing. i never let agents write for me. whether it's docs, or tweets, or shitposts, or even prompts (okay maybe that one's harder as agents manage other agents) there's a simple reason for that: i still strongly believe that writing is thinking. it's a skill i hope to never give up. it's also important to me that whatever i say is always authentic, mistakes and all. my favorite posts are the ones that i just full send without any editing i suspect it also helps with prompting. being able to clearly express what i want, and knowing the vocabulary i need to be able to direct my agents does seem to make for more effective usage of agents. in the same way that engineers have an advantage when we prompt agents to write code, because we are intimately familiar with the words that make up the discipline knowing the right word incantations to get an agent to produce a desired goal almost feels like spellcasting. as models get more capable, and that input box looks more and more like a magic box that can output anything, words become the interface between you and the genie there may come a time when we give up the keyboard in exchange for brain-computer interfaces. but until then, i'm writing every word.