NerdiMath @MathNerdi

@Ojakol_7 divar.ir/v/%DB%B8%DB%B2…

Great paper from Amazon AGI on generating RL environments for agents. (bookmark it) Also, pay attention to this important new AI engineering skill of creating RL environments. Seeing a huge shift towards this. The authors propose AutoGym which writes the task, the executable environment and the verifier together, starting from a small domain seed or past model trajectories. Paper: arxiv.org/abs/2609.22592 Chat with Paper: academy.dair.ai/papers/autogym…

“Recurrent Looped Transformer” This paper makes the decoder recurrent across every prompt and response token, while a causal encoder provides reusable global KV memory. Longer sequences then create deeper latent computation paths without adding more physical layers, while keeping the same state transition across pretraining, inference, and RL replay. alphaxiv.org/abs/2609.recur…

We are at the dawn of Superintelligence. Introducing the Recurrent Looped Transformer (RLT), We now have Transformers with Infinite Reasoning depth. From now on, we should pace progress at the Open Frontier of Superintelligence, Until Safe Superintelligence is achieved. github.com/yifanzhang-pro…

We’ve lost a pioneer. Prof. Dimitri Bertsekas (MIT) has passed away. Beyond his foundational mathematical contributions to dynamic programming and reinforcement learning, his textbooks shaped generations of researchers. Massive loss for AI

arxiv.org/abs/2510.04476 Compressed Convolutional Attention (CCA), a novel attention method which down-projects Q, k, v and performs the entire attention operation inside the shared latent space. This simple design dramatically cuts parameters, and speed up computation.

NYU’s new "Temporal Straightening" paper (May 2026) improves latent planning by forcing world models to learn flatter, more linear trajectories. ​​Result: More stable gradient-based planning & higher success rates ​Read: arxiv.org/abs/2603.12231 #AI #Robotics #MachineLearning

Transformer models are slow due to quadratic complexity. The paper proposes recurrent models with a vertically chunked inference method that makes memory usage constant regardless of sequence length. tinyurl.com/2wmhycz8

Deep Learning Interview preparation drive.google.com/file/d/1b7KjE3…

Stanford just made fine-tuning irrelevant with a single paper. It’s called Agentic Context Engineering (ACE) and it proves you can make models smarter without touching a single weight. Key takeaways (and get the 23 page PDF):

Google Research just dropped Titans & MIRAS—giving AI human-like long-term memory! 🧠✨ ​• Short-term: Attention for immediate context • Long-term: Neural net (MLP) to "learn" the past • "Surprise" Metric: Only memorizes what's actually important/new ​2M+ token context.

AI Agent Personas should simulate the structure of human reasoning. I’ve been arguing that you cannot "invent" a digital expert agent using just prompt engineering. You have to extract the expert via deep interviewing. A new NeurIPS paper, "Simulating Society Requires Simulating Thought" reinforces everything we've discussed about why thin, synthetic LLM personas fail. Most AI agents operate as "behaviorists." When you prompt an LLM to "act like a senior economist," it relies on surface-level correlations from training data. It generates text that sounds expert-like, but lacks any internal belief structure. 1. Logical Inconsistency: Without an internal model of how beliefs are formed, agents support a policy in one context but oppose it in another. The paper calls this "intervention-invariance mismatch" - beliefs don't update coherently when assumptions change. 2. Illusion of Consensus: In multi-agent simulations, LLMs converge toward the median view (even more positive emotions as the other paper mentions) of the training data. They agree not because of shared reasoning, but because their statistical priors push them toward the center. Your expert's contrarian, hard-won perspective gets averaged out. 3. Identity Flattening: LLMs reproduce stereotypical portrayals that erase intersectional variation. "The rich, positional knowledge of real-world stakeholders is replaced with monolithic, decontextualized simulations." To fix this, we have to move from simulating speech to simulating reasoning. The authors propose a "Cognitive Modeling" approach. "beyond output-level alignment toward aligning the internal reasoning traces of generative agents." Their solution is SEMI-STRUCTURED INTERVIEWS to extract what they call "cognitive motifs" - minimal causal reasoning units that capture how a specific person actually thinks. This is exactly why we built an interviewer system instead of a persona generator. You have to extract their actual belief structure through conversation. Instead of predicting the next word, the agent must possess "Reasoning Fidelity", a structured map of beliefs, causal logic, and cognitive motifs. How do you get this map? You can't prompt for it. You have to interview for it, with AI. The paper explicitly validates the architecture we’ve built: using semi-structured interviews to elicit "causal explanations" and "reasoning traces". This confirms why our Interviewer + Note-Taker multi-agent system is critical. - The Interviewer builds the "Peer Status" necessary to get the expert to open up. - The Note-Taker (the cognitive layer) extracts the "Cognitive Motifs", the distinctive logic blocks that define how that specific expert solves problems. We are moving beyond the era of "acting like an expert" to Generative Minds; agents that embody the positional individuality and causal logic of the people they represent. If you're building AI agents for strategy, decision-making, or stakeholder modelling, start by interviewing the human aspects of your agents.

Muratcan Koylan @muratcan

11 months ago

You should NOT use LLMs to generate synthetic human-like profiles. I just read the NeurIPS paper "LLM Generated Persona is a Promise with a Catch" and it confirms a suspicion we’ve held for a long time: You cannot "invent" a realistic human being using just statistics and an

56 73 545 103K 686

Everyone fears AI will kill jobs. Truth: AI + infinite virtual worlds = unlimited new jobs. Example: Today we have ~50k Fortnite builders/designers. In a fully AI-powered metaverse? Millions of people could get paid full-time to build, script, and run entire virtual civilizations

NeurIPS 2025 Best Paper Award: Attention lets language models decide which tokens matter at each position, but it has limitations—for example, a tendency to over-focus on early tokens regardless of their relevance. Gating mechanisms, which selectively suppress or amplify information flow in neural networks, have improved other architectures, so researchers have tried adding them to attention as well. However, prior attempts usually package gating together with other architectural changes, making its specific contribution hard to isolate. This paper separates those effects by systematically testing over 30 gating variants on dense models and mixture-of-experts models with up to 15 billion parameters. In a standard transformer layer, each attention head computes a weighted combination of values; the head outputs are concatenated and passed through a final linear projection. The winning approach identified in the paper inserts one extra operation before concatenation: each head's output is multiplied (element-wise or head-wise, with element-wise performing best) by a learned gate computed from the current token's representation. This allows each head to dampen or preserve its contribution depending on context. These architectural changes deliver practical benefits beyond small benchmark gains: 1. Training becomes more stable, supporting learning rates that cause baseline models to diverge. 2. The gating also greatly reduces "attention sinks"—the situation where early tokens absorb excessive attention—which in turn is associated with strong improvements on long-context benchmarks once the context window is extended using standard techniques. Talk to the paper on ChapterPal: chapterpal.com/s/c8685321/gat… Read the PDF: openreview.net/pdf?id=1b7whO4…

🚨 DeepSeek just did something wild. They built a math model that doesn’t just solve problems, it checks its own proofs, criticizes itself, fixes the logic, and tries again until it can’t find a single flaw. That final part is the breakthrough a model that can verify its own reasoning before you verify it. And the results are ridiculous: • Gold-level performance on IMO 2025 • Gold-level performance on CMO 2024 • 118/120 on Putnam 2024 near-perfect, beating every human score • Outperforms GPT-5 Thinking and Gemini 2.5 Pro on the hardest categories What makes DeepSeek Math V2 crazy isn’t accuracy, it’s the architecture behind it. They didn’t chase bigger models or longer chain-of-thought. They built an ecosystem: ✓ a dedicated verifier that hunts for logical gaps ✓ a meta-verifier that checks whether the verifier is hallucinating ✓ a proof generator that learns to fear bad reasoning ✓ and a training loop where the model keeps generating harder proofs that force the verifier to evolve The cycle is brutal: Generate → Verify → Meta-verify → Fix → Repeat. The core issue they solved: final-answer accuracy means nothing in theorem proving. You can get the right number with garbage logic. So they trained a verifier to judge the proof itself, not the final answer. The graph comparing proof accuracy across algebra, geometry, combinatorics, number theory, and inequalities shows DeepSeekMath-V2 beating both GPT-5 Thinking and Gemini 2.5 Pro across the board (this is on page 7). The wild part sequential self-refinement. With every iteration, the model’s own proof score climbs as it keeps debugging itself without human feedback. This isn’t “longer chain-of-thought.” This is “I will keep thinking until I’m sure I’m right.” A real shift in how we train reasoning models. DeepSeek basically proved that self-verifiable reasoning is possible in natural language no formal proof assistant, no human traces, just the model learning to distrust its own output. If you care about AI reasoning, this paper is a turning point.

x.com/i/spaces/1gqxv…

Matformer is a simple cool idea to train several models with the cost of just one: buy one get couple free!! huggingface.co/blog/rishiraj/…

Epistemic Uncertainty is vital for ML safety. ⚠️ ​It tells you when your model doesn't know the answer (out-of-distribution data). Ignore it, and you get wildly confident, wrong predictions. Know your model's limits! #ML #AI #Reliability

Mario Bunge on Causality This is a fascinating, profound book on causality as used in the sciences. I reread parts of it every now and then because it discusses ideas many philosophers and methodologists ignore. For information on the book: routledge.com/Causality-and-…

I’m recruiting PhD and Masters students at the University of British Columbia! Looking to work with students interested in LLM interpretability, and intersections with cog ling. The UBC NLP group is growing and awesome, and Vancouver is great. Come to 🇨🇦! isabelpapad.com/#joininglab

Isabel Papadimitriou @isabelpapad

2 years ago

Do you want to understand how language models work, and how they can change language science? I'm recruiting PhD students at UBC Linguistics! The research will be fun, and Vancouver is lovely. So much cool NLP happening at UBC across both Ling and CS! linguistics.ubc.ca/graduate/admis…

13 83 393 210K 133