Yihao Feng @yihaocs

High quality math is the secret sauce for reasoning models. The best math data is in old papers. But OCRing that math is full of insane edge cases. Let's talk about how to solve this, and how you can get better math data than many frontier labs 🧵

🚨 We’ve just published a recipe to train a frontier-level deep research agent using RL. With just 30 hours on an H200, any developer can now beat Sonnet-4 on DeepResearch Bench using open-source tools. (Thread 🧵)

New in-depth blog post - "Inside vLLM: Anatomy of a High-Throughput LLM Inference System". Probably the most in depth explanation of how LLM inference engines and vLLM in particular work! Took me a while to get this level of understanding of the codebase and then to write up…

Microsoft presents rStar2-Agent Agentic Reasoning Technical Report rStar2-Agent boosts a pre-trained 14B model to state of the art in only 510 RL steps within one week, achieving average pass@1 scores of 80.6% on AIME24 and 69.8% on AIME25, surpassing DeepSeek-R1 (671B) with…

Sir, we built this. A RL environment for learning reasoning at scale. GitHub: github.com/camel-ai/loong HF dataset: huggingface.co/datasets/camel… We extracted seed datasets from sources like textbooks, code libraries like sympy, networkX, Gurobi (math programming lib), rdkit…

Andrej Karpathy @karpathy

2 weeks ago

Sir, we built this. A RL environment for learning reasoning at scale. GitHub: github.com/camel-ai/loong HF dataset: huggingface.co/datasets/camel… We extracted seed datasets from sources like textbooks, code libraries like sympy, networkX, Gurobi (math programming lib), rdkit… https://t.co/yGb1jIGEz8

278 704 6K 580K 5K

Download Image

Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration "We dissect the popular GRPO algorithm and reveal a systematic bias: the cumulative-advantage disproportionately weights samples with medium accuracy, while down-weighting the low-accuracy…

We were able to reproduce the strong findings of the HRM paper on ARC-AGI-1. Further, we ran a series of ablation experiments to get to the bottom of what's behind it. Key findings: 1. The HRM model architecture itself (the centerpiece of the paper) is not an important factor.…

super cool to see this come together, incredible work spearheaded by @brendanh0gan, all-in-all an incredibly detailed recipe of what it takes to craft a specialist model for OOD tasks where frontier models really struggle paper/weights/data/code in brendan’s thread :)

Brendan Hogan @brendanh0gan

4 weeks ago

super cool to see this come together, incredible work spearheaded by @brendanh0gan, all-in-all an incredibly detailed recipe of what it takes to craft a specialist model for OOD tasks where frontier models really struggle paper/weights/data/code in brendan’s thread :)

keyboard_arrow_left Previous keyboard_arrow_right Next

19 92 720 120K 673

Download Image

A neat observation: Rejection sampling during GRPO allows you to directly factor in properties in your reward, and allows you to go from optimizing max_model Expected Reward(response) to max_model E {Reward(response) * Property(response)} iIn our case it's "small length", but…

Dimitris Papailiopoulos @DimitrisPapail

4 weeks ago

A neat observation: Rejection sampling during GRPO allows you to directly factor in properties in your reward, and allows you to go from optimizing max_model Expected Reward(response) to max_model E {Reward(response) * Property(response)} iIn our case it's "small length", but…

keyboard_arrow_left Previous keyboard_arrow_right Next

19 41 361 94K 273

Download Image

moving from vllm v0 to v1 made our async rl training crash! read how we fixed it we recently migrated from v0 to v1 as part of a larger refactor of prime-rl to make it easier-to-use, more performant and naturally async. we confirmed correct training dynamics on many…

Nice empirical paper investigating all your bag of tricks in reasoning LLMs arxiv.org/abs/2508.08221

🥳Thrilled to introduce SWE-Swiss! 🚀Our 32B model achieves 60.2% on SWE-bench, matching the performance of much larger models (DeepSeek-R1-0528, Kimi-dev-72B). Better methods, not just bigger models! 📑Notion: pebble-potato-fc6.notion.site/SWE-Swiss-A-Mu… 💻Github: github.com/zhenyuhe00/SWE…

🚀 Introducing XBai o4：a milestone in our 4th-generation open-source technology based on parallel test time scaling！ In its medium mode, XBai o4 now fully outperforms OpenAI−o3−mini.📈 🔗Open-source weights: huggingface.co/MetaStoneTec/X…✅ Github link: github.com/MetaStone-AI/X…

keyboard_arrow_left Previous keyboard_arrow_right Next

Everything about Llama-Nemotron-Super-V1.5 post-training is now open: Synthetic data: huggingface.co/datasets/nvidi… Human data: huggingface.co/datasets/nvidi… Reward models (trained on HS3 data): huggingface.co/collections/nv… RL toolkit: github.com/NVIDIA-NeMo/RL

All of the synthetic pre-training data pipelines results look extremely promising! In hindsight, it makes sense. It not only fixes low-quality web data but also works well on high-quality data 📚. Finally, we have a way to augment text data without making them too predictable.

LLMs can now self-optimize. A new method allows an AI to rewrite its own prompts to achieve up to 35x greater efficiency, outperforming both Reinforcement Learning and Fine-Tuning for complex reasoning. UC Berkeley, Stanford, and Databricks introduce a new method called GEPA…

The SkyRL roadmap is live! Our focus is on building the easiest-to-use high-performance RL framework for agents. We'd love your ideas, feedback, or code to guide the project: github.com/NovaSky-AI/Sky…

new blog: How to scale RL to 10^26 FLOPs everyone is trying to figure out the right way to scale reasoning with RL ilya compared the Internet to fossil fuel: it may be the only useful data we have. and it's expendable perhaps we should learn to reason from The Internet (not…

keyboard_arrow_left Previous keyboard_arrow_right Next

🤯 Your model might be forgetting how to reason during post-training. In RL training (e.g., Deepseek-R1-1.5B), 76.7% of AIME problems were solved at some point — but only 30% survived in the final model. We call this Temporal Forgetting. 🔧 Simple Fix: Temporal Sampling —…

Yuetai Li @yuetai12575

3 months ago

🤯 Your model might be forgetting how to reason during post-training. In RL training (e.g., Deepseek-R1-1.5B), 76.7% of AIME problems were solved at some point — but only 30% survived in the final model. We call this Temporal Forgetting. 🔧 Simple Fix: Temporal Sampling —… https://t.co/kcj6OXHuoB

4 19 109 25K 83

Download Image

let’s gooo - ByteDance dropped SeedCoder 8B (Instruct & Reasoning) - Beats Qwen Coder AND DeepSeek Coder - MIT Licensed 🔥 Works out of the box with Transformers, llama.cpp and vLLM 🤗