Excited to see Orthogonal Finetuning (OFT) and Quantized OFT (QOFT) now merged into LLaMA-Factory! 🎉
OFT & QOFT are memory/time/parameter-efficient and excel at preserving pretraining knowledge. Try them in:
🔗 LLaMA-Factory: github.com/hiyouga/LLaMA-…
🔗 PEFT:…
Falcon-H1 technical report is now available! The latest open hybrid Transformer–Mamba model family.
The 80+ page report details the key design decisions behind H1, from architecture innovations, data strategies to training recipes challenging conventional practices in the filed
New tech report out! 🚀
Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
An expanded version of our ProRL paper — now with more training insights and experimental details.
Read it here 👉 arxiv.org/abs/2507.12507
New tech report out! 🚀
Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
An expanded version of our ProRL paper — now with more training insights and experimental details.
Read it here 👉 arxiv.org/abs/2507.12507
📢📢📢 Releasing OpenThinker3-1.5B, the top-performing SFT-only model at the 1B scale! 🚀
OpenThinker3-1.5B is a smaller version of our previous 7B model, trained on the same OpenThoughts3-1.2M dataset.
Introduce Easy Dataset
No-code framework for synthesizing fine-tuning data from unstructured documents using LLMs/Ollamas
Supports OCR, chunking, QA augmentation, and export to LlamaFactory/Unsloth fine-tuning frameworks
huggingface.co/papers/2507.04…
PPO and GRPO — a workflow breakdown of the most popular reinforcement learning algorithms
➡️ Proximal Policy Optimization (PPO): The Stable Learner
It’s used everywhere from dialogue agents to instruction tuning as it balances between learning fast and staying safe.
▪️ How PPO…
Fine-tune Llama-3.1 8B with Llama-Factory on AMD GPUs with this step-by-step guide: bit.ly/4k14ORL
Discover more fine-tuning tutorials on the ROCm AI Developer Hub: bit.ly/4kLQiOQ
DeepSeek 671b and Qwen3 236b support with Megatron backend is now available as preview in verl v0.4.0 🔥🔥🔥
We will continue optimizing MoE model performance down the road.
DeepSeek 671b: verl.readthedocs.io/en/latest/perf…
verl v0.4: github.com/volcengine/ver…
469 Followers 597 FollowingPhD student at @Princeton University, focusing on LLMs. Language Modeling and Pretraining, LLM Reasoning and RL. Prev @UCLA, @Tsinghua_IIIS
106 Followers 955 FollowingFind statistics in Stocks, Blockchain, Tech.
Posting numbers I found while doing deep research for my own investments.
Find the data. Get the signal.
1K Followers 579 FollowingCS PhD Student @Berkeley_AI and @BerkeleySky. Prev. MS @Princeton_NLP, BS @HDSIUCSD and @CogSciUCSD; '25 @SiebelScholars; I work on multimodal models; He/Him.
535 Followers 2K Following手艺人Build&Sell
👩💻https://t.co/3BQxr054GI
🎵https://t.co/V64qINIM9D
🌍https://t.co/ldnu88Yksc
Run, don't walk, if you don't jump, your perish will never end
心存善念,每个人都在打一场人生硬仗
115 Followers 460 FollowingFather. Founder and CEO. ASD Advocate. Distincto: AI-First Apps from an AI-First Company, Delivering Big for the Global Neurodiverse Community
11K Followers 979 FollowingJan is the open-source ChatGPT replacement. We're building Open Superintelligence together. Community: https://t.co/NIyIbR60qQ
356 Followers 37 FollowingEfficient Systems for Foundation Models Workshop, ICML2025.
Join us if you are interested in the challenges associated with large models training & inference!
19K Followers 20 FollowingA high-throughput and memory-efficient inference and serving engine for LLMs. Join https://t.co/lxJ0SfX5pJ to discuss together with the community!
47K Followers 106 FollowingAdvancing AI innovation together. Built with devs, for devs. Supported through an open ecosystem. Powered by AMD.
#TogetherWeAdvance
14K Followers 3K Followingresearch @MIT_CSAIL @thinkymachines. work on scalable and principled algorithms in #LLM and #MLSys. in open-sourcing I trust 🐳. she/her/hers
2K Followers 56 FollowingAxolotl is the premier open source LLM fine tuning framework. find us on discord https://t.co/wlcE2wlJa9 or email us at [email protected]
1K Followers 7K FollowingAI inference, speculative decoding, open source. Built novel decoding algorithms – default in Hugging Face Transformers (150+ ⭐). Making AI faster + cheaper
1K Followers 103 FollowingAI/RL researcher, Assistant Prof. at @Tsinghua_Uni, leading the RL lab at @AntResearch_, PhD at @berkeley_ai, frequent flyer and milk tea lover.