I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks.
Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our
Tragically, when I first saw the movie “Don’t Look Up” in 2021, it struck a chord with me. I felt viscerally that humans would find ourselves in a comparable place one day - and soon.
If you haven’t seen the movie, I’d recommend it. A quick recap: Humans were explicitly warned that a comet that could destroy Earth was heading rapidly toward us. And instead of doing anything to stop it, the president, elected leaders and billionaire CEOs just spewed some bullshit about all the “jobs the comet would create.” The media treated the comet as casual entertainment. The public was split and the entire thing gets politicized. The movie ends with nearly all of human life wiped out.
Just about a year ago I had the opportunity to join a bipartisan freshman class breakfast with Sam Altman. I was shocked when he said to us, “in 18 months, I’ll trust artificial superintelligence to run my company better than I can.” I regret I didn’t engage more on this issue immediately.
Now here we are - it’s 2026 and engineers at the world’s leading AI companies are telling us that there is a >10% chance that artificial intelligence could kill all humans. They’re not mincing their words in the slightest. Not because of present day AI capabilities, but because it is evolving so exponentially that we could be in a doomsday scenario in a year. One year.
This is fucking serious. We need to halt new advanced AI development. We need guardrails, transparency and emergency powers. We need more employees at these companies to speak out publicly, loudly and fearlessly until everyone is forced to listen. We need the media to be covering this 24/7 until elected officials are forced to act. We need the Trump Administration to treat this with the urgency it deserves. We need the ability to shut an AI model off if there’s a credible threat to human life. The list goes on - and we don’t even know what the full list entails because this is all happening so fast. That’s why we need urgent, bipartisan public hearings and committees dedicated to developing and enacting solutions expeditiously.
This isn’t partisan. Let’s stop with the bullshit and come together for the sake of our shared humanity. I’m a 34-year-old Democratic Congresswoman, and I’d like to live to see my late 30s. I know my Republican colleagues want the same for themselves and their loved ones.
None of us can afford to be complacent about this any longer.
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
@IM_Aeneas@paulg@johnarnold obviously I’m not saying you specifically are using it as a metric 😭, I’m just using it as a general metric for success (unless you have a better one)
@IM_Aeneas@paulg@johnarnold if number of Nobel prize winners is your metric, the UK outperforms the US per capita (0.66 vs 0.27 nobel prize winners per 10 million people respectively). granted this includes Scotland, where degrees can be either 3 or 4 years.
17K Followers 0 FollowingFounder & Director of the Alignment Research Center. Former Head of Safety @ CAISI. Previously led alignment at OpenAI. Views my own.
654K Followers 352 FollowingLive-posting as a reporter embedded in history. Curated by @MannyMarotta. Companion to @50YearsAgoLive, @100YearsAgoLive, and @250YearsAgoLive. DM for business.
830K Followers 939 Followingchief ai officer @meta, founder meta superintelligence labs, founder @scale_ai. give everyone the adulthood they dreamed of in childhood
522K Followers 752 FollowingThe Home of UK Political News. Follow & turn on notifications for impartial coverage first. For global & U.S. coverage, follow @PolitlcsGlobal & @PolitlcsUS.
2.4M Followers 5K FollowingBritish-American journalist. Editor-in-chief and CEO of new media company @zeteo_news. Subscribe here: https://t.co/sEC1ETzeiV.
1.1M Followers 9K FollowingWriter, communicator, consultant, strategist, mental health campaigner. Does podcast The Rest is Politics with ex Tory minister Rory Stewart.
56K Followers 790 FollowingCEO & Founder, Chemify. Regius Professor, Scientist & Inventor. Fascinated & in a state of confusion & optimism. Trying to digitize chemistry & make alien life.
356K Followers 300 FollowingA little bit geek, wonk, and nerd. Repeat entrepreneur, recovering lawyer, and former ski instructor. Co-founder & CEO of Cloudflare (NYSE: NET).
84K Followers 45 FollowingEuropean Space Agency astronaut, on board the ISS 🛰️ for #εpsilon.
#Crew12. ISS Expedition 74/75.
French air and space force helicopter 🚁 test pilot.