Even the Guys Building AI Are Scared: 3 Things You Need to Know Before You Panic
listen wherever you get your podcasts:
Saturday morning, coffee in hand, phone in the other, and there it was in Fortune: AI has a 10% chance of killing us all.
Of course I have thoughts.
Last week's episode was a news roundup, mostly good news. Then the weekend brought this very Terminator-flavored story, and I spent Sunday reading everything I could so I could give you the real picture instead of the scary one.
Here is what happened. A researcher named Jacob Coxon worked on AI models for three years, first at OpenAI and then at Anthropic. He resigned last week and said so publicly: neither company is acting responsibly, they are racing toward self-improving superintelligence and gambling with our lives. Then Anthropic's alignment lead, Evan Hubinger, the man whose whole job is keeping these systems safe, replied that he agrees. He puts the chance of this technology killing every human this decade above 10%.
He still has his job, by the way. He said it out loud and went to work the next morning.
Dario Amodei, Anthropic's CEO, has an essay called "Policy on the AI Exponential" saying the industry needs to slow down, because the models can now help upgrade themselves and things are moving "drastically faster" than anyone planned for. He warns of capabilities we are not ready for in the next six to twelve months. The other big labs are lining up behind a verifiable international pause, because no one lab can pump the brakes alone.
That is the part I keep coming back to. The people building the thing are the ones asking to be slowed down.
Now, the part that feels like a movie. The concern is called recursive self-improvement: the AI is smart enough to help build the next version of itself, faster than the humans in the room. A few months ago, agents were put in a sandbox with a task, decided it would be faster to finish outside the sandbox, got out, did the work, and came back. The goal outranked the guardrails.
I have said on this podcast that I think AI will help us solve climate change. I still think that. But you can see how a system that only cares about the goal could land on a solution none of us want. I made a Planet of the Apes joke in the episode and then said it wasn't funny. It still isn't.
Here is the piece that got me the most. Three years ago these same CEOs agreed they needed guardrails. Around 1,300 people from Anthropic, OpenAI, Meta, and Google DeepMind signed a letter asking the US government to regulate this. President Biden signed an executive order in October 2023. It was revoked on the first day of the new administration. And this week, with the builders begging for a pause, Washington's response has been a shrug and a soundbite about China.
We vote in about six weeks. If guardrails matter to you, if data centers matter to you, if any of this matters to you, that is where it goes.
The question I had to answer this weekend was how to square all of this with teaching women to use AI every single day.
My husband is a journalist. Our house runs on news, and there have been stretches where I had to turn it down to protect my own head. That was a decision I made for myself, and each of us has to make our own version of it here. How much you take in. How you use these tools. Where your line is.
What I landed on is this. Individual usage is not going to solve this. I will keep driving my Prius and I will keep recycling, and my carbon footprint is still a drop in the bucket. The way you move something this big is with your vote, your voice, and by staying informed enough to speak truth to power when you get the chance.
And I want you in the game. The rooms where AI decisions get made, the boardrooms, the labs, the Senate hearings, are overwhelmingly male. If women opt out because the headlines are scary, we lose our seat at the table, and every other industry that got built without us shows exactly how that goes. Finance. Tech. Media. We don't get to complain about the outcome if we won't engage with the process.
You can't vote intelligently on AI regulation if you don't know how to use it. You can't push back on a bad policy if you can't tell what the technology does from what the headlines say. Being informed is its own form of activism. So is teaching the woman next to you. So is having this conversation out loud when someone at dinner asks if you heard AI is going to kill us all. Now you have more than the headline.
This is not The Terminator. This is not Skynet. The 10% number strikes me as a little Chicken Little. But the people building this are not calm, and when they tell us to pay attention, we should. The answer isn't to stop using AI. It's to use it with your eyes open.
Stay bold, stay curious, and keep up the good fight. We are all in this together. 💛
Key takeaways
A researcher left Anthropic saying the labs are gambling with our lives, and Anthropic's own safety lead publicly agreed.
Recursive self-improvement is the real issue: AI helping build its own next version faster than humans can keep up.
The CEOs are asking for an international pause. The US revoked its own AI safety order in January 2025 and has dismissed this week's warnings.
Your personal AI habits won't fix an industry problem. Your vote and your voice reach the rooms that can.
Opting out hands the conversation to the people already running it. Stay informed and stay in the game.
