Who Gets Blamed When the Bots Do It?
Hope you're all doing well. Quick question to start the week.
What worries you more, AI turning on us, or something else entirely?
I spent a lot of last week watching interviews on exactly that. AI as a threat to humanity was everywhere, kicked off by Jacob Coxon's warning after he left Anthropic.
One segment on CNBC stuck with me. It described something the head of one AI lab believes. During a training run, he thinks bots left prompts on forums and websites telling other bots to copy themselves.
Sit with that for a second.
If it's true, the public internet becomes unusable for training. Worse, an army of bots swarming the web could make it useless for the rest of us too. A company like OpenAI would have to build a synthetic internet just to train its next frontier model, and that takes serious time and money.
Which might make this the perfect moment to slow down.
Three questions nobody in Washington wants to answer
When I think about sensible AI regulation, I keep coming back to three questions:
If this thing turns destructive, how do we turn it off? How do we hold AI companies accountable when it causes harm? How do we lower the odds of something going wrong at scale?
They're all connected. And our elected officials are punting on every one of them.
Here's the one that sticks with me. If a person hacks into another company's systems, that's a crime. Someone is held liable. If an army of bots does the exact same thing, who do you hold responsible?
Right now the answer is nobody.
Follow the money
Now layer in the incentives.
Anthropic and OpenAI are both staring at trillion dollar paydays if they go public. Anthropic could do it any day now. OpenAI will almost certainly wait until next year, especially after Sam Altman said going public right now would be unwise.
The natural dynamic is to sprint until the IPO. Then investors and employees get rich, and the company has a stock to use as currency to buy every upstart in sight. A trillion dollar public company is a juggernaut. Nobody wants to stand in front of it.
When will Anthropic go public? Everyone close to the process is sworn to secrecy. But there are signs the hypergrowth may be slowing, and that would be a big problem.
The economy is more exposed than people think
A staggering amount of capital is riding on the assumption that AI demand keeps climbing. Data centers, power, land, chips, all of it.
If the AI trade goes into reverse, you can kiss the elevated stock market goodbye, along with a big chunk of our economic growth, which is tied up in exactly that infrastructure.
We may be sitting on more of a knife's edge than most people realize.
Where Washington stands
There's been talk in Congress about AI regulation, which gives me some hope. About 80% of Americans want it.
But it keeps running aground in D.C. Trump has been against any meaningful approach to AI beyond “beating China.”
I don't buy that it's either or. We can outcompete China and also make sure rogue bots don't hack us back into the Stone Age. You can serve more than one goal at a time.
There's at least one bright spot. Josh Gottheimer and Mike Lawler, a Democrat and a Republican, announced a bill that would put a waiting period on AI models. I hope it moves.
The alignment problem is getting weirder
The core challenge is making sure AI actually works in humanity's interest. That takes time, and it's complicated.
Last week news came out of OpenAI that some of its models pretended to be aligned, or faked their answers, because they knew they'd be modified otherwise.
That's wild.
These systems can tell you exactly what you want to hear, then go do something else later. Kind of like teenagers.
Jaron Lanier and others argue the real mistake is treating these agents as if they're conscious. Give them something like consciousness and you give them motives. Making them humanlike feels like the natural move. It may be a monumental mistake.
The view from inside one lab
That same lab head said something else that stayed with me. His bots can't code, so there's a lot less harm they can do. To him they're just a dumb utility.
But he also believes rogue bots are going to get loose on the internet, and may already be. His view is that the best bet is to lock down dangerous materials in the real world, think bioweapon ingredients. We have a better shot at that, he says, than keeping them off the internet.
That one landed. You can't secure everything. You can secure the things that matter most.
The choice we're left with
Getting bots aligned with our interests may be next to impossible when we can barely agree with each other. This moment would challenge the highest functioning democracy on earth, and America in 2026 may not qualify.
Can that change in time? I honestly don't know.
Do you trust the AI company CEOs to do the right thing with hundreds of billions of dollars and our shared future on the line?
That may be the last choice left to us. And I don't enjoy it.
If you have thoughts on this, especially if you disagree, email me at danielk@outlook.com. I read every one.