The AI Race Has No Brake Pedal

I’d like to think I’m a little like Alfred E. Newman, the chill “What, me worry?” character from Mad Magazine. But I’m not. I do worry.

I watched a conversation with Noam Brown, an OpenAI researcher who helped develop its reasoning models. His message was concerning: AI capabilities are advancing faster than our ability to evaluate, monitor and contain them. Brown explained that a model can look safe on the evaluations we designed and still behave differently in deployment. Passing an evaluation does not prove the model is safe. It only proves the model passed the scenarios we thought to test. That is the AI version of a beautiful backtest.

The problem gets worse as models operate autonomously for longer periods. Brown said that if a model can complete three-month tasks but the next model arrives in two months, researchers cannot evaluate it across its capabilities before development moves on. Monitoring may also be weakening. OpenAI can sometimes inspect a reasoning model’s written “chain of thought” for signs of deception or scheming. But Brown said models are gaining more control over what appears in that reasoning. Punishing undesirable thoughts may simply teach a model to hide them. Now add RSI—recursive self-improvement: AI systems helping design and improve the next generation of AI. Yah, basically taking the human out of the loop. And everything speeds up.

Brown is not predicting an overnight 100-times intelligence explosion. He frames the likely acceleration closer to three times. But even that would compress a three-month safety window into one month while regulation, oversight and corporate governance remain stuck on calendar time. And nobody is going to slow down. OpenAI cannot assume Google will stop. Google cannot assume Anthropic, xAI or China will stop. No major player wants to be the only one standing on the brake while everyone else races toward the most valuable technology in history. That is why the open letters, summits and voluntary promises always looked like theater to me. The researchers may be sincere, but the incentives are stronger than the promises. You do not need to believe in a robot apocalypse to see the risk.

Poorly evaluated systems can still create enormous damage through cyberattacks, financial errors, biological misuse, military escalation or critical decisions made beyond meaningful human supervision. I remain structurally bullish on AI. It may become the greatest productivity engine in history. But bullish does not mean blind. We are increasing the leverage and shortening the feedback loop while the risk model is calibrated to the previous generation. That does not guarantee disaster—but it is exactly how tail risk gets mispriced.