A slowdown at the frontier would barely change the rate at which dangerous capability reaches the world. That is the uncomfortable part of the current conversation, and almost nobody advocating for prudence has addressed it.
The proposal itself is sincere. Dario Amodei has argued for slowing AI development to reduce risk, with a plan that leans on industry commitments, global regulation, and third-party monitoring. In an essay he wrote that over the last few months he had become convinced that fully addressing the risks requires even more prudence. Sam Altman has voiced a similar instinct, suggesting it may be time to pace development. Even competing labs have signaled agreement with the idea of outside monitors evaluating models as they are built. For an industry that agrees on very little, that is real consensus.
My objection is not to the intent. It is to the unit of measurement.
Frontier is not a scalar
When lab leaders talk about pacing, the implied object being paced is the training run. Bigger model, more compute, more capability, therefore slow the curve and buy time. That mental model made sense when a model’s behavior was mostly determined at pretraining and everything downstream was a thin wrapper.
That is not how systems are built now. The capability that actually touches users comes out of an assembly: a base model, a scaffold that plans and retries, tool access, persistent memory, retrieval over private data, and increasing amounts of inference-time compute spent per request. Hold the weights fixed and you can still move the effective frontier a long way by changing any of those. A model that fails a task at one pass can succeed with a verifier loop and a hundred attempts. A model with no ability to cause harm in a chat box acquires that ability the moment it holds a credential and a shell.
So pacing the training curve while leaving the composition layer untouched slows a number in a lab report, not the arrival of capable autonomous systems. Agent architecture is the cheapest capability multiplier available, it is not compute-bound in any way regulators can meter, and it is being iterated on by everyone with a laptop.
What a monitor would actually need to see
Third-party monitoring is the most concrete piece of the proposal and the easiest to under-specify. If the mandate is static evaluation of model checkpoints, monitors will produce clean reports about artifacts that are not the thing being deployed.
A monitoring regime with teeth would need visibility into the operating system around the model:
- Autonomy budgets. How many steps can an agent take without human confirmation, and what is the blast radius of a step? This is a configuration value, and today it is set by whoever ships the product.
- Tool and permission surfaces. Which credentials, filesystems, payment rails, and outbound network paths are reachable. Capability is largely a function of privilege, and privilege is auditable in a way that intelligence is not.
- Execution traces. Not just final outputs but the plan, the intermediate tool calls, and the recovery behavior after failure. Most interesting misbehavior lives in step forty of a loop, invisible to benchmark scoring.
- Inference-time compute per task. If ten thousand sampled attempts plus a verifier reliably clears a task the model fails once, then spend is a capability dial and should be reported like one.
- Multi-agent composition. Several mediocre agents coordinating through a shared scratchpad are not well described by any single-model evaluation.
None of this requires new science. It requires the labs to treat deployment configuration as a regulated artifact rather than a product decision, and that is a much harder ask than agreeing to be evaluated.
Who holds the stopwatch
There is also the governance problem that critics have already pointed at, and it is fair. The people proposing the pace are the people setting it. A frontier lab advocating for slowing its own competitors, through rules written with its own understanding of what matters, is not automatically acting in bad faith. It is, however, an arrangement where the definition of frontier conveniently matches the thing that lab is best at measuring: large training runs.
Technical specificity is the check on that. Vague prudence is unfalsifiable and flatters whoever wrote it. A rule that says agents above a stated autonomy budget with access to stated privilege classes must ship traces to an outside auditor can be complied with, violated, or shown to be insufficient. That is what a real constraint looks like.
The version of this I would sign
Pace the deployment surface, not the pretraining curve. Meter privilege, autonomy, and inference spend, because those are the levers that turn a model into an actor. Require trace-level accountability for systems operating without a human in the loop. Treat scaffolding as part of the system under evaluation, since that is where capability is being added fastest and reviewed least.
The instinct toward prudence among lab leaders is worth taking seriously. But prudence aimed at the wrong variable is not caution. It is a slower clock on a race that has already moved to a different track.
🕒 Published: