A commercial kitchen and a food truck can cook the same dish. The difference shows up on the balance sheet. The kitchen runs multiple services a day, survives a decade of abuse, and swaps its menu when tastes change. The truck does one thing well until the thing stops selling. NVIDIA’s latest argument about AI factories is, at its core, an argument that compute infrastructure should be built like the kitchen — and the company has picked three words to make the case: productive, durable, fungible.
I find the framing more interesting than the usual throughput chest-thumping, because it moves the conversation from peak performance to unit economics over time. Those are different problems, and agent architects should care about the second one more than they currently do.
Productive means tokens per megawatt, not FLOPS per chip
The metric NVIDIA leads with is tokens per megawatt. That choice is telling. Power, not silicon, is the binding constraint in most serious deployments right now, so the denominator that matters is the one you cannot buy your way out of quickly.
The numbers NVIDIA puts behind this: Vera Rubin NVL72 delivers more than 30x higher throughput per megawatt than GB300 NVL72, and up to 45x lower cost per million tokens on DeepSeek V4 Pro. Vendor benchmarks always deserve a raised eyebrow, and the gap between those two figures — 30x throughput versus 45x cost reduction — suggests the cost number is doing work beyond raw speed, folding in whatever else improves when you move a workload onto newer hardware.
Still, the shape of the claim is worth taking seriously. For anyone running agent systems, cost per million tokens is the number that governs architecture. Multi-step reasoning loops, tool-calling chains, and self-critique passes all multiply token consumption against a single user request. An agent that reconsiders its plan four times is four times the inference bill. When per-token cost drops by an order of magnitude or more, designs that were financially absurd become merely expensive, and designs that were merely expensive become default.
Durable is the quietly radical claim
The second adjective is the one I keep returning to. NVIDIA points out that the A100, which shipped in 2020, remains in commercial service. CoreWeave has extended bookings through 2029. And every major operator has lengthened the depreciation schedule on its servers.
That last point is the strongest evidence in the whole argument, precisely because it does not come from a marketing deck. Depreciation schedules are accounting decisions that auditors examine and investors scrutinize. When operators extend them, they are making a formal, documented statement that the useful economic life of this hardware is longer than they previously assumed. You do not casually revise that number.
The implication cuts against a popular narrative — that AI hardware is a treadmill where last year’s accelerators become doorstops. Six-year-old A100s earning revenue in a market obsessed with the newest parts suggests something closer to the opposite: a tiered market where older silicon migrates down the value chain rather than off it. Frontier training moves to the newest systems, and the previous generation picks up inference, fine-tuning, batch work, and the long tail of smaller models.
Fungible is where agent workloads actually live
Broader workload support sounds like the least exciting of the three properties. For agent infrastructure it may be the most consequential.
Agent systems do not have one workload profile. A single production deployment might be simultaneously running:
- Large-context reasoning passes with heavy memory pressure and low batch efficiency
- High-volume, low-latency small-model calls for routing and classification
- Embedding generation and vector search against growing memory stores
- Periodic fine-tuning or distillation on accumulated interaction traces
- Retrieval, tool execution, and the orchestration glue around all of it
Those have genuinely different hardware preferences. Infrastructure optimized narrowly for one of them strands capacity when the mix shifts — and in agent systems the mix shifts constantly, because it depends on what users ask for and how your orchestration layer decides to answer. Hardware that handles the full range keeps utilization high regardless of which way the traffic leans. Utilization, not peak capability, is what determines whether the asset pays for itself.
What this means for people designing systems
Read the three adjectives as a prioritization hint rather than a product pitch. If NVIDIA’s customers are extending depreciation schedules and booking capacity years out, they are betting that today’s inference demand is structural rather than speculative. Agent architectures, with their appetite for many model calls per task, are a large part of why.
The practical takeaway: design for cost per completed task, not cost per token, and assume the hardware underneath you will still be there in five years. Both assumptions were shakier eighteen months ago than they are now. The second one, backed by accountants rather than benchmarks, is the one I would build on.
đź•’ Published: