Nvidia says the Groq 3 LPX has entered full production. Reporting from April 2026 also describes the same chip as sitting in early-access preview, with broad cloud availability expected late in the year. Both statements are true, and the gap between them is the most interesting thing about this launch.
Full production means silicon is flowing. Availability means racks are installed, cooled, networked, and exposed behind an API you can actually call. For an accelerator that ships as part of the Groq LPX rack inside the Vera Rubin platform, those two milestones can be six months apart without anyone lying. The chip is done. The deployment is not.
Why an inference-only chip now
The Groq 3 LPX came out of Nvidia’s $20 billion Groq acquisition, and Jensen Huang introduced it at GTC 2026 as an inference-specific part that works in concert with the Rubin GPU. Read that pairing carefully, because it is an architectural admission. Nvidia has spent a decade arguing that one family of GPUs can serve training and inference well enough. Buying a company whose entire thesis was a non-GPU inference processor, then shipping that processor as a companion rather than a replacement, is a concession that the two workloads have drifted apart far enough to justify separate silicon.
The stated design target is low-latency interaction with large models, and the headline number is 3,400 tokens per second. That figure matters less as a throughput brag than as a latency statement. Throughput-oriented accelerators get their impressive aggregate numbers from batching: pile up requests, run them together, amortize the weight loads. Batching is excellent for a document-summarization queue and actively hostile to anything that has to respond in a conversational loop.
The part that matters for agents
Agent systems are where this distinction stops being academic. A single-shot chat completion hides latency behind the user’s reading speed. An agent does not read. It plans, calls a tool, reads the result, revises the plan, calls another tool, and repeats. Each of those steps is a separate forward pass, and each one blocks the next.
That structure has three consequences that per-token speed hits directly:
- Latency compounds multiplicatively, not additively. A twelve-step agent trajectory pays the decode cost twelve times. Shaving per-token time does not improve one response, it improves every link in a serial chain.
- Batching works against you. The requests in an agent loop are causally dependent. Step four cannot be batched with step five, because step five does not exist yet. Hardware whose efficiency depends on wide batches has less to offer here than its spec sheet implies.
- Architectural choices are latency-gated. Tree search over candidate plans, self-critique passes, multi-agent debate, and speculative replanning are all known to improve reliability. Most teams strip them out because the wall-clock cost is unacceptable, not because they do not work.
That last point is the one I would watch. When inference gets meaningfully faster per token, the interesting outcome is not that existing agents feel snappier. It is that designs currently sitting in papers become deployable. Verification loops you skipped, ensembles you collapsed to a single call, deliberation steps you cut to hit a response budget — those come back onto the table. Faster inference does not just accelerate current agent architectures, it changes which ones are economically sane to build.
What the split fabric implies for system design
A platform where an inference part and a training-class GPU cooperate pushes complexity up the stack. Someone now has to decide which model runs where, how state moves between them, and what happens when an agent’s tool call needs a large model that lives on different silicon than the one driving its interactive loop. Routing between a Rubin GPU and an LPX rack is a scheduling problem, and scheduling problems are where theoretical speedups go to die. The chips are the easy part. The orchestration layer that decides what runs on which device, with what memory residency, is where real deployments will win or lose their gains.
What I am not claiming
Nvidia has not published enough for anyone outside its early-access program to characterize this chip properly. We do not know what model size or precision that 3,400 tokens per second describes, what the concurrency behavior looks like under realistic mixed traffic, how memory capacity constrains context length, or what the power and cost picture is per rack. Those numbers determine whether this is a genuine shift in agent design economics or a well-engineered speed bump for one narrow workload profile.
So treat the production announcement as what it is: a supply-chain fact, not a capability you can build against today. The chip exists. Late 2026 is when we find out what it actually does for the systems we are designing now.
🕒 Published: