Law firms are buying GPUs now.
That sentence would have read as a category error two years ago. Yet Latham & Watkins has purchased Nvidia servers to stand up in-house AI systems, with the stated goal of greater flexibility and independence in how it develops AI. The firm’s framing, per Law.com, is blunt: “Flexibility is critical.” Its infrastructure strategy is meant to give it more room in how it develops and tests.
I want to look past the novelty of a partnership buying rack space and ask the question that matters for anyone building agent systems: what does owning the metal actually change about the architecture?
Data gravity is the real constraint
Most discussions of self-hosted AI fixate on cost per token. That is the least interesting variable. The binding constraint for a firm like this is data gravity — the fact that the corpus an agent needs to be useful is enormous, sensitive, and immovable.
A legal agent worth deploying does not answer general questions. It reads matter files, deal precedents, internal memos, redlines, privileged correspondence. That corpus does not want to travel. Every architecture decision downstream of it inherits that constraint: where embeddings get computed, where indices live, where retrieval happens, where the context window gets assembled, where intermediate reasoning traces land.
When inference is remote, you end up designing around the boundary. You build redaction layers, tokenization proxies, scope-limited retrieval, contractual controls over training use. Those layers are engineering debt that exists purely because the compute is somewhere else. Move inference next to the data and a whole class of that machinery collapses. That is the flexibility argument, stated in architectural terms rather than procurement terms.
Agents need state, and state needs a home
Here is where I think the in-house move gets genuinely interesting for agent design.
A single-shot completion is stateless and easy to broker out. An agent is not. An agent accumulates working memory across a long task, calls tools, spawns subtasks, revises earlier conclusions, and produces an audit trail that is itself a sensitive artifact. The reasoning trace of a legal agent working through a diligence question may be more revealing than its final output.
That changes the hosting calculus:
- Trace retention. If intermediate reasoning is privileged material, it needs to live inside the trust boundary by default, not by exception.
- Tool privilege. Agents that touch document management, billing, or email need tightly scoped credentials. Co-locating the model with the tool layer shortens the privilege chain considerably.
- Latency compounding. A multi-step agent multiplies round trips. Twenty tool-and-model hops turn a tolerable per-call latency into an intolerable task latency.
- Model pinning. Hosted endpoints change underneath you. An agent tuned against specific model behavior is fragile to silent updates. Owning the weights and the hardware means you decide when behavior shifts.
That last point is underrated. Agent reliability work is largely the work of characterizing a model’s failure modes and building scaffolding around them. Every model update invalidates part of that scaffolding. Version control over the model is version control over your evaluation results.
What buying servers does not solve
I would push back on any reading of this as a shortcut. Owning Nvidia hardware buys you capacity, not capability. The hard parts remain hard:
Evaluation
Legal work has no clean reward signal. Correctness is contested, context-dependent, and often only knowable after the fact. Whoever runs these systems still has to build domain evaluation harnesses, and no amount of compute substitutes for that.
Utilization
Owned GPUs have a fixed cost whether they run at 90 percent or 9 percent. Cloud inference converts capital risk into variable spend. Buying hardware moves that risk onto your own balance sheet, which only pays off with sustained, predictable load — fine-tuning runs, batch document processing, continuous evaluation sweeps.
Operations
Serving infrastructure is a specialty. Scheduling, memory management, quantization tradeoffs, throughput tuning, failover. These are real disciplines, and a firm taking them on is taking on a small platform team’s worth of ongoing work.
A signal about who builds next
There is a fitting detail in the background. Latham represented Nvidia on an AI compute hosting and guarantee transaction for the PORTS-Pike Technology Campus in Ohio, a development described as up to 10 GW and among the largest single data center complexes planned. The firm has been sitting close to the plumbing of the AI buildout while deciding to install some of its own.
I read this as an early instance of a broader pattern. As agent systems move from demos to production inside data-sensitive organizations, the decision about where inference runs stops being an IT procurement question and becomes an architectural one. Firms that treat it that way will build agents that can actually reach their most valuable data. Firms that do not will keep shipping assistants that answer questions about documents they are not allowed to read.
🕒 Published: