Remember when spotting machine-written text was a party trick? You looked for the tell-tale tics — the throat-clearing intro, the tidy tricolon, the paragraph that ended by summarizing itself. You could do it over coffee, and you were usually right. That era is over, and its passing has quietly broken the way most people still think about AI detection.
Pangram’s Max Spero has been making a point that deserves more attention from anyone building or evaluating these systems: detection is not a “real or fake” question. Framing it that way is the source of most of the bad reasoning around it, including the bad reasoning inside the tools themselves.
Binary output, non-binary reality
A classifier that returns two labels invites you to treat text as having a single hidden property, like a coin with a machine side and a human side. But authorship is not one property. Text can be drafted by a person and smoothed by a model. It can be model-generated and then rewritten by hand. It can be written by someone who has read so much model output that their prose has drifted toward it. Every one of those cases lands somewhere in the interior of the distribution, and a binary label has nowhere to put them.
This is why false positives are the interesting engineering problem, not accuracy in aggregate. Spero’s framing centers on reducing them, and Pangram’s positioning against tools like Turnitin leans on false positive rates and ESL bias rather than a single headline accuracy number. That is the correct place to compete. A detector at high overall accuracy can still be socially useless if its errors cluster on non-native English writers, technical writers, or anyone whose style happens to be regular. Aggregate accuracy hides the distribution of harm; per-population false positive rates expose it.
Why the problem gets harder as models improve
Spero has noted that most detection tools can still catch copy run through “humanizer” services. That makes sense architecturally — those tools apply shallow perturbations to surface statistics, and shallow perturbations leave their own signature. The harder trend is different: as large language models train on and pull from a wider body of research and writing, the space of text they can produce keeps expanding into territory that humans also occupy. Detection difficulty scales with overlap, and overlap grows with capability.
On a recent conversation about how detection actually works, the host, Patrick, described the situation in a way I keep returning to — that model weights create a Schelling point with “its own gravity,” and a huge amount of it. That is the good-news-bad-news structure of this whole field. The gravity is what makes detection possible at all: widely deployed models converge on characteristic regions of output space, and a classifier can learn those regions. The same gravity is what makes detection fragile, because that convergence point moves every time a major model ships, and it pulls human writing toward it too.
What detectors can and cannot certify
Pangram’s own technical report makes a distinction that I wish every downstream user of these tools would internalize: AI detection is not a substitute for, and not a reliable way of, proving whether text is factually true. AI is frequently used for disinformation and scams, so the correlation is real and worth acting on. But the classifier is measuring provenance, not veracity. A machine-written sentence can be accurate. A human-written one can be a fabrication. Conflating the two turns a provenance signal into a truth oracle, which is exactly the failure mode that gets students falsely accused and gets real misinformation waved through because it reads as human.
Spero’s argument for why this work matters anyway rests on misinformation, and I think that holds — provenance at scale is genuinely useful context. AI-generated content is now everywhere on social platforms, LinkedIn especially, and knowing the likely origin of a piece of text changes how much scrutiny it warrants. That is a triage function. Triage is valuable. Triage is not adjudication.
The architectural takeaway
If you are designing systems that consume detection output, treat it as one signal in a pipeline with a known error profile, not as a verdict. Practically, that means a few things:
- Surface calibrated scores and per-population error rates, not just a label.
- Set thresholds according to the cost of a false accusation in your specific context, which is usually much higher than the cost of a miss.
- Assume model drift and plan for re-evaluation rather than treating a detector as a fixed component.
- Keep a human in the loop for any decision with consequences attached.
That last one is not a hedge. Human verification stays necessary precisely because the underlying question — who wrote this, and with what help — is not the kind of question a binary classifier was ever built to answer. The better tools get, the more clearly that limit shows.
🕒 Published: