Writing-process evidence vs AI detection: why one is forensic-grade and the other isn't

The short answer

An AI detector examines finished text and makes a probabilistic inference: does this look like it came from a machine? A writing-process record observes the composition event directly: here is what actually happened while this document was written. That is not a difference in accuracy — it is a difference in evidence type. One is circumstantial, one is observational. Courts, hearing panels, and risk-management frameworks treat these categories differently, and for good reason.

What AI detection actually does

Strip an AI detector down to its mechanism and you find a statistical classifier. It reads the completed text and asks whether the pattern of word choices, sentence lengths, and phrasing rhythms resembles the "human" examples or the "AI" examples in its training data. The two most-discussed signals — perplexity (how predictable each word choice is to a language model) and burstiness (how much predictability varies) — measure properties of the finished artifact, not of how it was made.

Weber-Wulff et al. (2023) independently tested twelve detection tools across multiple text categories. Two findings are especially important. First, paraphrasing collapsed detection accuracy dramatically: GPT-3 text paraphrased through SpinBot was detected at roughly 35% by most tools — near chance — because the surface patterns that classifiers rely on were disrupted even though the underlying AI origin was unchanged. Second, the tools designed for academic AI text failed on non-academic genres, showing that performance is domain-specific rather than general. The inter-tool disagreement compounds this: when two tools applied to the same document produce scores that differ by more than 50 percentage points, they cannot both be measuring something stable. That disagreement is not a calibration problem to be fixed — it is the structural limit of inferring a process from its output. The same surface text can, in principle, be produced multiple ways.

Turnitin, the most widely deployed tool in education, makes this caveat explicit in its own published guidance: the score "should not be used as the sole basis for adverse actions against a student" and "Turnitin does not make a determination of misconduct." The vendor frames its score as an input to human judgment. When it is treated as a verdict, that use contradicts the manufacturer's instructions.

What "AI probability" means: an AI detection score is the model's confidence that the finished text statistically resembles AI-generated text. It is not a statement about how the text was produced. A highly-edited AI draft and a carefully-written human essay can land on identical scores. The number describes the artifact; it says nothing direct about the event.

What writing-process evidence does

Writing-process evidence takes a different approach entirely: rather than inferring backward from the artifact, it records the composition event as it occurs.

The most rigorous work in this area comes from keystroke logging research in educational data mining. Crossley, Tian, Choi, Holmes, and Morris (EDM 2024) analyzed keystroke logs from student writing sessions and reported that a classifier built on process features — timing between keystrokes, revision patterns, insertion and deletion sequences — achieved 99% accuracy in distinguishing human-composed writing from transcribed (copy-pasted) text. The signal is not in the finished words; it is in the temporal and behavioral signature of how those words arrived.

Trezise, Ryan, de Barba, and Kennedy (2019) demonstrated in a large-scale learning analytics study that behavioral engagement data captured during writing — not the text itself — could detect anomalous submissions that surface-level analysis missed. The process data provided evidence that the artifact could not.

These results share a common logic: writing is an action, not just an artifact. The action has a timestamp, a tempo, a pattern of incremental change. Recording the action is observationally different from analyzing what it produced.

Why the difference matters in a dispute

Evidence law and hearing-panel procedure both distinguish between types of evidence. Direct evidence is observation of the fact in question — a witness who saw something happen. Circumstantial evidence is inference from related facts — physical properties that are consistent with a hypothesis. Courts admit both, but they treat them differently when weighing competing claims.

An AI detection score is circumstantial evidence. It says: the finished text has properties consistent with AI generation. It does not report an observation of AI generation occurring. A process record is direct evidence. It says: here is what was observed to happen while this document was written, at these timestamps, with these revision patterns.

The NIST AI Risk Management Framework (AI 100-1) addresses this class of problem under the principle of knowing an AI system's limitations and documenting them before deployment. The Framework calls for understanding not just aggregate accuracy numbers but the conditions under which a model's outputs should and should not be trusted. A detection score generated on a text that was written under conditions the model was never trained to distinguish — careful non-native prose, highly edited academic writing, domain-specific terminology — is operating outside documented reliability bounds. A process record makes no such assumption: it was present.

There is a further practical point about burden of proof. In academic integrity proceedings, the institution generally carries the burden of demonstrating a violation occurred. An AI score shifts the field: it creates a prima facie appearance that requires the student to rebut. A process record doesn't rebut — it predates. A sealed record of composition submitted along with the document is evidence created before any dispute, by a party with no motive yet to fabricate it.

The tamper question

Process records have one additional forensic property that detection scores fundamentally cannot offer: they can be made tamper-evident.

A detection score is computed from the artifact at the moment of checking. Run the same text through the same tool on two different days and you may get slightly different scores. There is nothing to seal. The score is a snapshot of a computation, not a record with an integrity guarantee.

A process record, by contrast, can be hash-chained and cryptographically sealed the moment writing ends. Change anything afterward — the content, the timestamps, the sequence of edits — and the seal breaks. This is the property that makes the record independently verifiable: a reviewer does not need to trust the writer, the institution, or the tool vendor. They check the seal against the record and it either holds or it doesn't. As the distinction is explained in tamper-evident vs tamper-proof, "tamper-evident" is the honest claim — it means that tampering cannot be hidden, not that tampering is physically impossible.

No AI detection score offers anything equivalent. A fabricated document can produce a high "human" score just as an authentic one can. The artifact cannot attest to its own origins.

Where each belongs

This is not an argument that detection tools are useless. They serve a function. The right framing is:

Role	AI detection	Writing-process record
Evidence type	Circumstantial (inference from artifact)	Direct (observation of event)
Created	After submission, by the reviewer's tool	During composition, by the writer with consent
Reliability	Varies by text type, language, population	Tied to what the recording system observed
Tamper-evident	No — score is a computation on the current artifact	Yes — can be cryptographically sealed
Appropriate role	Triage / screening signal	Evidence in a dispute

Detection tools are reasonable as a screening layer: a flag triggers closer examination, which then uses better evidence. They become a problem when they are the evidence — when the score is treated as the finding rather than as a reason to look further. Turnitin's own guidance makes exactly this distinction; the problem is not the tool but the misuse.

Process records are appropriate as evidence in a dispute precisely because they were not created for the dispute. They are the writing itself, witnessed.

What this means for writers

The argument above leads to a concrete implication: if a process record predates the dispute and is observational rather than inferential, then its value is entirely front-loaded. A record created during composition — before any accusation exists, when the writer has no motive yet to fabricate anything — carries evidential weight that no post-hoc document can replicate. The question is not "how do I defend this submission?" but "did I record this submission while it was happening?"

For writers who want that property, the practical step is capturing and sealing the process at composition time. Authmarks does this via the desktop Witness app, which records at the OS layer rather than through a document add-on; the sealed record can be verified by anyone at verify without an account. But the underlying principle holds regardless of how the record is made: a writing record created during composition, sealed on completion, and held by the writer serves exactly the function this article describes — direct evidence that was present at the event.

The full definition of writing provenance — and why observing the process rather than judging the artifact is the more defensible foundation — is in what is writing provenance.

Related reading: how AI detectors work and why they break · what a writing record cannot prove · how to prove you didn't use AI · keystroke logging and writing process research · how accurate are AI text detectors?