Writing evidence methods compared: a systematic evaluation across eight criteria

Short answer

No single method is sufficient. Every approach to writing evidence has a distinct failure mode — a question it cannot answer, a condition under which it breaks, or a fraud it cannot reliably detect. The practical question is not "which method is best?" but "which combination of methods addresses the most failure modes at an acceptable cost?" For contested cases, the minimum defensible combination is a sealed process record paired with oral examination: these two methods target orthogonal failure modes and together close the gaps each leaves open individually.

---

How to use this comparison

The comparison below evaluates six evidence methods against eight criteria. The criteria were chosen to reflect questions that arise in formal integrity proceedings, legal challenges, and institutional review — not just whether evidence is informative in a classroom context, but whether it can withstand challenge from a motivated adversary or a formal appeals process. A method can be useful for one purpose (screening) while being too weak for another (standalone justification in a contested case). The framework makes that distinction explicit.

The criteria are defined in the analysis section below. The table uses the following notation: Yes means the method reliably addresses the criterion under normal conditions; Partial means it provides some evidence but with significant caveats; No means the method does not address the criterion or addresses it so weakly that it should not be relied on for that criterion alone.

---

The comparison table

Criterion	AI detector score	Stylometric analysis	Version history	Oral examination	Keystroke record (unsealed)	Sealed hash-chained process record
Evidence type	Circumstantial	Circumstantial	Circumstantial	Observational	Observational	Observational
Created at time of writing	No — post-hoc	No — post-hoc	Partial — continuous but mutable	Yes — real time	Yes — real time	Yes — real time
Tamper-evident	No	No	No	N/A — live	No	Yes
Independently verifiable without vendor	Partial	Partial	No	Yes	Partial	Yes
Addresses paste / transcription events	No	No	Partial	Partial	Yes	Yes
Addresses identity (who was at the keyboard)	No	Partial	No	Yes — tests understanding directly	No	Partial — device attribution only
Adversarial cost to defeat	Low	Medium	Low	High	Medium	High
Appropriate role	Screening only	Corroborative	Corroborative	Corroborative / standalone	Corroborative	Primary

---

Analysis by criterion

1. Evidence type: circumstantial vs observational

This distinction matters more than its academic framing suggests. Circumstantial evidence infers past events from present artifacts — the submitted text is examined for properties that correlate with how it came to exist. Observational evidence was present at the event it documents; it records what happened, not just the residue.

AI detection, stylometry, and version history are all circumstantial in this sense. They analyze the finished product (or a post-hoc log of it) and draw inferences backward to the writing event. Oral examination, keystroke records, and sealed process records are observational — they either occur at the time of writing (keystroke logs, sealed records) or directly test the writer in real time (oral examination). The distinction matters for evidentiary weight: circumstantial evidence requires an inferential step that an adversary can contest; observational evidence does not. Courts and tribunal rules across common-law jurisdictions draw this distinction routinely in authenticating digital records (FRE Rule 901).

2. Created at time of writing vs post-hoc

Evidence created at the time of the writing event is structurally harder to fabricate because any fabrication must either (a) have anticipated the need for evidence before the writing occurred, or (b) convincingly reconstruct the event after the fact. AI detector scores are unambiguously post-hoc: the text is submitted, then scored. Stylometric analysis is the same — it is run on the completed text. These methods cannot, in principle, produce evidence of what happened during writing; they can only infer it.

Version history occupies an intermediate position. Platforms like Google Docs log revisions continuously, which looks like concurrent capture. But the log itself is not cryptographically sealed: the DocuSim project (publicly available on GitHub) demonstrates that a visually indistinguishable replica of Google Docs revision history can be synthesized, and the underlying XML format for Word revision history can be edited directly. The OIA's UK casework guidance on AI and academic misconduct notes that version history "may not be sufficient on their own to establish whether AI has or has not been used." Continuous capture is only as strong as the tamper-evidence protecting it.

Keystroke logs and sealed process records are created concurrently with writing. The sealing step — computing and storing a cryptographic hash of the log at capture time — is what distinguishes a sealed record from an unsealed one. An unsealed keystroke log is captured at the right time but can be modified afterward; it is observational in origin but not tamper-evident in form.

3. Tamper-evidence

A tamper-evident record provides a verifiable signal if its content has been altered since capture. Cryptographic hashing is the standard mechanism: a SHA-256 hash of the captured data at the time of sealing produces a fingerprint that changes if a single byte of the underlying record is modified. NIST SP 800-86, the Guide to Integrating Forensic Techniques into Incident Response, recommends hashing as the minimum integrity control for digital forensic evidence, precisely because it allows any verifier to confirm the record is identical to what was captured.

None of the post-hoc methods — AI detection, stylometry — have a tamper-evident dimension: they produce scores on the text as presented, and a different text produces a different score. Version history is not hash-protected; the log can be altered without a cryptographic indicator. An unsealed keystroke log is in the same position: it documents what happened, but the document itself can be modified.

Only a hash-chained sealed record is tamper-evident, in the technical sense that a verifier can recompute the hash and detect modification. If the record has additionally been anchored to an external timestamp authority — a third-party service that holds a signed record of the hash at a specific moment — even the original evidence holder cannot retroactively alter the record without the external anchor revealing the discrepancy.

4. Independent verifiability without vendor

Evidence that requires the original vendor's systems to verify creates a dependency: if the vendor is unavailable, changes their platform, or interprets their own data differently, verification is disrupted. The gold standard is evidence any third party can verify using only the record itself and publicly documented methods.

AI detector scores can be re-run by any party who has access to a comparable detector, but different vendors produce different scores on the same text — Weber-Wulff et al. (arXiv:2306.15666) found significant variance in detection outcomes across tools when tested against the same corpus, with several commercial tools performing no better than chance on some text categories. "Independently verifiable" is therefore only partially true: a different tool, or the same tool after an update, may yield a different score.

Stylometric analysis is in principle reproducible given the algorithm and training data, but in practice the tools are not standardized and results vary substantially. Version history requires access to the platform's interface or API; there is no platform-independent format or hash to verify against. An unsealed keystroke log can be read by any party with access to the file, but there is no cryptographic proof that the file is unmodified.

A sealed hash-chained record with a published hash — or with a third-party timestamp anchor — can be verified by any party using standard cryptographic tools (e.g., OpenSSL) without vendor participation. This is the definition of independently verifiable evidence under NIST's framework.

5. Addresses paste and transcription events

One of the most important behavioral signals in writing evidence is how text arrived in the document: was it typed character by character, pasted in a large block, or inserted in structured chunks? Paste events are highly diagnostic. Text pasted from an AI-generated source, a contract cheating service, or another document arrives without the inter-keystroke intervals that characterize composition. Crossley et al. (EDM 2024) found that keystroke logs classify authentically written versus transcribed essays with 99% accuracy in controlled conditions, with authentic composition showing "a greater number of pauses before sentences and words" — the cognitive planning signature of writing that transcription lacks.

AI detection and stylometry operate only on the final text; they have no access to how the text arrived in the document and cannot directly distinguish paste from typed input. Version history shows the state of the document at each revision snapshot but typically does not log the mechanism of text arrival (whether content was typed or pasted) with the granularity needed to classify individual events. Oral examination can probe understanding of the text and may reveal gaps consistent with transcription, but this is inferential.

Both keystroke records and sealed process records, by contrast, capture input events at the OS level. A paste event is distinguishable from typed input in the event stream: a large block of text appears without a sequence of key-down events preceding it. This is a direct, observable signal, not an inference from the finished text.

6. Addresses identity: who was at the keyboard

This is the single criterion where honest assessment requires the most care — and where the field's literature is most frequently overstated.

No method positively identifies who was at the keyboard from a single writing session without a biometric reference sample. This applies to all six methods.

AI detection scores are properties of the text; they carry no identity information at all. Stylometric analysis compares the submitted text to a corpus of prior writing attributed to a candidate author — it addresses identity in the sense of asking "does this match this person's known style?" but only if prior samples exist, the genre matches, and the comparison survives the degradation from LLM assistance that Schuster et al. documented. Version history is linked to an account login, not to a physical person.

Oral examination is the method with the most direct bearing on identity — not biometric identity, but functional identity: the ability to demonstrate understanding of the submitted work. A person who did not write a piece cannot reliably answer unprepared questions about its specifics. Chaski (2005), in a foundational paper on authorship attribution in digital evidence (International Journal of Digital Evidence), draws the distinction precisely: "identifying the person who sat at the keyboard" and "identifying the author of the document" are different problems. Oral examination addresses the second; biometric comparison would be needed for the first.

Keystroke process records identify the device, not the person. They establish that text was produced in a writing event on a specific machine at a specific time, which is valuable — it narrows the question considerably — but it is not personal identification. Acien et al.'s TypeNet system (arXiv:2004.03627) achieves a 0.3% Equal Error Rate for keystroke biometric authentication, but that result requires a large enrollment corpus and expert comparison; it is not a property of a single writing session. A sealed process record alone does not resolve who sat at the device.

The honest summary: identity — in the strict sense of "this specific person was at the keyboard" — is not fully established by any single method reviewed here. A sealed process record establishes device attribution; oral examination establishes comprehension of the work; a biometric comparison would be needed to connect the two with high confidence.

7. Adversarial cost to defeat

How difficult is it for a motivated adversary to defeat or circumvent each method? This criterion determines how the method performs under pressure — which is the condition that matters most when evidence is contested.

AI detection has the lowest adversarial cost of any method reviewed here. Weber-Wulff et al. tested detection tools against paraphrase attacks — running AI-generated text through a rewriting step — and found that several tools' detection rates dropped sharply. The text remains AI-assisted; the statistical fingerprint that detectors rely on is disrupted. The paraphrase step requires no technical sophistication: free web tools perform it in seconds. This is not a hypothetical; the paraphrase-attack literature is well-developed and the tools are publicly available.

Stylometry requires more effort to defeat: an adversary must either genuinely adopt the target author's style or use a tool fine-tuned on that author's prior writing. This is harder than paraphrasing but not out of reach for a determined actor.

Version history can be fabricated using publicly available tools. The DocuSim demonstration mentioned earlier is not sophisticated security research; it is a browser-based demo. A motivated actor with moderate technical skill can produce a version history that appears authentic.

Oral examination is the hardest method to cheat — it requires real-time unprepared engagement with the content. Pre-recording answers, using a substitute, or consulting an AI in real time during an oral examination all require either physical deception or technical bypasses that are detectable by appropriate supervision. There is no known low-effort attack on a properly supervised oral examination.

A sealed process record has high adversarial cost because defeating it requires fabricating both the behavioral timing signatures that distinguish authentic composition from transcription — including the cognitive pause distribution at linguistic boundaries that Crossley et al. identified — AND producing a sealed record with a correct cryptographic hash that passes independent verification. An automated typing script can replicate average typing speed; it does not replicate the distributional properties of compositional pauses without modeling the underlying cognitive process. The asymmetry is important: an honest writer produces the right signal automatically; a forger must solve a non-trivial inverse problem.

8. Appropriate role: screening / corroborative / standalone

Given the above, the appropriate use of each method follows directly.

AI detection is appropriate only for screening — identifying submissions worth closer examination. Its false positive rates (flagging genuine human writing as AI-generated), documented by Weber-Wulff et al. and others, make it unsuitable for standalone accusation. OIA casework guidance explicitly states that institutions should not take action based solely on a detector score.

Stylometry is corroborative — it can support a case that already has other evidence, but its ceiling of approximately 76.5% accuracy in standardized benchmarks (Tyo et al., 2022), and its degradation under LLM-influenced text, mean it cannot carry a contested case on its own.

Version history is corroborative at best, with the tamper-evidence caveats above. It is the most widely deployed writing evidence type, but its evidentiary weight in a formal proceeding is limited by its mutability.

Oral examination can function as a standalone method in some institutional contexts — it directly tests the cognitive relationship between writer and work. Its limitations are practical: resource cost, scheduling, and the inability to definitively establish what happened during the original writing event. It should be understood as testing current comprehension, not as reconstructing past process.

An unsealed keystroke log is corroborative: it provides behavioral process data that is hard to fabricate convincingly, but without a tamper-evident seal there is always a challenge available — that the log was modified before presentation. Unsealed logs are more valuable than no process data, but weaker than their sealed equivalent.

A sealed hash-chained process record is the only method in this comparison that combines observational evidence, real-time capture, and tamper-evident verification. It is the appropriate primary evidence type for a contested case — not because it is infallible, but because it addresses the most criteria simultaneously and carries the highest adversarial cost to defeat.

---

The two-method baseline: why sealed process record + oral examination is the minimum defensible combination for contested cases

The table above shows that no single method is adequate for a contested case. The question is which combination, at minimum, closes the most failure modes.

Sealed process records and oral examination together address orthogonal failure modes. The process record establishes what happened during the writing event: that text was composed with the timing signatures of authentic composition, that paste events were or were not present, that the session covered the required period, and that this record is cryptographically verified as unaltered. Oral examination establishes current understanding: that the person defending the work can engage with its content, reasoning, and structure in real time without preparation.

The process record cannot test understanding. A student who used automated typing tools to produce a superficially authentic-looking log would still need to pass an oral examination. Oral examination cannot reconstruct the original writing event. A student who learned a piece of AI-generated text well enough to pass a viva might still be exposed if no plausible writing process exists for the submission. Each method's strength targets the other's weakness.

The combination is not proof-beyond-doubt in every case. It does not resolve the identity problem in the strict biometric sense — it establishes that a process record exists for a specific device at the right time and that the person presenting the work now understands it. But it produces a case whose two components are both expensive to defeat and attack each other's weaknesses. For contested cases in academic integrity proceedings, employment disputes, or IP ownership claims, this combination represents the minimum defensible evidentiary standard.

Adding stylometric comparison as a third corroborating element further strengthens the case where prior writing samples exist, without changing the baseline. Institutions that collect process records as a matter of course — rather than reactively after suspicion arises — build the most defensible evidentiary base, because the record exists from the moment of writing and the adversarial cost argument applies in full.

---

What the literature does and does not establish

Claim	What it rests on	Status
AI detection tools have low adversarial robustness to paraphrase	Weber-Wulff et al. (arXiv:2306.15666); paraphrase attack literature	Established (experimental)
Stylometric attribution has a ceiling of ~76.5% on standardized tasks	Tyo et al. (arXiv:2209.06869)	Established (benchmark)
Keystroke logs classify authentic composition vs transcription with 99% accuracy	Crossley et al. (EDM 2024)	Established (controlled experiment); generalization to real-world cases uncertain
Biometric identification from keystroke timing requires a large enrollment corpus	Acien et al. (arXiv:2004.03627)	Established
Version history can be fabricated with publicly available tools	DocuSim (GitHub; proof of concept)	Demonstrated
OIA cautions against sole reliance on AI detection or version history	OIA casework note	Established (regulatory guidance)
FRE 901 requires authentication of digital evidence via a showing of process or system reliability	Rule 901(b)(9); NIST SP 800-86	Established
Sealed hash-chained records satisfy FRE 901 authentication requirements	Logical extension of NIST hashing standards	Well-founded; not yet litigated in writing-evidence cases
Oral examination is the only method that directly tests current understanding	Methodological analysis; no contrary evidence found	Undisputed but understudied as a formal metric
A sealed process record plus oral examination constitutes a minimum defensible combination	Synthesized from the above	Analytically well-grounded; no controlled study has tested the combination in formal proceedings
Cognitive pause distributions at linguistic boundaries are hard to replicate automatically	Crossley et al. (EDM 2024); behavioral writing research	Established directionally; no adversarial red-team study found
Sealed writing records have been tested in actual judicial proceedings	Not found in literature	Gap — not yet established

The most significant gap is the final row. The evidentiary argument for sealed process records is grounded in established cryptographic practice and closely analogous to NIST-recommended forensic evidence handling, but the specific use of sealed writing-process logs in formal legal or tribunal proceedings has not yet produced a public body of case law or published empirical study. The argument is well-founded; the track record is not yet there. Institutions building on this approach should treat it as a developing area of practice and document their methodology carefully — the logic holds, but it has not yet been stress-tested under adversarial legal scrutiny at scale.

---

Authmarks is designed to produce sealed, hash-chained process records — evidence that addresses the criteria in the first column of this comparison rather than the last.

Related reading:

• Temporal analysis in writing evidence
• Chain of custody for digital writing evidence
• Writing process evidence vs AI detection
• Investigating AI writing concerns
• How to prove authorship of a document