How the detector works, and how we test it

Model card

The detector runs in two tiers, both inside your browser:

• Instant: text clean-up (invisible and look-alike characters), language check, sentence splitting, and explanatory style counts. No download. These counts are shown for context and are not used in the verdict.
• Standard: a small open-licence classifier, chosen after testing several candidates on the same texts, scores overlapping windows of the text. A smoother joins the window scores into passages. This tier drives the verdict.

How a verdict is made

A calibrated combiner turns the model scores into a probability for the whole text and an AI fraction (the share of words in AI-like passages). Then a fixed rule decides the outcome:

• Under 50 words: no result.
• 50 to 149 words: confidence is always low, and Mixed is not allowed.
• AI when the probability clears the threshold for the text's length and the AI fraction is at least 80%.
• Human when the AI fraction is 10% or less.
• Mixed when at least one high-confidence AI passage is 120 words or longer.
• Inconclusive otherwise, and for text that isn't English.

Cut-offs are set per length bucket so that at most 5% of the calibration human texts are flagged. The same calibration gives the AI-likeness score: the share of measured human texts of similar length that scored lower. It is shown capped at 99% either way, since the calibration texts are a sample of human writing, not all of it.

How it is tested

The detector is tested on a frozen set of 4,120 texts that are never used to train the model, scored exactly the way your browser scores them. Part of the AI text is also kept out of setting the cut-offs, so the test is not marked by the same texts that tuned it.

• Human writing (2,799 texts): fiction, encyclopedia articles, non-fiction books, academic abstracts and plain-English encyclopedia articles, all openly licensed and written before 2022.
• AI writing (1,321 texts): fiction from StoryScope, a public research dataset whose authors generated stories with several commercial chatbots; essays, emails, blog posts, news and academic abstracts; and AI text rewritten by "humanizer" tools to sound human.

The test set is fixed: every new version of the detector is scored on exactly the same texts, so versions can be compared fairly.

What we measure

Detected at a 5% false-positive cap

Share of AI texts flagged (as AI or Mixed) by the shipped cut-off, which is set so that at most 5% of calibration human texts are flagged.

False-positive rate

Share of human texts labelled AI at the shipped threshold. A Mixed result on a human text counts as a false positive.

AUROC

How well scores separate human from AI across all thresholds, from 0.5 (coin flip) to 1.0 (perfect).

Every metric is computed with a 95% confidence interval from a bootstrap that resamples whole source documents, by kind of text, by length and for the humanized subset.

Long documents

There is no word or character limit, and every word is scored: the site never samples. The text is split into windows of 254 tokens and every window goes through the model. Results stream in as the scan runs, with a progress bar and a cancel button. Above 30,000 words the text is shown in pages of 5,000 words to keep the browser responsive. Chapters are found from headings, with a menu to jump to each one.

Narrative fingerprint

For fiction, the result can include descriptive narrative features, such as dialogue share, smell imagery, narrator thematic commentary, direct reader address and time-jump markers, computed with our own word lists. Each is shown beside the human and AI base rates published in StoryScope: Investigating idiosyncrasies in AI fiction (Russell, Rajendhran, Pham et al., arXiv:2604.03136, 2026). They are computed with our own word lists, inspired by that paper, and shown as context next to the verdict.

Changelog