Method changes

FactUnion Method 1.4

Other languages (2026-10-27). Checks of fact sheets, scripts and recordings in Spanish, French, German, Portuguese and Hungarian, besides English. Nothing about English checks changed. The statuses, the score formula, the gates and the rules did not change. For another language the reading model is told, on every step, the language of the material: claims, quoted words and passages stay exactly as written (never translated; a quoted passage is still checked word for word against the page), searches are written in the claim's language, and every sentence written for people (explanations, corrections, suggested rewrites) is written in the claim's language. Numbers are read the way each language writes them (8,7 millones; 1.234,5; 1 234,5). Sentences are split correctly around German and Hungarian ordinals and dates (am 3. Oktober; 1848. március 15.) and Spanish and French punctuation. Recordings are transcribed in their own language. Labels on the report stay as they are.

Corrections are never guessed (2026-10-27). A test check corrected "the first part of Don Quixote was published in 1705" to 1622; the right year is 1605 (one source spoke of another edition). A subject often has several real dates: a first edition and later parts, the start and the end of a building, a law passed and in force. Now a correction must rest on named passages that give the value of that same property, and when the sources give different values for it, or any source contradicts the corrected sentence when it is read again, no correction is written: the fact is removed with "the value stated here is wrong, and the sources we read do not agree on the right one, so we did not guess". A suggested rewrite of a script line follows the same rule (the line is still flagged; the suggestion is withheld).

Fairness test results for this version

• Mirror test (same claim, other side): not run yet.
• Stability test (same result each time): not run yet. It must pass before this version checks any public episode.

FactUnion Method 1.3 · 18 October 2026

What changed, and why. Creators asked to check a script before they record it, so a wrong claim can be fixed in the text instead of in the studio. Version 1.3 adds two kinds of check. The statuses, the score formula, the gates, the bar of 85 and the rules that decide a claim did not change: a script is read and researched exactly like a recording.

• Script check. A script or transcript (plain text, Markdown, subtitles or a Word file; speaker names are optional) is split into numbered sentences and checked with the same method as a recording. Each claim that is not verified comes back with its sentence, where that sentence sits in the text, the result, the source, and a suggested rewrite of the line that keeps the writer's words and tone and says only what the sources state. When nothing true can replace a claim, the suggestion is to cut it. The rewrite is a new instruction to our reading model, which is why the version number moved.
• Final audio check. After recording, the final audio (or its transcript) is matched word by word to the checked script. When at least 90 percent of the words are the same, the script's results stand and only the lines that changed or are new are checked. When less than 90 percent is the same, the audio is checked in full as a new episode. The page says what was checked, for example: Script verified · audio matched 97% · 3 changed lines checked.
• What is never decided by a model: which lines changed (a word by word comparison, case and punctuation ignored), the 90 percent line, and which findings ask for a correction (a claim sources contradict or a detail that is off) and which only ask for a look (a claim with weak support).
• A changed line whose script version was verified and whose new version is not is always listed for a look, even when the new claim comes back as "can't be checked" rather than contradicted.
• Test runs on two of our own episodes (test mode): the script checks found 80 and 82 claims (the checks of the same episodes' final audio found 66 and 83). Matching each episode's own final audio to its checked script came out 99.9 percent the same in seconds, with no research. Five lines edited by hand, one with a wrong number, were found as five changed lines and only they were checked; the wrong number was caught both times (once contradicted, once no longer verified).

Fairness test results for this version

• Mirror test (same claim, other side): not run yet.
• Stability test (same result each time): not run yet. It must pass before this version checks any public episode.

FactUnion Method 1.2 · 17 October 2026

What changed, and why. A fact sheet of true, sourced facts for one of our own episodes kept 4 of its 28 facts: most were removed because only one strong source, or only lower-weight sources, stated them, even though nothing said otherwise. That bar was stricter than any newsroom's. We changed it so a check is factual but not absurdly strict. The four statuses, the score formula, the gates and the bar of 85 did not change.

• One reliable source that clearly states a claim is now enough for Verified, when no reliable source says otherwise. A reliable source is an official or original record, an established outlet, a reference or academic work, or an organisation's own page about itself. A second independent source still makes the result stronger, but it is no longer required.
• Two or more independent lower-weight sites that our own searches found, and that agree with each other, also verify a claim when nothing we read says otherwise. Links from the piece's own description, a source the speaker named, an appeal, free hosting addresses and open platforms never count toward this. One lower-weight site alone still makes a claim Questionable.
• A quote is on record when its exact words are on any page we read that states it. Words we find nowhere still make a quote Questionable (misattributed).
• When our readings of the same pages still disagree, the reading by our stronger reading model decides, and the claim still shows that the readings did not agree. It is no longer marked Questionable for that reason alone. Serious claims about a named person or company keep their bar: two independent sources, including an official record.
• A technical failure never counts against a claim. When a reading never comes back, we ask again; a reading that still did not come back does not vote. When a comparison step fails, we split it and ask again, and if part of it still fails, the readings stand without that comparison. If most of a step fails, the whole check runs again later.
• Fact sheets for our own episodes: a fact stated only by a lower-weight source, with nothing against it, is kept with a short note naming the source. A fact the sources back only in part is trimmed to what they state and read again. A fact sources disagree about is corrected to what the reliable source says and read again. A fact is removed only when a reliable source says otherwise and no correction holds, when nothing supports it after a real search, when a quote is not found, or when a key record would not open (shown as "Couldn't be checked", never as false).
• Audit results for this version (calibration, test runs): on the fact sheet that kept 4 of 28 facts, 21 are kept now, 5 of them corrected to what the sources say (for example the right author of a 1955 paper); on the fact sheets of three published episodes, 133 of 149 facts are kept (89 percent), against 38 with the old bar on the same readings; of 12 true facts copied with one deliberate error each (a wrong year, number or name, an invented quote, a reversed claim, a false attribution), all 12 were caught: 5 removed and 7 corrected back to the true value.

Fairness test results for this version

• Mirror test (same claim, other side): not run yet.
• Stability test (same result each time): not run yet. It must pass before this version checks any public episode.

FactUnion Method 1.1 · 14 October 2026

What changed, and why. The first full test run on our own episode showed results we could check by hand and found wrong, so these rules changed. The statuses, the score formula, the gates and the thresholds did not.

• Every judgment is now made by a language model that answers in a fixed form, never by hand-written text rules or word lists. A small, fast model (Claude Haiku 5.5) compares each claim with each source passage: whether a number, date, time or amount in the source is the same as the one in the claim (it knows that 90 minutes is 1.5 hours and that "September 5, 2025" is a date), whether the page is the person's or organisation's own, whether the page got the claim from somewhere else, and whether a note about context is about the claim. Our code then checks that every word the model says it copied from a page really is on that page, and does the arithmetic. This comparison runs in each of our repeated reads, so if the model is unsure, the reads disagree and we say so.

• Which pages are read for a claim. Pages are now chosen by how much of the claim they talk about, with the source list as a smaller weight. Before, the same links from the episode description filled the reading slots of most claims, so many claims were never compared with their own search results.
• Pages found while checking one claim are also offered to the other claims of the same piece when they talk about them.
• "No public record found" now needs pages from the claim's own searches. Links from the description no longer count, and a claim that says something did not happen ("nobody has shown this") is never given this result, because an absence leaves no record. Such claims are marked "Unable to determine" instead.
• A page we could not open makes a claim "Unable to determine" only when it is the page the speaker named, or an official page that a model, from its address and title, judges to be the key record for the claim. We now try such a page twice. The line now reads "We could not open a key record for this."
• An organisation's own page (its press release, product page, filing or official blog) counts as the original record of what that organisation itself announced, launched, priced, decided or said. One such record is enough to verify that kind of claim. It never verifies a claim about whether something works, or about anyone else. The model decides whose page it is from the address and the author only, and the words it relied on must appear there.
• A study's own paper counts as the original record of what that study found.
• A page that says it got the claim from another work is never the original record of it, even on an official or journal site.
• Two news outlets count as two sources unless one says it got the story from the other, or they have the same owner. Reporting the same story is not enough to count them as one. Copies of one news agency story still count as the agency.
• A post or video counts as a person's own words only when it is that person's own account, never an account whose name merely contains theirs (for example a fan or news account).
• Pages on free hosting addresses (for example name.pages.dev) belong to whoever runs that address. They count as the lowest source level unless our list names them.
• Numbers. A number in a claim is compared only with a number in the source that is the same thing, as the comparison model judges it (minutes and hours are converted; times of day must match exactly). A number the source does not mention is no longer treated as a contradiction. Dates are one date, and model names such as Z6III or ML-DSA-87 are names, not numbers. Round numbers said with "about" get at most 10 percent leeway plus the usual 15 percent band.
• Context that changes a claim's meaning now has to come from sources that state the claim, with at least an official source or two news sources behind it, and the comparison model must find it is about the claim, not about the page.
• When our repeated reads give "Not verified" for two different reasons (sources say otherwise, or no record found), the claim is marked Questionable, because the two reasons are not the same result.
• Claims held back by the rule for serious claims about named people now show their own rule codes, and that rule no longer covers people known to have died.
• The piece's own credits and its line about how it was checked are no longer listed as claims. Sponsor claims are still checked.
• Lower running cost, with the same evidence: a page is fetched once per check, identical searches run once, the separate answer search runs only when the claim's own searches found too little, and pages that share almost no words with a claim are not read for it. Each page sends the reading step the parts that talk about the claim, with the sentences around them, instead of filling a fixed length, and a page that shares few words with the claim sends a shorter part. Claims that rest on the same original record share one search for it.
• Audit results for this version: mirror suite (30 pairs, simulated readings, tests the rules only): 100 percent identical, sign test p = 1. Stability audit (simulated readings, 3 runs): claim match 1.0, status agreement 1.0, spread 0. The live runs are added here when they run.

Later on 2026-10-14, after the first live production run on our own episode (same version, 1.1):

• In production, every reading and comparison step runs on Google Gemini, paid per use. The second read of each claim is now done by Gemini 3.6 Flash (it was Gemini 3.1 Pro), a different model from the first read (Gemini 3.8 Flash), because the larger model cost more than half of the whole check. Each comparison runs with the lowest thinking setting.
• Four publishers added to the source list: Digital Camera World (a trade magazine), Bellingcat and Forbidden Stories (edited newsrooms), and the AAAI conference proceedings (peer-reviewed papers). Sources: 550 entries.
• A publisher on our list that announces something it did itself (for example a news agency describing its own test) now counts as the original record for that, like a company's own announcement already did.
• When a provider refuses our key or our prepaid credit is used up, the check now waits and runs again later instead of failing.

On 2026-10-15, after a review of that run by hand (same version, 1.1; no public result had used it):

• "The fact holds, but the named source does not say it" now needs the named page to have been read and found to be about something else. A named page that backs part of the claim, a page that did not open, or a reading that did not finish never gives that result.
• A claim whose named page was read and backs part of it is "Questionable, sources only partly back this", never "no public record either way".
• When a source states something different, the words we quote from it are the part that differs, up to 30 words, copied exactly. Before, we quoted the first 30 words of its passage, which could agree with the claim.
• The comparison model now also says when a source gives the same number as the claim but for a different thing (for example labels instead of signals), with the source's words. That source then states something different from the claim.
• "Only one source states this" is no longer said of a source whose context note changes what the claim means.
• Source list: Reuters' agency site, Thomson Reuters and the Associated Press's own site (added on 2026-10-14) are now in the list's source file too, and Indicator (an edited newsroom whose audits others cite) is added. Sources: 554 entries.

On 2026-10-17 (same version, 1.1):

• Public wording only (2026-10-17): the status Unable to determine is now shown as Can't be checked; two reason lines reworded (private_knowledge, unstable). Rules, statuses and scores are unchanged.

Fairness test results for this version

• Mirror test (same claim, other side): not run yet.
• Stability test (same result each time): Simulated run: passed: 1 test episode, 3 full runs each; claims matched across runs 100 percent (needs 85 percent), same result 100 percent (needs 90 percent), scores differ by 0 points (at most 5). Offline run with simulated readings, which tests the rules but not the reading software. 17 October 2026.

FactUnion Method 1.0 · 13 October 2026

• First version. Four statuses (Verified, Not verified, Questionable, Unable to determine), reason codes and fixed reason lines, the status rules in code, and the FactUnion Score (tier shares 50, 35 and 15), with a badge threshold of 85 and seven gates.
• Every claim we check against sources is read at least twice: by a fast reading model with the pages in ranked order, and by a stronger model with the pages in reverse order. Central claims, serious claims about named people, and claims where the first two reads disagree are read a third time. If the reads still disagree, the claim is Questionable. A piece whose repeated reads give scores more than 5 points apart, or where more than 20 percent of claims are unsettled, gets no score. Claims are also found twice, with the transcript cut at different places.
• While we test, the reading is done by Anthropic Claude models. In production it is done by Google Gemini, paid per use. A check never mixes the two, and test runs are never public ratings. Search and page text come from Exa.
• Sources: 546 entries, sorted by what kind of publisher a site is, never by its politics or subject. Social, video and press release pages count as someone's own words only on that person's or organisation's own account. A site we do not list that came only from the piece's description, a named source or an appeal counts at the lowest level.
• Numbers are compared in code: each number in a claim against the source's number of the same kind, and the worst pair decides. Round numbers said without "about" get at most 5 percent leeway.
• Sources on both sides of a claim, with support clearly ahead, make it Questionable (sources disagree).
• A claim researched again on a later check is never marked "too new" again. A claim carries over to a new version of a piece only when its words are exactly the same.
• Fact sheets: research facts for our own episodes are checked the same way, then kept, corrected and read again, or removed.
• Audit results for this version: mirror suite (30 pairs, simulated readings, tests the rules only): 100 percent identical, sign test p = 1. Stability audit (simulated readings, 3 runs): claim match 1.0, status agreement 1.0, spread 0.

Fairness test results for this version

• Mirror test (same claim, other side): Simulated run: passed: 30 pairs, 100 percent gave the same result on both sides, no lean to either side (sign test 1). Offline run with simulated readings, which tests the rules but not the reading software.
• Stability test (same result each time): not run yet. It must pass before this version checks any public episode.