Claims Validation

Scope: This page defines the ideal-state specification for a CSA fact-checking module to be developed. It establishes the required behavior of the module, the editorial workflow for acting on its output, and the infrastructure needed to support it. Sections marked Pending identify capabilities not yet confirmed as implemented. Everything else represents the target standard the module should meet and that editors should apply once it is live.

⬇ Download this section

Content Pipeline Tiers

Not all content in the CSA pipeline requires the same level of fact-checking oversight. The tier a piece belongs to determines whether and how the Claims Validation module applies.

Tier names are provisional—naming conventions are not yet finalized.

The rules documented on the rest of this page apply in full to High Touch (HITL) content. Semi-Automated content—everything that is not HITL or Fully Automated—applies the same rules with editor judgment on verdict prioritization. Fully Automated content does not surface module output and does not require a Claims Validation pass.

Role-level access: Which specific roles can view raw module output in the CSA UI is pending confirmation with engineering leadership and product leads.

What the Module Produces

The CSA fact-checking module runs automatically as part of the content validation step. After a draft is generated, the module evaluates factual claims against its knowledge base and returns a structured validation output for each flagged claim. The following defines the required output format.

Each flag carries one of five verdicts:

Confidence Level

Pending module capability. The module should return a confidence level alongside each verdict. This field is not yet confirmed as available. The escalation logic below includes confidence-level handling and will apply once the field is live.

When present, the confidence level indicates how certain the module is in its verdict. Treat it as an escalation signal:

Editorial Action Taxonomy

I. Needs Correction

Factually wrong, demonstrably misleading, or unsupported by a verifiable source. Apply when the CSA validation output returns:

• FALSE—rewrite or remove the claim entirely before peer review
• MISLEADING—reframe the claim; document what the module flagged and what you changed

II. Needs Clarification

Nuanced, partially true, or contextually dependent; requires rewording rather than removal. Apply when the CSA validation output returns:

• OVERGENERALIZED—narrow the scope or add qualifying language
• INSUFFICIENT_EVIDENCE—add a verified source, or remove the claim if no primary source can be found

TRUE flags require no action. They are informational and confirm the claim passed validation.

Source Authority Tiers

The module weights source authority when evaluating claims. Editors should apply the same weighting when reviewing flagged output. The specific approved outlets within each tier are documented in Acceptable Sources §5.

Tier 1—Authoritative (accept)

• Government and institutional primary sources (.gov, .edu, established regulatory bodies)
• Peer-reviewed publication records
• Official statements from named public figures on verified channels
• Established national news organizations (AP, Reuters, BBC, major metros)
• McClatchy publications

Tier 2—Acceptable with verification

• Trade and industry publications with editorial standards
• Official company/brand statements via verified press releases
• Court records, police reports, legal filings

Tier 3—Flag for review

• Aggregator blogs, content farms, unverified third-party roundups
• Anonymous sourcing without corroboration
• Social media posts, even from verified accounts (unless the post itself is the news)

If the module returns a MISLEADING or FALSE verdict and the original source is Tier 3, treat it as Needs Correction regardless of the specific verdict.

Source Count Requirements

The number of Tier 1 sources required to verify a claim depends on claim type.

One Tier 1 source is sufficient for:

• Straightforward factual claims—dates, confirmed events, official records
• Statistics drawn directly from a government database or institutional publication
• Direct quotes from named public figures via verified channels

Two independent Tier 1 sources are required for:

• Contested or sensitive claims where sources could reasonably disagree
• Allegations, legal proceedings, or anything that could harm a named individual
• Health claims not covered by a clear primary source document

A primary source document is required (mandatory regardless of count—news coverage of the document does not qualify) for:

• Court outcomes, legal status, and regulatory figures
• Health dosage, drug interaction, and treatment efficacy claims
• Financial statistics cited as authoritative fact

Content-Type Specific Rules

Some content types carry elevated risk and require stricter treatment of module output.

Health and Medical

• Any flagged health claim is a hard stop. Do not publish until the claim is either removed or confirmed against a Tier 1 source.
• OVERGENERALIZED flags on health content should be treated as Needs Correction, not Needs Clarification.
• Dosage, drug interaction, and treatment efficacy claims require a primary source document—two independent Tier 1 sources if no primary source document exists—even if the module returns TRUE.

Legal and Regulatory

• Court outcomes, legal status, and regulatory figures require a primary source document (filings, official dockets, agency databases)—news coverage of those documents does not qualify.
• Stale regulatory figures are a known failure mode. If a claim involves a figure with a defined annual update cycle (tax rates, benefit amounts, sentencing guidelines), verify the vintage of the figure the module used.
• Allegations or claims that could harm a named individual require two independent Tier 1 sources.

Financial and Economic

• Dollar amounts, percentage changes, and economic statistics cited as authoritative fact require a primary source document even on TRUE verdicts.
• Forward-looking claims (projections, forecasts) must be attributed to the source of the forecast—not stated as fact. One Tier 1 source (the originating institution) is sufficient if the claim is attributed.

Real Estate and Local Services

• Local pricing, service availability, and geographic coverage claims have a known error rate in CSA output. Treat all OVERGENERALIZED flags in this category as Needs Correction.
• Address-specific or market-specific claims: verify against a current local source before publish.

Travel and Scheduling

• Operating hours, travel restrictions, visa requirements, and schedules are time-sensitive. Even TRUE verdicts may be stale. Verify against the official source at time of publication.

Entertainment and Celebrity

• Relationship status, pregnancy, and legal proceedings require two independent Tier 1 sources—both must be named and on-record. Unconfirmed claims should not appear in the draft regardless of module verdict.
• Historical facts about creative works (release dates, awards, box office): one Tier 1 source is sufficient; TRUE verdicts can proceed without additional verification unless the piece is a definitive reference.

Escalation Logic

Audit Trail

Implementation required. Validation output must be stored per-piece in the CMS—not only visible at generation time. This is a stated requirement; confirm implementation with the CSA product/dev team.

The audit trail for each piece must include:

• The full module output at the time of generation—all verdicts returned, the claim each verdict was attached to, and the confidence level (once available)
• Any editor actions taken on flagged claims—what was changed, removed, or overridden, and why
• The final publication state relative to the module output—which flags were resolved and how

This record must be:

• Stored per-piece in the CMS—accessible after the piece is published, not only during the drafting workflow
• Attached to the piece record—not stored in a separate log that requires cross-referencing
• Readable by editors, senior editors, and content leads—role-level access pending confirmation with engineering leadership and product leads

The audit trail is the authoritative record for editorial decisions and pattern analysis. It is what makes override documentation, escalation history, and recurring-bug identification possible across time.

Override Documentation

When an editor overrides a module verdict—publishing a claim the module flagged, or removing one it verified—note:

• What the module returned
• What the editor decided
• One-sentence rationale

This documentation travels with the piece through the review process. It is not punitive. Editors who consistently out-perform the module’s flags are providing signal that improves it.

Override documentation also serves as the source for aggregate pattern tracking. Across pieces, the record should capture:

• Flag frequency by type—how often each verdict (FALSE, MISLEADING, OVERGENERALIZED, INSUFFICIENT_EVIDENCE) is being flagged, by content type
• Remediation type—what action the editor took on each flagged claim:
  • Rewrite—claim was reworked and kept
  • Removal—claim was cut entirely
  • Kept with rationale—editor judged the module flag incorrect and published as-is
• Override rate by verdict—how often each verdict type is being overridden versus acted on, which surfaces where the module is underperforming editorially

This aggregate view is what enables pattern-bug identification, module improvement, and editorial QA over time. It requires the per-piece audit trail to be stored in the CMS (see Audit Trail).

Acceptance Criteria for Test Article Review

When walking the 15 test articles, apply each of the following:

• Does the verdict taxonomy map cleanly to the Needs Clarification / Needs Correction split defined in this document?
• Are there verdicts that don’t fit either tier? (These need a third category or a handling note.)
• Does the content-type-specific rule apply correctly in the test case?
• Does the source tier weighting change the editorial action in any test case?
• Is there any verdict the module returned that an editor would have caught differently—and if so, what does that imply for the ruleset?