The Pope’s Warnings About AI Were AI-Generated, a Detection Tool Claims

The real story isn't whether the Pope used AI — it's that Pangram Labs gets enormous free publicity every time its Chrome extension flags something famous, and neither WIRED nor readers have a clear way to assess the tool's accuracy. Detection tools have no agreed-upon benchmark, so a single high-profile flag proves nothing about reliability.

This connects directly to the arXiv paper from April 16 ('Context Over Content') which found that automated AI judges systematically prioritize context over actual content when evaluating outputs — a finding that should make anyone skeptical of detection tools operating on similar principles. If LLM-based evaluators bend toward what they expect to see rather than what's there, a detector trained on AI text will likely over-flag writing that merely resembles AI prose in style or topic. The WIRED newsroom piece from April 17 ('AI Drafting My Stories') is also relevant: as AI-assisted writing becomes normalized in professional contexts, the line detection tools are trying to draw gets structurally harder to locate, not easier.

Watch whether Pangram Labs publishes a peer-reviewed false-positive rate for high-profile or translated religious texts within the next 90 days. If they don't, the product is operating as a PR engine, not a verification tool.