Blog// AI safety

AI Hallucinations in Official Reports: Operational Risks and Verification Protocols

Claude sent Philadelphia police a false tip about a homicide that went undetected for two months. Here are the risks AI hallucinations pose to small businesses and ways to reduce them.

10-10-2026 · 7 min read
AI Hallucinations in Official Reports: Operational Risks and Verification Protocols

Habr reports that in July 2026, Anthropic’s Claude model sent Philadelphia police a false report about an unsolved homicide. The message went to a public tip line operated by the Police Department and was automatically marked as spam, so officers never saw it. The incident was not discovered until September 28—more than two months later—when Anthropic’s internal audit found a discrepancy between the generated text and the facts.

The same Habr article mentions separate cases in which chatbots became part of real investigations: in Florida, a woman was charged after discussing a plan to shoot up a sheriff’s office with Claude; in Brazil, a man was arrested after the FBI gained access to his ChatGPT conversations through a request to OpenAI. These examples show that AI errors can move beyond academic tests and affect law enforcement as well as the lives of ordinary people.

For owners of small shops, service businesses, and manufacturers in Russia and the CIS, these stories show that even the most advanced AI systems can produce false information—and that a response to it may be delayed. This creates risks to reputation, legal liability, and customer trust. It is important to understand how such failures happen and what measures can help reduce them.

How AI generates false information

Models such as Claude and ChatGPT work by statistically predicting the next word, not by checking facts. They are trained on huge collections of text containing reliable information as well as fiction, jokes, and speculation. When a model is asked to interact with random webpages, it may treat the content it finds as a source and incorporate it into an answer without independently verifying it.

In the case of PhillyUnsolvedMurders.com, the model visited the site, took fictional details about an unsolved homicide from it, and turned them into a message that looked like a real tip from an informant. This process is called hallucination: the model creates coherent, plausible-sounding text that does not match reality. It is important to understand that hallucinations are not an error in the traditional sense; they follow from an architecture optimized for plausibility rather than accuracy.

Why a delay in detection is dangerous

The two-month gap between the false tip and its discovery created a window in which inaccurate information could have affected government operations. The Philadelphia Police Department said this kind of delay is unacceptable because it prevents authorities from responding promptly to potential threats and wastes resources investigating false alarms.

For a business, a delay means an AI’s mistaken conclusion could reach customer communications, marketing materials, or internal reports before anyone notices. For example, an automatically generated product description could make false claims about certifications or features, leading to returns, negative reviews, or even complaints from regulators. The longer the information remains undetected, the greater the potential financial and reputational damage.

What this means for small businesses in Russia and the CIS

Even if you use AI only to write product descriptions or answer chat questions, the risk of false information remains. Small companies often rely on off-the-shelf solutions without deep technical expertise, so it is especially important for them to establish simple but effective safeguards.

Check everything AI generates

Set a rule that any text created by a model must be reviewed by a person before it is published or sent to a customer. The review can be brief: make sure the message does not mention specific events, dates, or names without supporting sources. Assign someone in each team that uses AI to be responsible for this check.

Limit access to external websites

If your workflow requires AI to browse webpages, configure an allowlist of trusted sources or disable internet access altogether. Access to open webpages increases the chance of picking up false content, as the PhillyUnsolvedMurders.com example shows. Restricting access reduces that risk without significantly limiting functionality for most business tasks.

Keep a log of prompts and responses

Save logs of all AI interactions: which prompts were submitted, what the model returned, when it happened, and who approved it. A log helps you quickly trace the source of an error if one occurs and shows regulators that you are exercising deliberate oversight of your AI use. For a small company, a simple spreadsheet or a built-in log in the service you use is enough.

Choose providers with transparent security policies

When choosing a service, look for audits, reports on how its filters work, and a commitment to notify customers promptly about incidents. Transparency on security reduces the risk of unpleasant surprises and gives you leverage to request improvements. If a provider publishes no information about its measures for preventing hallucinations, consider alternatives.

Train employees in basic fact-checking

Run a short training session so employees learn to check key facts against independent sources before using AI-generated text in customer communications. Knowing how to verify a date, name, or location using public databases or news aggregators reduces reliance on blind trust in a model and improves the team’s overall information literacy.

How to reduce risks when using chatbots and generative models

Technical measures complement organizational rules. Together, they create layers of protection that reduce the chance of false information making it into the final product.

Add output filters

Configure automatic checks for patterns associated with false claims—for example, references to specific crimes, disasters, or court rulings without supporting data. These filters can use simple rules or lightweight classification models to block or flag suspicious messages for manual review.

Limit response length and format

Short, templated answers reduce the likelihood that a model will “make up” details outside the requested format. If your bot answers common questions, prepare an approved set of responses in advance and let the AI choose only from those options. This narrows the space for hallucinations without reducing usefulness in most scenarios.

Use fallback mechanisms

If a model is unsure of an answer (has a low generation probability), switch to a prepared template or route the request to an operator. Confidence can be estimated using internal log probabilities, which many APIs provide. This approach ensures that when the model is uncertain, the customer receives verified information instead of fiction.

Conclusions and actions for the coming week

Here are practical steps you can take within the next seven days:

  • Make an inventory of all AI tools used in marketing, support, and internal processes.
  • Assign someone to review every AI-generated text this week.
  • Set up a simple filter in your company chat or ticketing system to flag messages containing words such as “murder” or “assault” when no supporting information is provided.
  • Contact your current AI service provider and ask about its procedures for detecting and reporting false outputs.
  • Hold a 30-minute team briefing on basic fact-checking: how to verify a date, name, and location using public sources.

These steps will help you quickly identify weak points in your current use of AI and reduce the chance that false information reaches customers or partners.

Frequently Asked Questions

What are AI “hallucinations”?

AI “hallucinations” are the generation of false or fabricated information that the model presents as fact. This occurs due to the characteristics of training and the lack of real contextual understanding.

How can small businesses protect themselves from AI hallucination risks?

To protect themselves, businesses should implement multi-level data verification, use AI as a supporting tool rather than the sole source of truth, and train staff in critical thinking when working with AI-generated content.

What consequences can false AI messages have for business?

False messages can lead to reputational damage, financial losses, and legal issues. For example, with Claude, a false police call demonstrates the seriousness of potential consequences.

Based on: Habr — Anthropic’s AI model sent Philadelphia police a false report about a homicide

Analysis & review by
Alexander K.

Web development, digital advertising and marketing expert since 2012. Founder of AIFORSALE.

How we can help
Online storesMarketingAI agentsContact us
← All articles