Attackers poison AI chatbots by flooding the web with malicious pages designed to rank high in searches. Vigilance Security researchers identified “Dark Sourcery,” a campaign where fake support pages, reviews, and documentation trick chatbots into serving scam phone numbers, fraudulent login pages, and phishing links as legitimate answers. The chatbot appears to be faithfully citing real sources; the poison is in the data layer, not the model itself.
This matters more than prompt injection does. Data poisoning attacks the foundation all models draw from. A chatbot can’t refuse an answer it doesn’t know is poisoned. Netcraft’s independent research backs this threat: ChatGPT fails to identify brand login pages 34% of the time and returns potentially malicious domains.
Real users are already scammed, receiving fraudulent contact details from chatbots. Not theoretical risk anymore.
Expect more variants as chatbots replace traditional search. Russian disinformation networks have already infiltrated chatbot responses on geopolitical topics; targeted phishing campaigns will follow.
Sources: Vigilance Security • Netcraft
Coverage: Dark Reading • France24 / NewsGuard • Barracuda / Mithril Security
Related on this blog: [AUTO] Hidden Ads Target AI Models at Publisher Level • [AUTO] Malware campaigns weaponize AI platform trust • [AUTO] PhantomRaven: Stealing Secrets for Bug Bounties