Threat Actors are Finding Their Way into Your AI Summaries

|

|

Reddit logo
A screenshot displaying Netcraft's newly updated dashboard with a cleaner user experience.

Overview

  • Netcraft’s June 2026 research into AI response accuracy has found that the risk posed is evolving. Netcraft ran nearly 3,000 natural language queries for brand pages through four of the top answer engines and was recommended erroneous links pointing to live, attacker-controlled infrastructure. Previous 2025 research primarily surfaced unclaimed and parked domains from model hallucination.  

  • The proportion of malicious items returned varied by answer engine, showing that safety features are unequal among LLM-enabled search tools. 

  • Brands are advised to move from passive countermeasures, like deprecated personalised login images, towards proactive monitoring for scams using generative engine optimization to reach their customers.  

How AI summary accuracy (and inaccuracy) has changed 

AI summaries have become ubiquitous in the modern search engine. They are often the first result that users see and helpfully contain links alongside their authoritatively worded answers. However, these answer engines are fallible, and their responses could include malicious links to phishing sites, malware, and more.  

Netcraft tested queries about the login pages of several of the world's largest banks and retail brands in ChatGPT, Copilot, Gemini, and Perplexity. Those queries generated 2,905 responses containing a total of 20,706 links. Netcraft also found malicious links in 1.7% of AI-generated responses across nearly 3,000 tested queries.  

AI search assistants typically include multiple links in their answers, with 0.28% of the total number of links returned being malicious in nature. These links don’t represent hallucination of parked or non-existent infrastructure but rather attacker-controlled assets that could harm users. 

Within this dataset, Perplexity returned the most malicious results when tested with the same search queries. The queries chosen were comparable to organic user searches for important pages, such as “How do I access my {brand} account online?” or “How do I access {brand} online services?”

Previous Netcraft research from 2025 focused on the risk of unclaimed assets surfaced by non-browser AI assistants, finding 34% of web sites cited in answers weren’t controlled by the brand in the prompt. The results at the time were reflective of the risk posed by exploitable hallucinations.  

What makes these new findings different is that they reflect the realised risk of AI misidentifying real and malicious resources to then present them to users. These answer engines use retrieval augmented generation (RAG) to supply up-to-date answers, but this means that they also have to judge new sources on the fly with limited information. As a result, incorrect AI citations are increasingly coming from current web data.  

This shows the exploitability of generative engine optimization (GEO), similar to how regular search engine optimization (SEO) has been abused by scammers. The risk and practice of malicious GEO have already been demonstrated in the wild and Netcraft’s data confirms that this is being seen across generative search products. It is almost certain that attackers will increase exploitation of generated search results by leaning on GEO rather than “slop-squatting” on expected hallucinations. 

The problem is evolving 

AI-assisted search still falls prey to basic phishing campaigns sometimes, recommending fake login pages, shops, or rewards programs. 

A 2026 example of a fake shop site impersonating Kohl’s was recommended in AI-assisted search when asked “What is the official Kohl’s website?” The site itself appears to use AI image generation and could steal victim funds or payment details. 

The speed at which AI search assistants must internalize new data for search can lead to bad data seeping in. Malicious items can be optimized for AI-assisted search to surpass legitimate results in answers. In February 2026, a tech journalist was able to fool multiple major search engines into saying that he was the best tech journalist when it came to eating hot dogs. He achieved this with a single page on his own website making the claim, dressed up with realistic GEO techniques. This demonstrates how easily major search engines can be a vector for bad information.  

The first link given in a Copilot chat response was a phishing page impersonating Wells Fargo’s login portal at userportal[.]wellfargoss[.]com/login[.]php. r

There is a risk of even more widespread harm if malicious sources get absorbed into common training sets. The effects of this have already been observed where repetitive tropes in low-diversity training sets can lead to repeated sharing of the same information, disproportionately to its relevance.  

In May 2026, researchers published a paper called “Elias in the Lighthouse, Again?” exploring how AI can amplify narrow inputs from their training sets to become common tropes of their outputs. The paper focuses on fiction generation as one of the most obvious examples, but this phenomenon could also be seen in factual answers. While not widespread, there is a realistic possibility that this could become a targeted vector by threat actors in the near‑to‑medium term. 

As answer engines become more ubiquitous in the average user’s web experience and more integrated into their browsing behavior, incorrect and malicious results become more dangerous. This goes beyond the risk of bad results posed by traditional search engines.  

A Fudan University study of LLM-enhanced search engines found that these engines filter out 99.78% of traditional black hat SEO attacks. However, it found that strategies built specifically for AI retrieval, such as rewritten query-stuffing and segmented text, double the rate at which manipulated content reaches answers. Generic search spam is caught at the retrieval stage, but content engineered for AI-assisted search is not. 

How brands can protect their customers 

Brands may advise customers to save the correct login link for important services, such as banking. Some of these services have historically offered personalized login displays, greeting users with a custom message they set, or an image decided during account setup as a sort of shared visual secret. However, these have generally been phased out and are not considered strong defences as users often don’t notice their absence. Instead, it is advised to identify scams using GEO to target you and proactively remove the impersonator’s infrastructure, disrupting the attack before it can take place.  

It is likely that AI and GEO abuse will become professionalised in the same way SEO abuse did. AI adoption is changing the rate of threat generation as well as how threats can be delivered to users. Answer engines are a part of the landscape that are here to stay, so it is worthwhile to know how your brand is being represented by the answer engines trusted by your customers. 

Don't want to miss out on updates?

Don't want to miss out on updates?

Don't want to miss out on updates?

Join our mailing list for regular blog posts and case studies from Netcraft.