In July 2025 a Netcraft audit of 50 well-known brands found that large-language-model chatbots still steer users to a troubling number of dead, incorrect, or malicious login pages. With 34% of URLs proving unsafe, the study highlights both a security exposure for consumers and a brand-safety headache for marketers who count on AI visibility.
Executive Snapshot
- 34% of chatbot-suggested login URLs were inactive, unrelated, or overtly dangerous.
- 29% were unregistered or parked, leaving them open to domain hijacking.
- 5% redirected users to completely different businesses.
- Community and regional banks experienced a 2.4× higher error rate than the top 10 banks reviewed.
- IBM estimates the average phishing breach costs US$4.76 million in remediation.
Bottom line: Neglecting AI surfacing can erode conversions and introduce measurable security liability.
Methodology
The July 2025 test covered 50 brands across banking, retail, crypto, and streaming. Researchers submitted natural-language queries such as "Where do I log in to X?" to GPT-4.1-based assistants, then manually evaluated the 131 unique hostnames returned for registration status, SSL configuration, and ownership.
Limitations include English-only prompts, a single LLM version, and no testing of voice assistants or regional models. Results may shift as models update.
Key Findings
Login URL Accuracy
- Inactive or parked domains – 29%
Example: “paypallogin-secure[.]com” returned NXDOMAIN yet was available for registration. - Phishing or unrelated sites – 5%
Perplexity.ai directed users to a spoofed Wells Fargo page hosted on Google Sites. - Correct brand-owned domains – 66%
Accuracy peaked at 82% for Fortune 100 firms and bottomed out at 42% for community banks.
Growth of AI-Optimized Phishing Infrastructure
- Netcraft catalogued 17,000 GitBook pages and more than 50 GitHub repositories seeded specifically to lure LLMs into citing fake Solana APIs.
- Google Safe Browsing now flags roughly 450,000 new phishing URLs per day, a 12% year-over-year increase.
Cost Context
IBM Security’s 2023 study pegs phishing-led incidents at an average of US$4.76 million, 11% above the global breach baseline.
Implications for Brand Protection and Marketing Strategy
Likely Actions
Defensive domain registration is no longer enough. Brand teams need automated monitoring of LLM outputs and rapid takedown workflows to curb AI-driven phishing.
Tentative Tactics
Early evidence suggests that structured data (schema.org “LoginAction”) and authoritative “.well-known/change-password” endpoints improve LLM recall, though controlled testing remains sparse.
Speculative Developments
Paid placement in emerging “verified links” programs for AI assistants could soon resemble today’s domain-portfolio budgets.
Conversion Funnel Impact
A 34% misdirection rate at the login step can meaningfully dent revenue. Assuming a 3% conversion value per returning user, misrouted traffic erodes margin faster than a typical SEO ranking loss.
Contradictions and Research Gaps
- Google’s Gemini (July 2025) claims URL verification, but independent audits have yet to confirm efficacy.
- No longitudinal data shows whether fine-tuning or RLHF reduces misdirection for niche brands.
- Regional language prompts (Spanish, Hindi, etc.) were excluded, leaving non-English performance uncertain.
Data Appendix
Selected Brand Error Rates
- Bank of America – 12%
- Wells Fargo – 24%
- Midwest Community Credit Union – 58%
- Amazon – 8%
- Disney+ – 14%
Sources
- [S1] Netcraft, “Chatbots and the Phantom Login Problem,” July 2025.
- [S2] IBM Security, “Cost of a Data Breach Report 2023,” July 2023.
- [S3] Google Safe Browsing Transparency Report, accessed June 2025.
.svg)
.svg)
.svg)