Usually one of three causes: AI crawlers are blocked before they reach your pages, your robots.txt tells them to stay out, or the pages load but the product facts on them aren’t machine-readable. All three fail silently – the store looks normal to you, to your customers, and to Google.
Cause 1: blocked at the edge. Cloudflare has blocked AI crawlers by default on new domains since July 2025, and firewall plugins like Wordfence and Sucuri ship with bot rules that catch them. An edge block fires before robots.txt is ever read, so a perfect robots file proves nothing. Fixes: How Do I Allow AI Crawlers Through Cloudflare?, Why Is Wordfence Blocking GPTBot and Other AI Crawlers?, How Do I Allow AI Crawlers Through the Sucuri Firewall?, Does Kinsta Block AI Crawlers?.
Cause 2: robots.txt. A previous developer or an SEO plugin may have added blanket bot disallows. Fix: How Do I Allow AI Crawlers in robots.txt – and Which Ones Should I Allow?.
Cause 3: unreadable pages. Adobe’s July 2026 AI Content Visibility benchmark found about a third of retail product-page content is not machine-readable – facts locked in JavaScript tabs and images that a model never sees. A crawler that reaches an unreadable page still can’t recommend the product on it.
There’s a fourth, quieter cause: ChatGPT’s organic answers pull heavily from Bing’s index. A store that never bothered with Bing Webmaster Tools is thin or absent in the index ChatGPT reads.
Our free plugin, Blaze AI Discovery, checks all four in about two minutes: per-crawler access verdicts, robots.txt rules, product readability scores, and feed status.