Perplexity Faces Scrutiny Over Stealth Crawling and Cloudflare Disputes
TL;DR
Cloudflare accused AI search engine Perplexity of stealthily scraping websites despite being blocked, sparking debate over whether AI agents should be treated like humans or bots. Many defend Perplexity, arguing that accessing public content on behalf of a user is acceptable, while Cloudflare criticizes the behavior as inappropriate. The controversy highlights broader issues about AI web crawling, website blocking, and the future of internet traffic, with concerns about malicious bots and the impact on website revenue and access.
- Some people are defending Perplexity after Cloudflare ‘named and shamed’ it TechCrunch
- Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives The Cloudflare Blog
- Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites The Verge
- Perplexity accused of scraping websites that explicitly blocked AI scraping Yahoo Finance
- Perplexity Says Cloudflare Is Blocking Legitimate AI Assistants Search Engine Journal
Want the full story? Read the original reporting
Read on TechCrunch