News

Cloudflare is launching a new marketplace that reimagines the relationship between publishers and AI companies.
Cloudflare’s Pay-Per-Crawl is a clever idea. It’s the first genuine attempt to attach a meter to data before it gets ...
The age of the AI scraping free-for-all may be coming to an end. At least if Cloudflare gets its way.
Cloudflare reshapes the AI landscape by blocking Big Tech AI bot crawlers by default, allowing websites to demand payment for content access.
Cloudflare takes a stand on AI crawlers, which could enable publishers to finally make peace with the bots scooping up their ...
The Impact of Cloudflare's AI Bot Block Cloudflare, one of the biggest internet infrastructure providers, has launched a new AI Bot Block feature, heralded as a potential ‘game-changer' for content ...
Cloudflare is using Workers AI to generate this content ahead of time. The pages are stored in R2 storage for fast retrieval, and care is taken to prevent cross-site scripting vulnerabilities.
The era of unrestricted AI crawling appears to be ending - well, at least for the fifth of the internet that flows through Cloudflare's pipes.
Roughly 16% of global internet traffic goes directly through Cloudflare's CDN, the firm estimated in a 2023 report. "AI crawlers have been scraping content without limits. Our goal is to put the ...
Google declined Ars' request to confirm whether talks were underway or if the company was open to separating its crawlers.