AI bot & crawler traffic
Copy linkAI bots scan public websites to collect data used in search engines, AI model training, and other data processing tasks
Latest AI blog posts
Copy linkPosts tagged with "AI" from the Cloudflare blog
Rust-powered core systems, post-quantum upgrades, developer access for students, PlanetScale integration, open-source partnerships, and our biggest internship program ever — 1,111 interns in 2026.
Nikita Cano, Korinne Alpers
Cloudflare will soon automatically create an AI-optimized search index for your domain, and expose a set of ready-to-use standard APIs and tools including an MCP server, LLMs.txt, and a search API.
Celso Martinho, Anni Wang
It turns out we've all been using MCP wrong. Most agents today use MCP by exposing the "tools" directly to the LLM.
Kenton Varda, Sunil Pai