Block AI training without losing Google search traffic
Cloudflare split AI training from search crawling, but picking the wrong option can hide your site from Google entirely.
Cloudflare added a setting called Disallow AI Training that tells crawlers not to use your content to train AI models while still letting them crawl your site for search, explains Alex Groberman. Until now, turning off AI training on some crawlers could also knock you out of search results.
do not confuse Disallow AI Training with Block
Applebot, Googlebot and Bingbot are what Cloudflare calls mixed-use crawlers
The reason is that Googlebot, Bingbot and Applebot each do both jobs with one crawler. Cloudflare calls these mixed-use crawlers. The new setting publishes a no-training preference in your robots.txt file (the small text file that tells crawlers what they may access) while still letting those crawlers in for search. Cloudflare says Apple, Google and Microsoft currently qualify as what it calls Accountable crawlers. For OpenAI, Anthropic, Amazon and Meta, search and training already run on separate crawlers, so the training one can be blocked on its own.
The button you must not press
There is a trap. Cloudflare changed what its Block option does. Groberman warns that if you pick Block for Training, mixed-use crawlers including Googlebot, Bingbot and Applebot can now be blocked completely, search access included. That means your pages could disappear from Google. If you want to stay in search, choose Disallow AI Training instead.
Two more limits are worth knowing. Google-Extended, Google's own training control, can be disallowed without affecting Google Search rankings, but Google says it also affects whether your content can be used for grounding in Gemini Apps. And Bing does not yet support Cloudflare's new robots.txt no-training preference automatically. Microsoft says support is targeted for early 2027, which lines up with the earlier move toward a single robots.txt signal for AI training.
What to do
If your site sits behind Cloudflare and you want out of AI training, open your Cloudflare AI crawler controls and select Disallow AI Training, not Block. If you are happy for AI tools to read your content, do nothing.
Groberman's wider point is that blocking everything is rarely the right call for a small business. Some AI systems send customers to you, some help customers act, and some just take your words for training. Those are now separate decisions, and this change is the first time most site owners can make them separately.
Be the person who already knew
One email a day. Under three minutes. You will hear about a Google change before it shows up in your traffic.