Cloudflare now blocks Google by default
A September 15 change means some sites behind Cloudflare may be shutting Google out without the owner knowing.
Cloudflare, a service many websites use for speed and security, switched to blocking Google by default on September 15. That means pages sitting behind it can stop being crawled by Google, and pages Google cannot crawl can stop showing up in search.
The change was flagged by Glenn Gabe, who pointed to Cloudflare's own post from September 15 for the details. Gagan Ghotra called it a "really weird move by Cloudflare" and said the default should be that Google is allowed.
really weird move by Cloudflare
What Cloudflare's CEO said
Gabe was writing about an interview with Cloudflare CEO @eastdakota, who said that if Cloudflare blocks Google, it breaks search ranking, because roughly 20% of the web becomes invisible to Google and PageRank gets skewed. PageRank is Google's old measure of how pages link to each other.
Gabe pushed back on the 20% figure. That number would only hold if every single site owner using Cloudflare blocked Google, and that is not what is happening. So the honest reading is this. Not every Cloudflare site is blocking Google right now, but the default changed, so you cannot assume yours is fine.
The CEO's wider argument, according to Gabe, is that the deal between Google and publishers needs to change, because it is not a fair give to get anymore. He said deals with some large publishers are better now and he wants that extended to the rest of the web. Gabe notes the interview was recorded before news broke about a system for Google paying publishers for AI Search usage, which is being tested now.
What to do
If your site runs on Cloudflare, log in and check your bot and crawler settings to confirm Google is allowed. If you are not sure whether your site uses Cloudflare, ask whoever built or hosts it. If you are not on Cloudflare, do nothing.
Cloudflare switched to blocking Google by default on 9/15 so you should check your settings
Be the person who already knew
One email a day. Under three minutes. You will hear about a Google change before it shows up in your traffic.