Cloudflare Crawler Hints vs AutoIndexNow for IndexNow

How Crawler Hints works

Crawler Hints is a switch under Caching → Configuration, available on every Cloudflare plan. According to Cloudflare's documentation, it uses a cache status of MISS to determine when content has likely been updated and sends that to IndexNow. It is site-wide; Cloudflare suggests an X-Robots-Tag: noindex header (from your origin or a Transform Rule) for pages that should stay out. It needs no key file and no code.

Side by side

What triggers a notification: Crawler Hints
A cache MISS at Cloudflare's edge, which Cloudflare treats as a sign the content has likely been updated. It needs a request for the page to happen.; AutoIndexNow
A scheduled check of every URL in your sitemaps, whether or not anyone visited the page. A deploy hook or the API can start a check right away.
What counts as a change: Crawler Hints
Cache behaviour. A MISS also happens when a cached copy expired or was never cached, so it is a heuristic about change, not a reading of the content.; AutoIndexNow
The rendered page's normalized content and indexability, confirmed by a second render. Timestamps, counters and tokens never count.
New pages nobody visited yet: Crawler Hints
Only once a request reaches the edge.; AutoIndexNow
Found in the sitemap on the next check and submitted as new.
Control: Crawler Hints
One site-wide switch; Cloudflare's docs suggest noindex headers to keep individual pages out.; AutoIndexNow
Per-site ignore rules, per-URL submission, plan limits you can see, and a pause switch.
What you can see: Crawler Hints
Cloudflare's documentation does not describe a per-URL log of hints or endpoint responses.; AutoIndexNow
Per-URL history with the reason for each check, every submission attempt with the endpoint's status code, CSV and API access.
Setup and cost: Crawler Hints
A toggle on any Cloudflare plan, including Free. No key file. Your site must be proxied by Cloudflare.; AutoIndexNow
A key file on your host and a paid plan. Works with any host or CDN.

Aspect	Cloudflare Crawler Hints	AutoIndexNow
What triggers a notification	A cache MISS at Cloudflare's edge, which Cloudflare treats as a sign the content has likely been updated. It needs a request for the page to happen.	A scheduled check of every URL in your sitemaps, whether or not anyone visited the page. A deploy hook or the API can start a check right away.
What counts as a change	Cache behaviour. A MISS also happens when a cached copy expired or was never cached, so it is a heuristic about change, not a reading of the content.	The rendered page's normalized content and indexability, confirmed by a second render. Timestamps, counters and tokens never count.
New pages nobody visited yet	Only once a request reaches the edge.	Found in the sitemap on the next check and submitted as new.
Control	One site-wide switch; Cloudflare's docs suggest noindex headers to keep individual pages out.	Per-site ignore rules, per-URL submission, plan limits you can see, and a pause switch.
What you can see	Cloudflare's documentation does not describe a per-URL log of hints or endpoint responses.	Per-URL history with the reason for each check, every submission attempt with the endpoint's status code, CSV and API access.
Setup and cost	A toggle on any Cloudflare plan, including Free. No key file. Your site must be proxied by Cloudflare.	A key file on your host and a paid plan. Works with any host or CDN.

Which one you need

Crawler Hints is enough when your site is already proxied by Cloudflare, gets steady traffic, and you do not need to know what was sent or what the endpoints answered. It costs nothing to switch on.
AutoIndexNow fits when you need new pages announced before anyone visits them, want rotating widgets or cache churn not to count as changes, need a record of each submission and its response, or are not on Cloudflare.

Running both

Both may notify for the same URL. That is allowed, but repeated notifications for unchanged pages are what endpoints rate-limit. AutoIndexNow submits only confirmed changes, so the overlap is small; if submissions start getting 429 responses, switch one of the two off.

Hosting your key file on Cloudflare

Cloudflare Pages: put <key>.txt in the build output (for example public/) so it is served at the root.
Any proxied site, including Webflow, Featurebase and other hosted tools: a Worker with a route on exactly <your host>/<key>.txt answers that one path and lets every other request through. Workers are available on the Free plan.

// Route: <your host>/3f9a2c7e0b1d4e8fa6c5b4d3e2f1a0b9.txt  (only this path)
const KEY = "3f9a2c7e0b1d4e8fa6c5b4d3e2f1a0b9";

export default {
  async fetch() {
    return new Response(KEY, {
      headers: {
        "content-type": "text/plain; charset=utf-8",
        "cache-control": "public, max-age=3600",
      },
    });
  },
};

A Redirect Rule cannot do this: it only redirects, and a key file that redirects to another host is not something the IndexNow documentation covers, so AutoIndexNow does not accept it. Snippets can run code like this too, but they are not available on the Free plan.

Verify the setup

Open the key file URL and run the key checker. Then submit one URL with the free submitter and confirm a 200 or 202.

Sources checked on 2026-10-19: Cloudflare's Crawler Hints, Workers routes and Snippets documentation, and the IndexNow FAQ.

A 200 or 202 from an IndexNow endpoint means the request was accepted, not that the URL is indexed. Google does not participate in IndexNow.