Prepare a minimal extract
Use a defined period and necessary fields: time, path, status and technical identity to verify. Mask personal parameters and avoid publishing visitor IP addresses. Record retention limits, time zone and missing periods. Log requests are neither unique people nor search-result positions.
Verify identity and intermediaries
“Googlebot” in a user-agent string does not establish identity. Follow Google’s official verification procedure instead of trusting a string match. Behind a CDN, a logged address can belong to a proxy. Trust an origin header only within an explicitly authorised proxy configuration; direct clients can send arbitrary headers.
Find a specific repair
Group requests by URL and response. Repeated 404 requests for an old address can justify looking for a genuine equivalent. They do not justify redirecting every missing page to the homepage. Separate resources, documents and parameter variants. Repeat observations after the change with a comparable scope. A crawler request establishes access, not successful indexing.
- Minimal extract
- Verified identity
- Specific defect
- Repeat observation
Worked example
Fictional example: in one day /guide/ returns twenty 200 responses while /old-guide/ returns eight 404 responses. First establish whether the old page has an equivalent. If it does, test a suitable redirect. Twenty responses do not mean twenty readers or twenty indexing events.
Check before moving on
These checkboxes are not saved.
Sources & method
Sources reviewed on 1 October 2026. Worked calculations use explicitly hypothetical inputs.
Topic provenance: Wordpress Crawl Rate Tracker Plugin : guettez les robots sur votre blog (2008-02-25). Search Engine Feng Shui. Byline retained in the consulted version: François Houste.
This guide is newly written in 2026. It does not reproduce the earlier text and is not attributed to its historical author.