SEARCH ENGINE
FENG SHUI

Technical

Noindex and robots.txt: decide what stays accessible and indexable

Preventing crawling and requesting exclusion from results are different decisions. First decide whether a page should remain available to readers.

Specify the expected outcome

Classify pages by purpose: public guide, temporary filter result, bibliographic record, private area or removed content. Define the expected reader and search-engine outcomes. Private documents need access control; a crawler instruction does not protect confidential information. Removed content needs an appropriate missing response, not a blank screen served as a normal page.

Avoid contradictory instructions

Google’s documentation states that page instructions must be accessible to be read. Blocking access while expecting an engine to read noindex in the document is a practical contradiction. A canonical proposes a preferred version; it is neither protection nor an exclusion instruction. Check suitable headers for non-HTML resources rather than imagining a meta tag inside a file that has none.

Inspect the response actually served

Check status, headers and the document received by an authorised client, then the rendered version when scripts change it. Ensure a template change has not unintentionally affected every page. Keep your editorial sitemap consistent with pages intended for discovery. An instruction’s presence does not prove immediate search-engine processing: distinguish checked configuration from later observations.

A four-step method: Purpose → Reader access → Instruction → Checked response
Original infographic — a process, not a statistical result.
  1. Purpose
  2. Reader access
  3. Instruction
  4. Checked response

Worked example

In this edition, a record preserving only a historical title and context remains readable with noindex,follow. A substantial original guide is included in the sitemap. This is an explicit editorial decision, not a promise that every guide will be indexed.

Check before moving on

These checkboxes are not saved.

Sources & method

Sources reviewed on 1 October 2026. Worked calculations use explicitly hypothetical inputs.

Topic provenance: Utiliser la balise NOINDEX, les pistes de Google (2008-02-26). Search Engine Feng Shui. Byline retained in the consulted version: François Houste.

This guide is newly written in 2026. It does not reproduce the earlier text and is not attributed to its historical author.

Keep reading

All guides ↗