Template:INDEX/doc

Usage
This template sets the  behavior switch, a magic word which instructs search engines to index the page. The only difference between the template and the magic word is that the template may optionally display a message.

On English Wikipedia,  and   should only be used on User and User talk pages, where they override the default, which is not to index these pages.

Technical details
overrides $wgNamespaceRobotPolicies, but not. It obeys the $wgExemptFromUserRobotsControl variable. On English Wikipedia, MediaWiki:Robots.txt is used in lieu of $wgArticleRobotPolicies.

On English Wikipedia, the default robot policy for the entire wiki is to index all pages. This default has been overridden in these namespaces: Additionally certain selected pages, primarily in Wikipedia and Wikipedia talk, are not indexed, as defined in MediaWiki:Robots.txt.
 * User talk is not indexed, as of September 2008
 * Draft and Draft talk are not indexed, as of December 2013
 * User is not indexed, as of November 2015

Mainspace, Draft and Draft talk are exempt from user control, so  is disabled in these namespaces; such use is not tracked (see below), and this template consequently has no effect there.

As all other namespaces are indexed by default, and robots.txt cannot be overridden by, the only namespaces where it overrides the default are User and User talk; however use in other namespaces is tracked.

Usage tracking
This template was originally used as a tracking mechanism for  usage; valid uses are now tracked by Category:Indexed pages.

History

 * Added $wgNamespaceRobotPolicies to allow customisation of robot policies on a per-namespace basis (June 4, 2006)
 * Added $wgContentNamespaces, which allows for articles to exist in namespaces other than the main namespace, and still be counted as valid content in the site statistics (June 12, 2006)
 * Added $wgArticleRobotPolicies (May 31, 2007)
 * These override the per-namespace robot policies; en.wiki uses MediaWiki:Robots.txt in lieu of this configuration setting
 * The default robot policy for the entire wiki is now configurable via the $wgDefaultRobotPolicy setting (February 6, 2008)
 * The default policy is to encourage indexing and following of links:
 * Wikimedia blog: robots.txt (April 29, 2008)
 * (April 29–30, 2008)
 * : add user talk namespace of En.Wikipedia to Robots.txt (April 30, 2008)
 * Added $wgExemptFromUserRobotsControl (July 23, 2008)
 * New magic words __INDEX__ and __NOINDEX__ control whether a page can be indexed by search engines, Wikipedia Signpost, July 28, 2008
 * Template:NOINDEX is created on August 9, 2008‎
 * Template:INDEX is created on August 30, 2008‎
 * Search engine indexing updates (Sept 13, 2008)
 * Control en.wiki's robots.txt file from the wiki at MediaWiki:Robots.txt
 * Update InitialiseSettings.php:, implementing
 * Search engine indexing, a January 2009 proposal which failed to achieve consensus
 * : Tracking category for the NOINDEX and INDEX behavior switches (Jan. 11, 2009)
 * On February 22, 2009, NOINDEX begins populating Category:Wikipedia noindex pages. INDEX does not populate any categories
 * (Feb. 27–March 5, 2009)
 * Add tracking categories for __INDEX__ and __NOINDEX__ • code review (Sept. 20, 2009)
 * Category:Noindexed pages is created on October 7, 2009; concurrent with this discussion. See also:
 * Category:Indexed pages is created on October 9, 2009
 * Per, Category:Wikipedia noindex pages is deleted on July 18, 2010
 * Create "Draft" namespace on the English Wikipedia (Dec 17, 2013)
 * : prevent search engines from indexing userspace on enwiki (July 5, 2015)
 * noindex user namespace on en.wikipedia.org (Nov 23, 2015)

MediaWiki

 * MediaWiki:Robots.txt
 * mw:Manual:Configuration settings
 * Default robot policy (github.com)
 * Default robot policy (git.wikimedia.org)