How to Use Robots Meta Tags

A special meta tag can be put in a page’s HTML to give instructions to bots.

The form of the tag is:

<meta name="robots" content="noindex" />

The name attribute declares that the tag is for robots. The content attribute is where you put the instructions.

Here are some possible content values to become familiar with:

• noindex — this tells search engines not to display the page in the search results (in contrast with robots.txt which only gives instructions not to crawl the page).
• nofollow — this tells search engines not to follow any of the links on the page.
• none — if you see this, it means noindex and nofollow.
• noarchive — tells bots not to cache the page, for example, in Google’s cache.
• nosnippet — this tells Google not to show a text snippet or other preview in the search results.
• max-image-preview:large — this tells Google that it can display large preview images in their services like Discover. We’ll go over it in a separate tutorial.

See Google’s documentation for a full list of possible values.

A Caution about robots.txt and Robots Meta Tags

A common mistake that people make is to both block a page with robots.txt and use a noindex meta tag on the page.

That probably won’t work!

The robots.txt file will prevent Googlebot from fetching the page, so Googlebot won’t see the noindex meta tag. Since a robots.txt doesn’t prevent displaying the page in the search results, the page might show up in Google even though it has noindex meta tag.

If you really want to be sure that a page won’t show up in Google, use the robots meta tag or the X-Robots-Tag header, but don’t block it with robots.txt.

If you made a mistake with that and need to quickly remove content from Google, see these instructions.