Googlebot Meta Tags

Thursday, 20th November 2008.

Posted on Saturday, 15th October 2005 by Aleeya

Posted in Uncategorized | Comments (0)

Last Updated:2008-09-23 @ 8:44 pm

(2 votes, average: 3 out of 5)

2,016 views

Besides the keyword and description meta tags being the most important ones, there are some meta tags that are proprietary to Google’s own Googlebot. According to Google, the tags that would apply specifically to Google’s Googlebot are the ones that have name="googlebot" in the meta tag.

There are many reasons not to have a Search Engine cache a page. If it is copyrighted, if it contains private information that you do not want displayed from a search engine, or for any other reason you do not want the page crawled by a search engine, you can add the following code to the meta tags of a page and it will keep the page from being indexed:

If you would like specifically to keep Google from indexing the page, but allow other Search Engines to index it, you can place the following meta tags in your headers of the page:

On the other hand, if you would like the robots to index the file, but not follow the links on the page, you can use this code.

When the Search Engines spiders, bots or robots crawl and index a page, they take a snapshot of a page. This is called a cached copy or a cache of the page. If for some reason, the site is unavailable when you visit it, a Search Engine can show the cached copy of the page. Or, if the cached copy of the page has a different version, then for various reasons, someone might want to use this cached version. It will be what the page looked like at the time of the crawl. So, if a robot crawled the site at 12:00pm on Saturday, and Monday more articles were added and changed on the page that was cached, you could still view what the cached version of that page looked like if the last time the bot crawled it was before it was changed on Monday. If you do not want Search Engines to cache a copy of this page or retain an older version of the page, you can add the following meta tags to the header of the page:

If it is specifically Google’s Search Engine that you do not want to have a copy of that page, but you want the other Search Engines to have a copy, then you would use this meta tag in the headers:

Using the above meta tag for Google will result only in the cached copy not being stored. Google’s Googlebot will still continue to index the page with what is known as a snippet for that link on the Search Engine. A snippet is a preview of the webpage that is indexed. This is the preview that appears below the title of the page when you search on Google. If you set it to use no snippet, it will display what is in the Description Meta tag of the page instead of a snippet.
.
Note: The snippet will also remove the cached page:

Aleeya dotNet
Daily life information for the family and people of all ages. Household Tips,
DVDs, Books, Reviews, Search Engine Optimization, News, Amateur Radio and more …
www.aleeya.net/ - 34k - Oct 13, 2005 - Similar pages

Without a snippet using the Description Meta tag:

Below is a result for a search for Aleeya (in Google) and an example of what the results look like when it is not using a snippet or a Description Meta tag.

Aleeya.net: October 2005
… and kept the non technical site as http://www.aleeya.net . But, from previous,
Aleeya.net only had a 4/10 on Google’s page rank when I used PHPnuke. …
aleeya-net.blogspot.com/ 2005_10_01_aleeya-net_archive.html - 21k - Cached - Similar pages
[ More results from aleeya-net.blogspot.com ]

So, to set it so the Google Bots will index a page, but not use a snippet - but use the Description Meta Tag, use the code below:
This will remove the cache of the page:

There are many other meta tags available to use for HTML. The ones above are just a sample and also demonstrate the ones that are proprietary to Google.