Sunday, August 10, 2008

What is Google Infinity Space and how to get rid out of if?


Torrey Hoffman from Google Webmaster said: When Googlebot crawling website, it find website that have very large number of links that do not provide any useful content (information) to the crawler, this is called “infinity space”. Crawling these types of URLs use unnecessary bandwidth and Googlebot do not like to crawl this type of links. If your website having same unnecessary links then its possible that Google fail to crawl the original content that you like Googlebot to crawl.

Google recently started to notify webmasters about this problem through Google Webmaster Tool Message Center.

Shopping website is the example of Infinity space, as they provide many ways to filter result. Like you can filter result by brand name, price range, store name, color, size, gender, age, etc. So, Googlebot find thousands of unnecessary number of links.

To rectify this problem you can use robots meta tags to tell Google and other search engine not to crawl this page. You can use following robots meta tag to stop crawling such unnecessary link

<meta name="robots" content="noindex, nofollow">

This meta tag tell search engines not to crawl and follow links in this page.

Bhavesh Goswami.