Monday, 19 November 2007

SE XML Sitemap Autodiscovery using robots.txt

Tell Google, MSN, Yahoo, etc, how to find your XML Sitemap with Sitemaps Autodiscovery. This is something I have just implemented at work, not a tool as such but I thought it was a pretty important/useful piece of information.

Search engines will check your robots.txt file for a link to an XML sitemap file, then get the file from that location. This is a big plus because all the major search engines regularly check robots.txt files as part of their ordinary crawling. My Robots.txt file now does all the work for me in telling all the major search engines where to find my XML sitemap.

Search Engines Unite On Sitemaps Autodiscovery

No comments: