View Sidebar

Post Tagged with: sitemap

Robots & Sitemaps

Robots & Sitemaps

robots.txt This file is used to inform the search engine crawler what folder and files of our site you want to crawl or don’t want to. Robot file is normally located on the root of the website. A typical robots file would contain following to allow crawler to view entire […]

February 20, 2013Read More