Lets you post a sitemap of your web site to Google. A sitemap is sort of a map that helps Google locate and recognize the many pages on your website. txt file is then parsed and can instruct the robotic regarding which pages will not be to generally be crawled. https://www.youtube.com/watch?v=jfBrROL69Lk