# /robots.txt file for http://www.btpnews.com User-agent: * #Disallow: / Disallow: /archive Disallow: /art Disallow: /email Disallow: /phpinfo.php User-agent: ShopWiki Crawl-Delay: 10 User-agent: Slurp Crawl-delay: 10 User-agent: Googlebot Crawl-delay: 10