# This file contains User-Agent mappings for robots # #WWW Search Engines #Googlebot http://www.googlebot.com/bot.html (Google) #FAST-WebCrawler http://fast.no/support/crawler.asp #Slurp http://help.yahoo.com/help/us/ysearch/slurp/index.html (Yahooo) # #Robots and crawler to be ignored # #User agent: ?? #Disallow: / # #Allow all other agent's but exclude the following areas # User-agent: * Disallow: /myaccount/ Disallow: /errors/ Disallow: /core/