# Welcome All Search Engine Friendly Robots # robots.txt for http://www.taphunter.com/ # Welcome Humans as well # If you are interested in working with us, please see taphunter.workable.com. # If you notice something out of the ordinary, please email security@. # http://mj12bot.com/ User-agent: MJ12bot Disallow: / User-agent: * Disallow: /admin/ Disallow: /management/ Disallow: /ajax/ Disallow: /scripts/ Disallow: /tasks/ Disallow: /api/ Disallow: /bigscreen/ Disallow: /trck/ Disallow: /tracking/ Disallow: /user/ Disallow: /u/ Disallow: /beer/*/*/review/ Disallow: /beverage/*/*/review/ Disallow: /location/id/ Allow: / # Delay crawlers slightly to attempt to avoid rate limiting # Well behaved robots should not get rate limited # Bing and Yahoo honor this while Googlebot ignores it Crawl-delay: 1 Sitemap: https://www.taphunter.com/sitemap.xml