# robots.txt for http://www.wadra.law.ecu.edu.au/ #MSNBot is a piece of junk User-agent: msnbot Crawl-delay: 360 User-agent: Googlebot-Image # RufusBot? doesn't even exist yet. no crawling! User-Agent: RufusBot Disallow: / # Alexa/Amazon search. keeps mangling URLs and generally crapping in the logs User-agent: ia_archiver Disallow: / # no, we don't need our pictures indexed User-agent: psbot Disallow: / # twiceler, doesn't even exist yet. no crawling! User-Agent: twiceler Disallow: /