# see # http://web.nexor.co.uk/mak/doc/robots/norobots.html # http://info.webcrawler.com/mak/projects/robots/robots.html. User-agent: * Disallow: */images Disallow: /images Disallow: /cgi-bin Disallow: */cgi-bin Disallow: */usr Disallow: */bin Disallow: */lib Disallow: */src Disallow: */home Disallow: */dev/ Disallow: /recursion Disallow: */RCS Disallow: */CVS Disallow: /chicken # test User-agent: Arachnophilia User-agent: webvac Disallow: /