# robots.txt, Site Policy versus robots, spiders and so on...

User-agent: *
Disallow: /cgi-bin/