# /robots.txt file for controlling indexing User-agent: * Disallow: / User-agent: webcrawler Disallow: / User-agent: googlebot Disallow: / User-agent: NextGenSearchBot Disallow: / User-agent: Teoma Disallow: / User-agent: msnbot Disallow: / User-agent: Slurp Disallow: / User-agent: ia_archiver Disallow: /