User-agent: Slurp 
User-agent: Googlebot 
User-agent: msnbot 
User-agent: Mediapartners-Google 
User-agent: Adsbot-Google 
User-agent: Googlebot-Image
User-agent: Googlebot-Mobile
User-agent: Teoma
User-agent: ia_archiver-web.archive.org