#This is for every spider User-Agent: * #Stay away from this Disallow: /css/ Disallow: /img/ Disallow: /admin/ Disallow: /js/ Disallow: /newsletter/ Disallow: /scripts/ Disallow: /files/ Disallow: /newsgroups/ Disallow: /_autocomplete/