# robots.txt file for domain http://www.crane.navy.mil User-agent: Mozilla/4.0 (compatible; MSIE 4.01; Windows NT; MS Search 5.0 Robot) Allow: / # No robot will spider the domain User-agent: * Disallow: /