# Keep Fasterfox out User-agent: Fasterfox Disallow: / # All robots User-agent: * Disallow: /admin Disallow: /inloggen Disallow: /fotos Disallow: /_ Allow: /contact.asp Allow: /fotos.asp Allow: /default.asp Allow: /kaart.asp Disallow: / Crawl-delay: 120 # found on Wikipedia Request-rate: 1/60 # maximum rate is one page every 5 seconds Visit-time: 0200-0700 # only visit between 06:00 and 08:45 UTC (GMT)