#
# URL: http://wwwoud.niwi.knaw.nl/robots.txt
#
# Contents: robots.txt for http://wwwoud.niwi.knaw.nl/
# Maint.  : webmaster@niwi.knaw.nl
#
# This file specifies which WWW robots are (dis-)allowed to access what.
# The latest specs for the contents of a robots.txt file can be found at
# the URL:
#
#     http://info.webcrawler.com/mak/projects/robots/robots.html
#
# At our site all robots (*) are allowed to access anything, except the
# sitestat counter cgi-bin (disallow /sitestat-public/sitestat.gif) for
# Linbot, which is our internal link-checking robot, run periodically.
# Furthermore, our site has been actively registered at a large number
# of catalog and index meta sites on the net or will be in the future.
# To see where this site has been registered and is to be registered, and
# how it has been registered (!), see URL:
#
#     http://wwwoud.niwi.knaw.nl/... to be done yet (if decided so)
#
# Questions and/or remarks can be send by e-mail to <webmaster@niwi.knaw.nl>.
# This WWW server became operational on 1997-09-01 (September 1, 1997).
# It was registered from ... to be done yet (if decided so)
#
User-agent: Linbot
Disallow: /sitestat-public/sitestat.gif
Disallow: /guests
Disallow: /us/column/help.htm
Disallow: /nl/column/help.htm
Disallow: /cgi-bin/opac_seek_journal.pl
Disallow: /cgi-bin/ejournals_seek.pl
User-agent: cache4.grnet.gr
Disallow: /us/ia2001/home.htm/
User-agent: *
Disallow: /pdf
Disallow: /pdf/
Disallow: /guests/derma-m/
Disallow: /cgi-bin/
Disallow: /us/ia2001/