[Previous] [Next] - [Index] [Thread Index] - [Previous in Thread] [Next in Thread]

Subject: Re: UKNM: robots.txt - are you at risk?
From: Craig Pickup
Date: Wed, 28 Jun 2000 10:02:46 +0100

Tim wrote:

>Nothing listed in a 'robots.txt' file will turn up in a search query; but
>once a person has seen the directory and file names it contains, they can
>type them directly into their browser to access the various subdirectories
>and pages which the site administrators would rather keep hidden. These are
>of course the very subdirectories and files most likely to be of interest

Correct me if I am wrong (which is often the case) but the search engine
bots, spiders, etc will only index pages either specifically submitted or
linked to from somewhere else. If it ain't linked to the bot won't find it.
So if something is of such merit that you don't wish crackers to find it
then don't link to it (or they will find it anyway) and you will not need
to include it in the robots.txt file.

Craig Pickup

Digitrade Ltd
17 Gateway Business Centre
210 Church Road
London, E10 7JQ

Tel: 020 8988 9660 Fax: 020 8988 9661

the UK's totally managed affiliate marketing solution.
ukaffiliates.com >> the net.working
http://www.ukaffiliates.com / salesatukaffiliates [dot] com (mailto:salesatukaffiliates [dot] com)
telephone: 020 7691 1880 / fax: 020 7691 1881
To unsubscribe or change your list settings go to
http://www.chinwag.com/uk-netmarketing or helpatchinwag [dot] com

  RE: UKNM: Email list management ?, paul youlten
  UKNM: robots.txt - are you at risk?, Tim Ireland

[Previous] [Next] - [Index] [Thread Index] - [Next in Thread] [Previous in Thread]