[Previous] [Next] - [Index] [Thread Index] - [Previous in Thread] [Next in Thread]


Subject: Re: UKNM: robots.txt - are you at risk?
From: Lee.Rickler
Date: Tue, 27 Jun 2000 19:32:11 +0100

hhhmmm ... very similar article to:
http://howtocc.hypermart.net/articles/robots.html

Also ...

robots.txt files are fine but for real security simply put an index.htm
file into the folders that access is denied to.
write in a bit of code that redirects to another page.

<!- Lee Rickler ->
<!- Technical Director ->
<!- Joshua Active Media ->

"Tim Ireland"
<[email protected] To:
<uk-netmarketingatchinwag [dot] com>
city.com> cc:

An interesting news article from The Register:
http://www.theregister.co.uk/content/6/11174.html

"Our friend 'fravia+' recommends searching for this file, called
robots.txt,
in the main directory of a target site, by entering a URL with the
following
pattern: http://www.targetsite.com/robots.txt. The robots.txt file is used
to tell search engines which directories and files they should not index.
Nothing listed in a 'robots.txt' file will turn up in a search query; but
once a person has seen the directory and file names it contains, they can
type them directly into their browser to access the various subdirectories
and pages which the site administrators would rather keep hidden. These are
of course the very subdirectories and files most likely to be of interest
to
crackers."

I had a bit of a poke around with this, and AFAICT the most common method
used to beat this amateur hack is to 'hide' this sensitive info in a
folder
and instruct the robot not the enter it (rather than provide an exact file
name that it should avoid).

Of course if you have nothing to hide, you could borrow this little gem:

# Don't even bother trying to use our "robots.txt" file to hack us - we
# aren't that stupid!

;o)

Tim Ireland
www.designercity.com
www.another.com
www.buymybook.co.uk


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
the UK's totally managed affiliate marketing solution.
ukaffiliates.com >> the net.working
http://www.ukaffiliates.com / salesatukaffiliates [dot] com (mailto:salesatukaffiliates [dot] com)
telephone: 020 7691 1880 / fax: 020 7691 1881
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To unsubscribe or change your list settings go to
http://www.chinwag.com/uk-netmarketing or helpatchinwag [dot] com



Replies
  Re: UKNM: robots.txt - are you at risk?, Chris Garrett
  Re: UKNM: robots.txt - are you at risk?, Kief Morris

[Previous] [Next] - [Index] [Thread Index] - [Next in Thread] [Previous in Thread]