There is one thing you have learned about robots, it is that there is
absolutely no pattern to them. Most robots are stupid and wander randomly.
For example, 50% of robot hits to my sites ask for the robots.txt page and
then go away never asking for anything else. Then they come back a week
later, ask for the same thing and then go away, again. This happens over
and over again for months. You will never never figure it out. What are
they doing? If they wanted to see if the Web site was really a Web site,
they could just Ping it. This would be much faster and much more efficient.
They seldom visit another page and if they do, they ask for one other page
every visit or so. Some come in and issue rapid-fire requests for every
page in the We View the rest of this article
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment