Stopping Automated Web Robots Visiting ASP/ASP.NET Websites
page 3 of 6
by Brett Burridge
Average Rating: This article has not yet been rated.
Views (Total / Last 10 Days): 51464/ 73

Why robots are a problem

There are a number of problems associated with robots.

Large amounts of web robot traffic cause an increase in the bandwidth consumed by the website. On top of the increased financial cost of bandwidth, the bandwidth usage can reduce overall server performance, especially if the robots are making large numbers of requests to resource intensive pages, such as database search results pages.

Automated web traffic can distort website statistics, especially if there is a large amount of robot traffic or the robot traffic varies significantly from month to month.  This can lead to awkward questions from senior management if they notice unusual traffic peaks.  It also makes it difficult to gauge the success of marketing campaigns, etc.

The robots may be doing something with your website's content.  Stealing website content and republishing it in order to benefit from pay per click advertising is a highly profitable industry.

View Entire Article

User Comments

No comments posted yet.

Product Spotlight
Product Spotlight 

Community Advice: ASP | SQL | XML | Regular Expressions | Windows

©Copyright 1998-2021  |  Page Processed at 2021-11-29 9:43:33 AM  AspAlliance Recent Articles RSS Feed
About ASPAlliance | Newsgroups | Advertise | Authors | Email Lists | Feedback | Link To Us | Privacy | Search