If you have access to your web server's log files then it is
extremely worthwhile spending time examining them in order to identify
Even if you use a website's statistics analysis application,
there is a good chance that irregular behavior will go unreported. This is
largely due to the fact that most websites' statistics packages are concerned
with analyzing the typical usage of website users, rather than flagging up
non-standard website usage.
For example, I once worked at a large organization that was
periodically experiencing regular surges in the number of website visitors. Although
the commercial website's statistics package in use by the organization was able
to identify when the peaks in traffic occurred, it took a manual analysis of
the IIS log files to discover the cause of the traffic peaks.
Although manual examination of web server logs is effective,
it is worthwhile investigating the various automatic forensic log tools
available. A particularly useful utility is Microsoft's Log Parser. This
utility allows any type of log file to be queried by using a standard SQL
syntax, making it a powerful tool for extracting, sorting and displaying
summaries of activity from web server log files. Log Parser may be obtained from
the Microsoft download site or from http://www.logparser.com/.
A particularly useful article about using Log Parser to examine web server log
files for abnormal activity is "Forensic Log Parsing with Microsoft's
LogParser" at SecurityFocus, http://www.securityfocus.com/infocus/1712.
When manually looking at log files, it is also recommended
to use a text editor that offers a fuller feature list than the Notepad
application supplied with Windows. Two text editors that are particularly good
at handling log files are TextPad (http://www.textpad.com/)
and UltraEdit (http://www.ultraedit.com/).
Both of these editors will open files much larger than those that can be
opened in Notepad. They also offer more sophisticated search facilities and
allow specific lines to be pasted into new documents.
Unfortunately, there does not appear to be any software
packages available that will automatically analyze log files in order to
identify suspicious activity. To a certain extent this may be because the determined
hackers with full access to the compromised system will often modify the log
files so that they are undetected anyway. It is, however, possible to obtain
Intrusion Detection Systems (IDS) that can be used to identify suspicious
activity in near real time.