Features

The main features of the Index Server Companion are:

• Enables Index Server to allow searching of potentially any web server or ODBC compliant database.

• Integrated web robot extracts content from websites. Includes support for robots.txt files and robots meta tags. Robot can negotiate sites using HTML Frames. Optional mode allows QueryStrings to be treated as distinct URLs (e.g. treat http://www.aspalliance.com/brettb/WebJobMarket.asp?Skill=ASP as being a distinct URL from http://www.aspalliance.com/brettb/WebJobMarket.asp?Skill=JSP).

• Ability to retrieve binary files from servers, including Adobe Acrobat PDF, Microsoft Office documents and even images.

• Support for full or incremental project updates of both web and database content, meaning that Index Server only has to re-index content that has changed.

• Configuration of the Index Server Companion is through the editing of a plain text configuration file.

• Index Server Companion can be run from the command line, and scheduled using the Windows Task Scheduler.

• Full reporting of activity to an external plain text log file.

• Flexible output options mean that administrative access to Index Server is not necessarily required.

• Facility for creating a basic table of contents page for the sites that are crawled [sample from my own website, sample from Lachesis.biz].

• Facility for creating a site summary page that can be used to optimise sites for search engines, by showing what keywords are used on a site, the <title> tags used by each page etc. The site summary page shows:
  • The top 40 most used keywords listed in the keywords meta tags in all of the pages crawled.
  • The body of the summary page contains the following from the pages encountered in the web crawl:
    • A list of page titles.
    • A list of page headings (H1 to H3 inclusive).
    • A list of bold (and strong) text.
    • A list of external hyperlinks in the site.
    • A list of italicised text.
    • A selection of relevant paragraph text. Paragraphs are assumed to be relevant if they contain one or more of the 40 most frequently used keywords.
    • A list of page description meta tags.
    • A list of page keywords.
    • A list of page description meta tags hyperlinked back to the URL from which each description was extracted.
    • A list of page keywords hyperlinked back to the URL from which it was extracted.

• Fully documented VBScript examples show how to make use of the Index Server Companion in ASP pages. There is also an example showing how to use the Index Server Companion to create a website search facility using ASP.NET and the C# programming language [including usage of the DataGrid control with Index Server search results].

• Detailed documentation in Microsoft's HTML Help format.

• Fully documented Perl source code.

• Access to product updates and technical support.

Figure 1. The Index Server Companion contains fully searchable documentation in Microsoft's HTML Help format.

System Requirements

The Index Server Companion can be used on a machine running Microsoft Windows 95 or any subsequent version of Windows. Windows NT 4.0 or Windows 2000 is recommended.

It also (of course) requires a server running either Index Server on Windows NT 4.0 Server, or the Indexing Service on Windows 2000. Note that Index Server Companion does not have to be run from the machine on which the Index Server is installed.