Configuring and Running the Index Server Companion
The Index Server Companion executable file or Perl script needs to be run from the Windows command line. Fortunately there is only a single mandatory parameter, which tells the script which configuration file to use. So to run the Index Server Companion for the Sample Project, an MSDOS Command Prompt is opened in the folder where the Index Server Companion files are installed installed and the following is typed:
IndexServerCompanion.exe --c="SampleProject/SampleProject.ini"
It is of course possible to run the Index Server Companion from .bat scripts, which can then be scheduled using the AT command or the Windows Task Scheduler. This makes it straightforward to update the Index Server's index of website and database content at specific times and frequencies.
The configuration file (in this instance it is called SampleProject.ini) is a plain text file containing a number of settings.
The Index Server Companion is supplied with full documentation in Microsoft's HTML Help format that describes each of the configuration settings.
When the script is run, the Index Server Companion will display details of its status in the Command Prompt window. A detailed log file is also created.
How the Index Server Companion Works
The Index Server Companion script contains a fully functional web robot that is able to extract the content from all of the required pages of the specified website. It contains support for the Robots Exclusion Standard specification, and support for the robots meta tag contained within individual pages. Each file extracted from the website is modified to contain a special meta tag that give the original URL (for web content). It is then saved to disk from where it can be indexed by Index Server. The contents of these special meta tags can then be used by the ASP page displaying the results of a web search, so that clicking on a search result item will display the original URL. Unfortunately Index Server will not allow you to retrieve the content from custom meta tags without making a minor modification in the Index Server's Microsoft Management Console (MMC), so there is also a special mode in the Index Server Companion that appends the original URL into the page's HTML <title> tag.
The Index Server Companion is also able to index content from database tables, queries (Microsoft Access) and stored procedures (SQL Server). Database connectivity is achieved through the use of ODBC, so potentially any type of database that has an ODBC driver is supported.
Searching Web Content with the Index Server Companion
Index Server Companion allows content from remote websites to be retrieved and consequently indexed by Index Server. A working example of this may be seen on my website at the following URL:
This is a search page running on Internet Information Server 4.0 (Windows NT 4 Server) that allows you to search my ASPAlliance site (including the article you are presently reading!), together with the articles I have written for Ariadne.ac.uk and ASPToday.com. Since I don't have administrative access to the Index Server on the machine hosting the search page, I have used the feature of the Index Server Companion that allows the document's original URL to be appended to the original title. For example the <title> tag of the ASPToday article "ASP Documentation Systems" at http://asptoday.com/content.asp?id=1435 is modified in the file saved to read:
<title>ISC_URL=http://asptoday.com/content.asp?id=1435 ASP Documentation Systems</title>
The URL and original title are separated by a tab character. The search results page then contains a small piece of ASP code to split this title back into the article's URL and original title.
The ASP code for the sample search page may be seen below:
Searching Databases with the Index Server Companion
The Index Server Companion is able to index content from database tables, queries (Microsoft Access) and stored procedures (SQL Server). It is of course entirely possible to search databases using Structured Query Language (SQL), but by making use of Index Server Companion, it is a lot more straightforward to integrate database searches with Index Server search results from web page searches. There are also other advantages: Index Server contains sophisticated pattern matching, and it is often lot faster at returning search results than an equivalent SQL statement would be when using a database such as Microsoft Access.
Index Server Companion is able to retrieve the rows of a specified database table and make an HTML file containing the data from a specific database row. Index Server can then be used to index the HTML files, and it is possible to extract the details of the table and row from which the data originated so that the search results page can be modified to point to the original database data. A sample page produced from the SQL Server sample pubs database is shown below:
<html>
<head>
<meta name="ISC_title_id" content="MC2222">
<meta name="ISC_title" content="Silicon Valley Gastronomic Treats">
<meta name="ISC_type" content="mod_cook ">
<meta name="ISC_price" content="19.99">
<meta name="ISC_pubdate" content="6/9/1991 12:00:00 AM">
<meta name="ISC_notes" content="Favorite recipes for quick, easy, and elegant meals.">
<meta name="description" content="Favorite recipes for quick, easy, and elegant meals."></head>
<title>Silicon Valley Gastronomic Treats</title>
<body>
</body>
</html>
In this example, the title field is optionally used to give the page a title, and the notes field is used for the description meta tag.
Each of the custom ISC_ prefixed meta tags can be queried using Index Server, although to retrieve their contents a minor configuration change to Index Server is required. It is straightforward to create a page which for example, will return the records where the value of the ISC_type meta tag is "mod_cook".
The Index Server Companion can also modify the HTML's <title> tag to include the table name and row ID, e.g.:
<title>ISC_Table=titles ISC_KeyField=title_id ISC_RowNumber=MC2222 Silicon Valley Gastronomic Treats</title>
Summary
The Index Server Companion allows Microsoft Index Server to index content from remote websites and ODBC databases, making it a cost effective way of significantly extending the functionality of Index Server.
