Introduction

In order to evaluate and implement a search application it is important to understand the basic elements of search technology, which form the basis of both search software and search application products. There are a number of distinct elements in the process, from capturing content to presenting it within a browser on demand:

The challenge in selecting a search application is that every vendor carries out these processes in a slightly different way. Understanding the similarities and differences between the products on the market is important in ensuring that user requirements are fully accommodated. A good understanding of search technology is also important when it comes to implementing and ‘tuning’ a search engine. There will be a continuing need to look at the types of search being carried out, and the results they obtain, and then changing various elements of the search process (especially the way in which ranking is carried out) to ensure that users obtain the maximum number of relevant results.

Content definition

The initial task is to define the content that needs to be searched, and hence indexed. For smaller applications the index is generated on the server that also contains the content to be indexed, a common situation with website searching. The search query is then run against this index. However, the index to a set of documents may typically be between 15 and 50% of the file size of the content collection and querying the index also requires a substantial amount of processing.

As a result the indexes and query management are accomplished on a separate server. A number of options are then available.