Search engines use very fast computers to help users locate information on the Web. Essentially, the user predicts words that are likely to appear on suitable Web pages and the search engine looks for a match. Search engines use raw power to match search terms and, whatever is claimed, do not display much intelligence.

Selecting the best search engine for a particular purpose is an important search skill and this page will indicate the types of engine and how they work. Only a selection can be included here and many useful engines have been omitted for space reasons.

Easy to Use - Google is straightforward to use and often effective
Large indexes - largest Web coverage with advanged search facilities
Multi-search - use several engines at once to cover more of the Web
How they work - a concise description to aid searching
Issues - why search engines can return too little or too much!

Google - easy to use http://www.google.com

A new search engine that even looks easy to use! Google makes searching straightforward by keeping the rules simple. Each word typed into the search form must appear in the target page (ie an AND search). Short 'stop' words such as a, the, but, I are ignored. Phrase searches eg. "pond life" are supported.

The usefulness of the sites returned by Google is improved by measuring the popularity of sites and by looking for sites in which the search terms are close together. The advanced search option is remarkably easy to use. Google's ease of use and its effectiveness make it an excellent starting point for Web searchers.

Search engines with large indexes

These attempt to cover as much of the Web as possible, resulting in a large number of returned sites. Search features include the '+' operator to require a search term and '-' to exclude a search term, as well as the Boolean AND, OR, NOT.

AltaVista     http://uk.altavista.com
A popular engine that covers about 20% of the Web. Some say that the main page is too complicated and that the real information is obscured by advertising.

Northern Light      http://www.northernlight.com
With a database of 320 million pages, this is one of the largest, but still covers only 23% of the Web. Surveys show that the Northern Light database is reasonably up to date.

Fast Search      http://alltheweb.com
A clean interface with one small advertisement makes Fast Search easy to use. Simply typing in several words is effective (ie an AND search). 500 million indexed pages makes this one of the largest engines.

These large index search engines are recommended for topics that are not so common and where other search engines have failed. The advanced search features are well worth learning.

Multiple-search engines - meta search

Multi-search (or Meta search) engines increases the coverage of the Web by passing the search on to a number of 'ordinary' search engines, and combining the results.
Examples:

Ixquick               www.ixquick.com
Inference Find  www.infind.com
Metacrawler     www.metacrawler.com
Copernic          www.copernic.com

While multi-search engines can find a larger number of sites, the search focus can become blurred as search engines interpret searches variously. Some people prefer to develop expertise with a couple of 'ordinary' search engines.

Copernic is interesting as it claims to refine searches to eliminate duplicates. Although it requires a program to be downloaded (free), Copernic is worth investigating for advanced searches.

How search engines work

There are four main stages to the operation of a search engine:

1. Finding Sites: A site may come to the notice of a search engine by being submitted by the owner, or by a link to a site already known. The new site is 'visited' by a software spider or robot that records the words in the title, headings, main text and any meta-data. There is often a delay of weeks before a site is visited and indexed. Recently, some search engines have added editor-produced catalogue information to their spider database.
2. Compiling the Index: The text information from visited sites is compiled into an index. As it may cover the words on a hundred million or more Web sites, the index will be very large and has to be organised in such a way that it can be searched in a couple of seconds.
3. Search Form: The search engine user interface usually includes a form into which the search terms are entered. The search engine matches these terms with those in the index to locate possible sites.
4. Results List: Lastly the search engine returns a list of matching sites, usually with the URL, the title of the page and a description taken from the page meta-tag or just the first 25 words of the page. These matches are usually ranked in order with the best match first.
   

Search engine issues

• Most search engines are funded by advertising, which can make some search engines difficult to use by younger pupils as the adverts can hide the returned page list.
• New sites or changes take time, sometimes many weeks, to be included.
• In some search engines, commercial interests can pay to get their sites into a higher position in the list returned to the searcher. There is a possibility that educational sites may become more difficult to find.
• Until effective standards for including meta-data (information about the contents such as keywords, author, date, type of work) are designed, search engines will not be able to locate all appropriate material efficiently.