Web Directories and Search Engines The landscape of Web search services is changing constantly. And, unfortunately, most of us miss out on some of its more important and interesting changes because we have fallen into the easy habit of always relying on one or two of our favorite services. In this section, we offer a variety of sources on Web search services including those providing a behind-the-scenes look at how search engines actually work. In addition, you can consult the latest news about and reviews of the most popular search engines. Finally, there are several activities that explore searching using Web directories, search engines, and metasearch engines. How Search Engines Work Consult the always reliable HowStuffWorks for Curt Franklin's succinct look at How Internet Search Engines Work. Web search engines compile indices of only a relatively small portion of pages that are available on the Web. This hidden portion of the Web--called the "invisible" or "deep" Web is not accessible for several reasons. BrightPlanet offers Michael Bergman's essay The Deep Web: Surfacing Hidden Value. The piece outlines how search engines work as well as their limitations, especially in dealing with "dynamic" content. References
Danny Sullivan' popular guide providing background, ratings, and reviews for all the popular Web search services. Activities Web directories are listings edited by humans. They are typically organized by topic and arranged hierarchically. These hierarchies are often different because different editors manage them. The activities introduce the student to some of the popular Web directories and expose them to differences and similarities in their organizations. Search engines employ databases created by automated systems that troll the Web for relevant entries. As a result, search engines offer better coverage than Web directories. But, for a given search engine, how extensive is its coverage? Unfortunately, this is very difficult to measure. Instead, we often assume that the size of the database (i.e., the number of entries indexed) is a good proxy for estimating coverage. These activities explore some of the problems involved in measuring the actual size of search engine databases. The set of experiments here compare the relevant size of popular search engine databases. These experiments can be conducted by individuals, teams, or the entire class. Web directories and search engines offer keyword searching for forming user search queries. The precision of keyword searches is increased in most cases with the judicious use of multiple keywords. In order to use multiple keywords effectively, it is important to understand how a search engine interprets them. The activities are intended to show the student how multiple keyword searching works and its effectiveness. They also offer practice at developing searching skills using multiple keywords with Boolean operators. A metasearch engine queries other search engines. The results from these search engines are listed in various ways. Some merely list them; others summarize or classify them in useful ways. The student is introduced to popular metasearch engines in order to compare their value-added features. ©Abernethy and Allen, 2004. |