Authors: Lindemann, Christoph
Title: Innovative Basistechnologien für eine skalierbare, intelligente Internet-Suchmaschine
Language (ISO): de
Abstract: As a consequence of the tremendous size, explosive growth, and rapidly changing nature of the Web, a major challenge for search engine design and implementation lies in providing means for scalability at large. In this paper, we introduce WebSearchBench: a parallel software architecture for Internet search engines running on commodity-of-the-shelf components (a Linux cluster comprising of Intel Pentium IV Xeon dual-processor PCs connected by a Gigabit Ethernet). The presented performance study shows that WebSearchBench running on 8 nodes of a cluster can crawl and index 40 million Web pages per day. Another 4 nodes of the cluster can manage an index of 200 million pages and answer more than 25 million search queries per day. The repository for storing 200 million pages requires 7 additional nodes. Furthermore, our study indicates that WebSearchBench running on 32 nodes should be able to manage an index of 2 billion pages and answer about 120 million search queries per day.
Issue Date: 2002-10-17
Provenance: Universität Dortmund
Is part of: 06. InetBib-Tagung vom 18. bis 20. September 2002 in Göttingen
Appears in Collections:06. InetBib-Tagung vom 18. bis 20. September 2002 in Göttingen

Files in This Item:
File Description SizeFormat 
lindemann.pdf654.1 kBAdobe PDFView/Open

This item is protected by original copyright

Items in Eldorado are protected by copyright, with all rights reserved, unless otherwise indicated.