Search Engines and How They Work
Web search engines do their job by storing information about many web pages, which they then retrieve from the html itself. These pages are retrieved by a Web crawler (sometimes also known as a spider) — an automated Web browser which follows every link on the site. The contents of each page are then examined [...]
