How Search Engines Work

How Search Engines Work

Crawling

Search Engine Thinks: • Hey, nice lookin’ site! •• When was the last Can I download thistime I got this page? the page? •page? Can I Process • When was the lastabout? time • Robots Exclusion is the page • What Maybe I’ll save this the page Protocol •page… What areupdated? all the links? • Network/ Error • How does thisServer compare to past versions? • How does this compare to related URLs • Does anything look abnormal? (title, h1, text, links)

Store the Page

Ranking

Rank Is…. • Base value of every page in index • Primarily based on quality and relevancy of inbound links •AND quality of content on your page • Computed for each URL • Computed for each FQDN • Computed periodically

High quality link Low quality link

Searching Let me think…. • Is it spelled right? • Do they want a navigational link, a question answered, a video, some websites or maybe an advertisement? • What content can best fulfill their need? • How should I order those results?

Summary Crawli ng

Query Parsin g Dynam ic Rankin g

Index

Rankin g

All to get the best result

Where to next?

http://ninebyblue.com

http://janeandrobot.com

http://SearchDeveloperDay.com

http://webmaster.live.com

http://google.com/webmaster

http://siteexplorer.search.yahoo.com