How search engines work
- Crawling: programs called crawlers follow links and sitemaps to discover pages.
- Indexing: the pages are analyzed and stored in a huge database, the index.
- Ranking: for each search, algorithms pick and order the most relevant, useful results from the index.
- Serving: the results page shows them, often together with ads, maps, images and AI-generated answers.
Search engines and AI
Search engines now use AI to understand questions and, more and more, to write answers, such as Google’s AI Overviews and Microsoft Copilot in Bing. AI assistants such as ChatGPT and Perplexity also search the web to answer questions. They all depend on content they can crawl and trust, so the basics of SEO apply to them too.
Why it matters for businesses
Most buying journeys start with a search. Being easy to find for the searches your customers make, in both organic results and ads, is one of the most dependable ways to win new business online.
How a search works: an example
You search “how to repot a cactus”. The search engine has already crawled millions of gardening pages and stored them in its index. It looks for pages that match the words and the meaning, checks quality signals such as links and usefulness, considers your location and device, and returns an ordered list, possibly with a featured snippet, images and an AI-written summary. All of this takes a fraction of a second.
Search engines compared
| Engine | Notes |
|---|---|
| By far the largest worldwide | |
| Bing | Microsoft’s engine; its index also supplies other services |
| DuckDuckGo | Privacy-focused; relies heavily on Bing results |
| Yandex, Baidu, Naver | Leading in Russia, China and South Korea |
For current market shares see our guide to the top search engines in the world.
Search engine mistakes site owners make
- Optimising only for Google and ignoring Bing indexing.
- Blocking crawlers by mistake.
- Assuming rankings are the same for everyone. Results vary by location and device.
Search engines FAQs
How do I get my site into a search engine?
Make sure it is crawlable and submit a sitemap. See crawlers and Bing Webmaster Tools.
Is AI replacing search engines?
AI features are being added to them, and assistants are another way to find answers. See answer engine optimization and DuckDuckGo.