How Search Engines Work
Google works in three steps — it crawls pages, stores them in an index, then ranks them for each search.
What you will learn
- Name the three jobs a search engine does
- Explain crawling and indexing simply
- See why a page must be indexed before it can rank
Three jobs Google does
Before Google can show your page, it has to find it, understand it, and decide where it belongs. That is three separate jobs:
- Crawling — software robots called crawlers (or spiders) follow links across the web and read pages.
- Indexing — Google stores what it found in a giant library called the index.
- Ranking — when someone searches, Google picks the best pages from the index and puts them in order.
A library analogy
Picture a huge library. The crawler is the assistant who walks every aisle and reads every new book. The index is the catalogue that records what each book is about. Ranking is the librarian handing you the most useful books first when you ask a question.
| Step | Library version | What it means for your site |
|---|---|---|
| Crawl | Assistant reads each book | Google must be able to reach your pages |
| Index | Catalogue stores the book | Your page is saved and can appear |
| Rank | Librarian orders the books | Google decides your position |
These three steps happen in order, and each one depends on the one before it. A page that is never crawled can never be indexed; a page that is never indexed can never rank. So a missing page is almost always a problem at step 1 or step 2 — not bad ranking, but no entry in the library at all.
A worked example: three pages, three outcomes
Say Sweet Crumbs has three pages. Watch where each one gets stuck:
| Page | Crawled? | Indexed? | Can it rank? | Why |
|---|---|---|---|---|
| /eggless-cakes | Yes | Yes | Yes | Linked from the menu, good content |
| /new-offer | No | No | No | Brand new, no link points to it yet |
| /thank-you | Yes | No | No | Crawled, but marked “do not index” |
Note: Only the first page can show in Google. The second was never found (no link leads to it — add one or submit a sitemap). The third was read but deliberately kept out of the index. The lesson: before worrying about position, make sure your important pages are crawled AND indexed.
Check if a page is indexed
You can ask Google whether it has a page in its index. Type a special search using the site: command:
site:sweetcrumbs.in eggless cakeNote: This searches only the website sweetcrumbs.in. If the eggless cake page shows up, it is indexed. If nothing shows, Google has not stored that page yet — so it can never rank.
Watch out: A page that is not indexed cannot rank at all. If your page is missing from Google, fixing crawling and indexing comes first — before any other SEO.
Tip: New pages are not found instantly. Google has to crawl them first, which can take hours to weeks. Good internal links and a sitemap (later lessons) speed this up.
Q. What is the correct order of the three jobs a search engine does?
✍️ Practice
- Run a
site:search for any website you like and count how many pages Google shows. - Explain crawling and indexing using your own everyday example.
🏠 Homework
- Do a
site:search for a local business website. Note whether its main pages appear to be indexed.