Follow this blog

Wednesday, January 16, 2013

How often Google crawl

You make a new web page and ask yourself when Googe will crawl my new web page (or new blog post)?

Let's try to understand how Google crawl the web.

Google spiders

Google use spiders to crawl the web.

Spiders (also called GoogleBots) are software robots which discovers new and updated web pages and add those pages to the Google index.

Google crawling

When Googlebot (or a Google spider) visit one web site it detects all links on each page of this web site and adds them to its list of pages to crawl.

So Googlebot crawl from one page to another and follow their links. This is how Google crawl almost all pages on the internet.

When Google find some web site it will come back to crawl this site again and reindex it.

How often Google crawl

How often Google will crawl some web site depends on many factors such us PageRank, links to a page...

One of the factors is how often you change your site or add new pages. If changes are happening more frequently then Google will probably crawl your web site more frequently.

If your site or blog is new and is not indexed by google find out how to force google to crawl your web site or blog.

No comments: