Google search engine optimization
So, how does the Google updates its database? The question is quite a wide, but I will try to explain each step of this system checking monthly its database for the conformity with the highest standards.
The majority of people and companies understand that for achievement of the highest search engines ranking you should think over all steps beforehand, before starting website optimization, and plan your activities for future as well. For today, Google is one of those few search engines working on free basis for sites – participants. Also, links to your resource in its database are shown as one of the first, and, according to the information received on July, 10 2003, worldwide Google database has more than 3.4 milliard pages! Despite the fact, that this is only a drop in the ocean, since many pages are not available for indexing by system spiders.
As it is in life, there are many difficulties and risks, which sites owners, web-designers and professionals – optimizers face starting their marketing programs. Though the majority of experts agree with that Google sends robots before and after certain periods, but practically no one can confidently say when exactly the system will perform general scanning and totally renew its database. In this article we shall discuss main parts of the “Google Dance”, and also how and when to recognize robots, and how to take consider this fact to optimize your site.
This amazing Google Dance
If you think that you need to write a letter in order to receive an invitation to dance party held annually at Google headquarters in GooglePlex, then I think you should definitely read this article to have a clear idea of that how Google database is and how does it work. Though there are a lot of information known about annual databases renewal (they are also called “Google Dance”), but for the last time they are becoming less and less similar to each other, and, therefore, for new webmasters, this is a great jump in emptiness, because there is nothing clear around. That is why they are waiting for monthly updates impatiently, with hope for mysterious luck. Each “dance” starts with the main, global search. Let’s call it “Search A”. And what is going on there at this time? Materially, nothing special – the spiders visit all the database (more than 3.4 milliard pages according to the last calculations). For that purpose Google has more than 15.000 inexpensive computers (ordinary PCs), which are scattered all over the world and located in different data-centers. At this moment, Googlebots visit all pages stored in database and also search for new ones, which were recently created. After “Search A” completion, when all the pages are fixed in database for the next update, the second search runs, approximately 2 weeks after the first one.
The Google meanwhile completely renews its database, and the results have become available at wwҨ.google.com and www3.google.com. Also, there simultaneously goes the main database renewal, but how it was stated above – Google uses more than 15.000 servers, and, therefore, sometimes results of the search in different parts of world may differ, until the total database renewal takes place. The “Google Dance” will be going on some days more, but, usually not more that one week (exceptions are occasions when the algorithm itself has been changed, as it was done on April, 2003.)
Anyway, as well as during, and immediately after the databases renewal, Google will start the second global search; let’s call it “Search B”. At the time of “Search B” all pages existent in current database, and also new, recently launched resources being already noticed by spiders, will be visited. After that search, the cycle begins all over again for the next month.
How to “catch” Googlebot in due time?
Every skilled webmaster knows, that in order to get into Google database, or renew already exiting information, it is necessary to plan everything thoroughly and “catch” Googlebot exactly in due point of a monthly cycle. The majority of experts on optimization select the first search, which is done by Googlebot at the beginning of a month, and the second one – meanwhile and just immediately database update.
So, we have a task to place the site in Google’s database. The question: is the inclusion of a site in database guaranteed at visiting it by the system robot at the search? Proceeding from our experience – not always. Being more precise, if spider comes to the site at the beginning of month, so you will have all chances of that site wouldn’t be included into this month renewal. If spider comes to site at the time of the second search following after the renewal, so it is probably (but not guaranteed) that it will come there at the next search, and then include the site into next month renewal.
Sometimes, spider can just enter new site and look through its front page and Robots.txt file. This is a good sign, since it means that Google will return here at the time of the next global search, and this site will be included in up-to-date, which follows the second search. So, in order to place your site into database, Google must visit your site twice, though, the exceptions are also available. In order to ensure quicker pages’ indexing and including them into database, it is possible to undertake such steps: if spider visits the site for the first time at renewal or after it, so current site will be included to the Google database next moth with practically all guarantees. If spider doesn’t come to your site at that time, but did it at the time of the next visit, so the time of waiting for site’s appearance in database will be significantly increased.
With respect to all said above, what can ordinary webmaster do in order to “catch” Google bot in due time? Of course, you may pray, or light candles in a church, or jump with tambourine around the server, but sometimes it is much more easier to make a plan. If you already have definite resources, which are located in the Google database, you can watch the process of “global search” and all renewals, and, in accordance with this, plan new projects. If we don’t still have such sites, we can watch the updates on www.google.com.
Though, there is no 100% guarantee of that your page will be indexed (partially or fully), there are, however, some methods giving Googlebot a sign that this site must be visited. The first one is exchange of links with sites having high Page Rank. The higher site’s Page Rank is, the more chances of being frequently visited by Googlebot this site has, what, in its turn means that your URL will get into the database much more quicker. Few words about links relevance: if site devoted to the furniture selling, and links to sites of companies – manufacturers, distributors and so on, so Google will rank your site higher than if you would place links not corresponding with your site theme.
The following method is about placing your site into the database by using the “Add url” section -http://www.google.com/addurl.html. Even though this gives no guarantee, it is better not to neglect this. The third method is that webmaster can install Google Toolbar and surf his site using browser with installed toolbar. From the second half of 2002, the direct dependence between including site to database and attendance with active Google Toolbar have been noticed.
It is also useful to list your site in various catalogues and directories including Yahoo, ODP, etc. This will cost $Ꮻ annually in Yahoo and free in ODP. Yahoo service is also good because they usually place sites in catalogue within only 7 days. Consider listing your site in http://www.uptimebot.com/friends.php directory to enhance your site’s crawling and link popularity. It is also useful to add counter code on your page and receive one or more relevant links from counter ranking pages – you can find counter code at http://www.top100categories.com.
All technical information available for webmasters and SEO masters, which is related to the periods of the Google spiders activity and terms of database updates, can undoubtedly help at picking of methods of site’s planning and optimization for search engines. In great extent this knowledge can help new projects and updated launch, which must be launched strictly at certain moment in order to appear in search engine database. Though Google still remains the best traffic source, it is better to understand the principle of it’s functioning.
It’s rather inconvenient to check all search engines manually in case if you want to find out how much pages have been crawled by a search engine and how much backward links does your site has in a particular search engine database. Use Link Popularity Check tool in order to monitor your site’s statistics in search engines.
