craig-ellyard.com

Spiders And How Search Engines Work

There are many ways to attract visitors to your website but it will be a search engine which brings most of your free traffic. And they do this by ‘spidering’ your website.      

By submitting a sitemap to search engines such as Google and Yahoo, or submitting the site details to other search engines such as Live, you are in effect requesting the spiders to visit your site and place your pages in their index.

A ’spider’ is an automated program that is run by the search engine system. A spider will visit a web site, read the content and meta tags and also follow the internal links throughout the site.

The spider then returns all that data back to the search engine which then indexes all the information.

Importantly, the spiders will also follow the external links from your site and index those as well, which is why it is important to have as many of your own links as possible on other peoples sites.

The spider will periodically return to the site to check for any information that has changed or for new links which have been added.

The search engines produce rankings according to the information gathered by the spiders. The formula by which these calculations are carried out are called algorithms.

A very important point to note here is that NO ONE knows EXACTLY how the search engine algorithms work and different search engines produce different rankings because not every search engine uses the same algorithm to search through and index a website.

The best way to improve search engine rankings is by producing good quality keyword rich content, building quality links and regularly updating your website.