skip navigation

how search engines work in simple terms

Web terminology (including meta tag definitions)

This section explains in simple layman's terms, some terms frequently used by web designers. For example information on html meta tags, and the difference between search engines and directories.

How search engines work

Search engines rely on robots (sometimes called spiders), which visit pages in the world wide web by going from link to link. Usually once the homepage of a web site e.g. http://www.sfol.co.uk is found, the robot will visit other pages at the site. Web designers can, however, instruct robots not to visit one or more pages at a web site.

Robot are mainly looking for keywords, but at the same time they gather other information such as web page titles and descriptions. Other information may be gathered by robots from the following areas:

  • The body text of the HTML code i.e. what a viewer sees
  • The keyword meta tag
  • The Alt code "behind" images

Once a robot has gathered the information it requires, it sends it back to the search engine. It then follows the links on the web page to other web pages. Periodically (typically every six weeks) the search engine updates its database using the information returned from the robots. Web pages added are said to be indexed.

When a user types in criteria at the search engine, the search engine uses its database to find indexed pages that "best match" the search criteria.

Robots can be targeted to visit specific web pages. Search engines ensure their robots periodically visit their indexed pages. If the robot repeatedly fails to find an indexed page, this page may be removed from the search engine database.

In theory, if your web page has a link from another "indexed" web page, there is no need to submit it to search engines, it should be found.

Submitting your web page, however, ensures that it is added to the robots schedule, hence it is likely to be visited sooner. Note however, not all pages visited are indexed and there will be a delay between when your pages are visited and when they are indexed.

How directories differ

Unlike search engines, directories do not rely on robots. Directories are generally human compiled. When submitting a web page to a directory, additional information must be provided (e.g. the appropriate category, a description of the site, site title). The directory editor decides if the page will be added, after assessing the page for suitability. Editors are looking for good sites and will typically be considering the following:

  • Is the site original
  • Is it interesting and of value to the internet community
  • What is it offering the viewer
  • Does it download quickly
  • Is it easy to understand and navigate
  • Is it complete
  • Are there any dead links
  • Is it legal and moral

HTML meta tags defined

Meta tags are areas of text that are hidden from the viewer of a page but accessible to browsers and robots. They are located in the <head> section of the html code. Robots use meta tags to find out more about a web page, hence they can effect the search ranking. The Meta Tags of most interest for search ranking are the meta tag description and meta tag keywords.

Note however, not all search engines use meta tags.

Meta tag description

The description Meta Tag is used by some search engine to provide a brief description of the web page for the search result. The content of the description meta tag, should provide a description of the page.

using meta tags - meta description tag html syntax

<meta name="description" content="web page description goes in here">

Meta tag keywords

The keywords meta tag was originally intended to provide search engines with keywords for web pages that were mainly graphical. Over the years some web designers have abused the keywords meta tag by loading repeated keywords or keywords that are out of context with the page. As a result some search engines now ignore the keywords meta tag, others implement complex anti spamming algorithms intended to detect attempts to spam (cheat). It is easy to inadvertently be considered as spamming.

using meta tags - meta keyword tag html syntax

<meta name="description" content="web page description goes in here">

Meta tag refresh

The refresh meta tag, redirects the browser to an alternative URL after a specified period. It can be useful for leaving a "forwarding address" however many search engines will not index pages using the refresh meta tag.

using meta tags - meta refresh tag html syntax

<meta http-equiv="refresh" content="3;URL=http://www.sfol.co.uk/index.html">

The content arguments are a number (duration in seconds) ; the URL to redirect to.

Spamming

Spamming is attempting to raise the profile of a web site or page by using means considered by search engines as cheating. What is considered as spamming varies from search engine to search engine, however the following practices are some examples:

  • Invisible text to promote keywords
  • Tiny text to promote keywords
  • Excessive repetition of keywords
  • Meta keyword tags containing keywords out of context with the page contents
  • Over submission of a page to a search engine
  • Over submission of a page to a search engine

Redirection pages

  • Duplicate pages or mirror sites

Copyright 2009 SFOL Database Services.