         
Registry Cleaners
Related Links:
• 121 Ideas To Start A Home Business
• A completely automatic marketing system
• A Good Headline Will Help You Sell
• Are Top Web Sites Good For Everybody
• Are you receiving your mail
• Do You Need a Sales Follow Up System
• E mail Signatures A Great Help For Selling Online
• Email Marketing Alternatives
• Ezine Marketing Another Low Budget Means Of Advertising
• Gateway Pages Can Be What You Are Needing
• How Can You Know If Your Web Site Is A Good Website
• How To Build Your Own Online Auction Site
• How To Improve Your Clickbank Affiliate Link
• How To Improve Your Profit With A Better Google PageRank
• How To Keep Robots Out Of Your Web Site
• How To Make Profit With Endorsement Marketing
• How To Make Profit With Market Segmentation
• How To Sell An Item Online
• How To Use The Click Exchange Programs
• How To Write A Good Headline
• My Experience About Beginning A Home Business
• Pay Per Click Search Engines Are They Effective
• Pay Per Listing Versus Pay Per Click
• Reciprocal Links Exchange Programs Are They Good
• Spam Guard Your Business
• Targeted Traffic Is The Secret For Good Sales
• The best Home Based Business Ideas For You
• The Favorite Bookmarks and Favorites
• The Truth About The Award Winning Web Sites
• The Truth About Web Site Banners
• The Unlimited Automatic Responders Resource
• These Are The Niche Marketing Secrets
• Use Doorway Pages or Pointer Pages
• Web Site Rings For Everybody
• What Is a FFA Web Site
• Why Should You Care About Return Traffic
• Writing Classified Ads That Generate Sales
• You Can Have Your Own E book Online
• You Can Make Money With What You Know
• Your Best Affiliates Program Is Your Own Affiliates Program
|
How To Keep Robots Out Of Your Web Site
You know that search engines have been created to help people find information quickly on the Internet, and the search engines acquire much of their information through robots (also known as spiders or crawlers), that look for web pages for them.
The spiders or crawlers robots explore the web looking for and recording all kinds of information. They usually start with URL submitted by users, or from links they find on the web sites, the sitemap files or the top level of a site.
Once the robot accesses the home page then recursively accesses all pages linked from that page. But the robot can also check out all the pages that can find on a particular server.
After the robot finds a web page it works indexing the title, the keywords, the text, etc. But sometimes you might want to prevent search engines from indexing some of your web pages like news postings, and specially marked web pages (in example: affiliateīs pages), but whether individual robots comply to these conventions is pure voluntary.
ROBOTS EXCLUSION PROTOCOL
So if you want robots to keep out from some of your web pages, you can ask robots to ignore the web pages that you donīt want indexed, and to do that you can place a robots.txt file on the local root server of your web site.
In example if you have a directory called e-books and you want to ask robots to keep out of it, your robots.txt file should read:
User-agent: * Disallow: e-books/
When you donīt have enough control over your server to set up a robots.txt file, you can try adding a META tag to the head section of any HTML document.
In example, a tag like the following tells robots not to index and not to follow links on a particular page:
meta name="ROBOTS" content="NOINDEX, NOFOLLOW"
Support for the META tag among robots is not so frequent as the Robots Exclusion Protocol, but most of major web indexes currently support it.
NEWS POSTINGS
If you want to keep the search engines out of your news postings, you can create an an "X-no-archive" line in of your postings' headers:
X-no-archive: yes
But although common news clients allow you to add an X-no-archive line to the headers of your news postings, some of them donīt permit you to do so.
The problem is that most search engines assume that all information they find is public unless marked otherwise.
So be careful because though the robot and archive exclusion standards may help keep your material out of major search engines there are some others that respect no such rules.
If you're highly concerned about the privacy of your e-mail and Usenet postings, you must use some anonymous remailers and PGP. You can read about it here:
www dot well dot com/user/abacard/remail.html
www dot io dot com/~combs/htmls/crypto.html
world dot std dot com/~franl/pgp/
Even if you are not particularly concerned about privacy, remember that anything you write will be indexed and archived somewhere for eternity, so use the robots.txt file as much as you need it.
Written by Dr. Roberto A. Bonomi
Learn how to increase your monthly income! Easy Home Business http://www.easy-home-business.com is the best internet marketing training site that you will find on the web. Dr. Roberto Bonomi is a successful writer that shares his home business experience at this site, and invites you to post free your own articles, or find free articles for your web site or ezine at http://articles.drbonomi.com
|
|
