InternetNews.com - Webopedia: Online Dictionary for Computer and Internet Terms
 


   

Partner With Us
















Enter a keyword...
...or choose a category.

robots.txt
Last modified: Wednesday, May 06, 2009  

Robots.txt is common name of a text file that is uploaded to a Web site's root directory and linked in the html code of the Web site. The robots.txt file is used to provide instructions about the Web site to Web robots and spiders. Web authors can use robots.txt to keep cooperating Web robots from accessing all or parts of a Web site that you want to keep private.

Related Categories
  • Search Engines
  • World Wide Web
  • Related Terms
  • search engine
  • search engine marketing
  • search results
  • SEO
  • SERP
  • Web
  • Web page
  • Web site
  • Shopping
    robot dot Products
    Compare Products,Prices and Stores

    Shop by Category:
    Sport and Outdoor
    23 Store Offers

    Toys
    2 Store Offers

     
    Related Links

    Webopedia's "Did You Know...?" Section
    Use this Webopedia knowledge section for an in-depth overview of specific technologies and occurrences in the areas of Computer Science, The Internet, and Computer Hardware and Software.

    Webopedia's Quick Reference Section
    Use this Webopedia reference section for information on common Internet and computer facts and occurrences.


    Add internetnews.com
    to your browser search box.

    IE 7 | Firefox 2.0 | Firefox 1.5.x

    Receive news
    via our XML/RSS:
    feed

    More InternetNews.com


    Hardware Software Mobility Web Content
    Search Government Developer Business
    Storage E-Commerce Networking Security




    The Network for Technology Professionals

    Search:

    About Internet.com

    Legal Notices, Licensing, Permissions, Privacy Policy.
    Advertise | Newsletters | E-mail Offers