• Most Popular
• Blogs
• Subscribe
• Contact Staff

Droid First Step in iPhone Fade Away? Coupon Site Targets Black Friday, Cyber Monday Microsoft Axes Office Accounting Software PayPal Opens Up Payment Platform to Devs Windows 7 Drives a 49% Spike in PC Sales Red Hat Virtualization Manager for Windows Only? Apache at 10: You Can't Buy Us Oracle Outlines Sun Software Plans nVidia Blows Past Revenue, Earnings Projections Kaspersky Offers Macs a Security Blanket

Xbox 360 tests = FAIL Rush to publication Skype Founder Shakes Money Tree For Seesmic SCO is BACK !?!?! ' Linux Next ' Begins To Take Shape Yahoo Moves Ahead With Layoffs Three Hours Of No Email: Priceless Encyclopedia Brown and the Case of the Missing Patch Red Hat Accelerates JBoss Mozilla's New Look Firefox 3 Beta 3   [ More ]

Select a newsletter and click Join to sign up! Newslinx Daily Internet Daily InternetNews Business Report Boston News DC News NY News SiliconValley News InternetNews.com Podcast InternetNews.com Channel XML/RSS Feeds InternetNews.com IT Business News Developer News Ecommerce News Software IT News Hardware IT News Government News Search News Enterprise IT Planet News Networking News IT Security News Storage News Mobility News Web Content News Internet Stats Linux Today

Partner With Us

spider Last modified: Wednesday, March 03, 2004
A program that automatically fetches Web pages. Spiders are used to feed pages to search engines. It's called a spider because it crawls over the Web. Another term for these programs is webcrawler. Because most Web pages contain links to other pages, a spider can start almost anywhere. As soon as it sees a link to another page, it goes off and fetches it. Large search engines, like Alta Vista, have many spiders working in parallel. Also see How Web Search Engines Work in the Did You Know . . . ? section of Webopedia. Sponsored listings Web Site Search Functionality - Drastically improve the user experience on your website or intranet with the ISYS suite of search, navigation and discovery software. Free trial!		Related Categories Search Engines World Wide Web Related Terms Alta Vista cloaking dynamic URL link farming pagejacking query string robot search engine SEO SERP site map spider trap Shopping spider Products Compare Products,Prices and Stores Shop by Category: Toys 3008 Store Offers Sport and Outdoor 2275 Store Offers Clothing 2827 Store Offers Musical Instruments 80 Store Offers Auto Parts and Accessories 1039 Store Offers
Making Your Pages Easy for Search Engines to Index  Achieving high rankings for Web sites on search engines has become a black art. An article from the Web Developer's Virtual Library. Invite Search Engine Spiders Into Your Dynamic Web Site Increase the traffic to your dynamic site by making your valuable content visible to search engine spiders. LiveLink Spider home page Describes LiveLink Spider, the crawler software used in the Open Text Index, available from Open Text Corporation. Optimization for Dynamic Web Sites With just a few quick selections by the user visiting your site, a page can be generated on the fly based on nothing more than the user's specific interest or request. Yahoo!'s web searching robots and spiders page Yahoo!'s directory of web searching robots and spiders.