earticle

영어

While the World-Wide Web offers an incredibly rich base of information. organized as a hypertext it does not provide a uniform and efficient way to retrieve specific information. Therefore. it is needed to develop an efficient web crawler for gathering useful information in acceptable amount of time. In this paper. we studied the order in which the web crawler visit URLs to rapidly obtain more important web pages. We also developed an internet agent for efficient web crawling using hyperlink structure and fitness of hypertext documents. As a result of experiment on a website. it is shown that proposed agent outperforms other web crawlers using BackLink and PageRank algorithm.