earticle

영어

With the explosive growth of the information on the Internet, the number of web pages is also expanding rapidly. A common web page typically contains many different blocks. Unfortunately, apart from the main content blocks, it usually has such blocks as navigation bars, copyright, and advertisements, which are known as noise. Studies have shown that these noisy blocks can do harm to Web mining, search engine performance, and Web page clustering and classification. It is a significant work to effectively clean these noises up and extract main content blocks from the Web pages. In this paper, we propose a method of automatic template detection and noise elimination based on an new idea defining two sorts of templates called Style Template and Content Template relatively. There is a basic observation that, the Web pages from the same site are always similar in presentation style and contents, however, the main content blocks are always different in contents. Therefore, we design an algorithm to detect the two kind of templates from the samplings from a particular site, and extract the main content out of the noises using the template. We apply our method to template problem, and the result shows that our methods performs very well, and has a remarkable advantage on time consumption over previous methods.