Robust Web Extraction: An Approach Based On A Probabilistic Tree-Edit Model
MOD(2009)
摘要
On script-generated web sites, many documents share common HTML tree structure, allowing wrappers to effectively extract information of interest. Of course, the scripts and thus the tree structure evolve over time, causing wrappers to break repeatedly, and resulting in a high cost of maintaining wrappers. In this paper, we explore a novel approach: we use temporal snapshots of web pages to develop a tree-edit model of HTML, and use this model to improve wrapper construction. We view the changes to the tree structure as suppositions of a series of edit operations: deleting nodes, inserting nodes and substituting labels of nodes. The tree structures evolve by choosing these edit operations stochastically.Our model is attractive in that the probability that a source tree has evolved into a target tree can be estimated efficiently in quadratic time in the size of the trees making it a potentially useful tool for a variety of tree-evolution problems. We give an algorithm to learn the probabilistic model from training examples consisting of pairs of trees, and apply this algorithm to collections of web-page snapshots to derive HTML-specific tree edit models. Finally, we describe a novel wrapper-construction framework that takes the tree-edit model into account, and compare the quality of resulting wrappers to that of traditional wrappers on synthetic and real HTML document examples.
更多查看译文
关键词
Wrappers,Xpath,Probabilistic Tree-Edit Model
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络