Document Processing with LinkIT

We present a linguistically-motivated technique for the recognition and grouping of simplex noun phrases (SNPs) called LinkIT. Our system has two key features: (1) we efficiently gather minimal NPs, i.e. SNPs, as precisely and linguistically d efined and motivated in our paper ; (2) we a pply a refined set of post- processing rules to these SNPs to link them within a document. The identification of SNPs is performed using a finite state machine compiled from a regular expression grammar, and the process of r anking the candidate significant t opics uses frequency information that i s gathered in a single pass through the document. We evaluated the NP identification component of LinkIT and found that it outperformed other NP chunkers in p recision and recall. The system is currently u sed in several applications which are described, such as web page characterization and multi-document summarization.