Clustering web documents using hierarchical representation with multi-granularity

Faliang Huang, Shichao Zhang, Minghua He, Xindong Wu

Research output: Contribution to journalArticlepeer-review

Abstract

Web document cluster analysis plays an important role in information retrieval by organizing large amounts of documents into a small number of meaningful clusters. Traditional web document clustering is based on the Vector Space Model (VSM), which takes into account only two-level (document and term) knowledge granularity but ignores the bridging paragraph granularity. However, this two-level granularity may lead to unsatisfactory clustering results with “false correlation”. In order to deal with the problem, a Hierarchical Representation Model with Multi-granularity (HRMM), which consists of five-layer representation of data and a twophase clustering process is proposed based on granular computing and article structure theory. To deal with the zero-valued similarity problemresulted from the sparse term-paragraphmatrix, an ontology based strategy and a tolerance-rough-set based strategy are introduced into HRMM. By using granular computing, structural knowledge hidden in documents can be more efficiently and effectively captured in HRMM and thus web document clusters with higher quality can be generated. Extensive experiments show that HRMM, HRMM with tolerancerough-set strategy, and HRMM with ontology all outperform VSM and a representative non VSM-based algorithm, WFP, significantly in terms of the F-Score.
Original languageEnglish
Pages (from-to)105-126
Number of pages23
JournalWorld Wide Web
Volume17
Issue number1
Early online date15 Jan 2013
DOIs
Publication statusPublished - 31 Jan 2014

Keywords

  • web document clustering
  • hierarchical representation
  • multi-granularity

Fingerprint

Dive into the research topics of 'Clustering web documents using hierarchical representation with multi-granularity'. Together they form a unique fingerprint.

Cite this