Bidirectional hierarchical clustering for Web mining

A new bidirectional hierarchical clustering system for addressing challenges of Web mining that aims to maximize the intra-cluster similarity in the bottom-up cluster-merging phase and ensures to minimize the inter-clustering in the top-down refinement phase achieves better clustering than existing one-pass approaches.

Abstract

We propose a new bidirectional hierarchical clustering system for addressing challenges of Web mining. The key feature of the approach is that it aims to maximize the intra-cluster similarity in the bottom-up cluster-merging phase and it ensures to minimize the inter-cluster similarity in the top-down refinement phase. This two-pass approach achieves better clustering than existing one-pass approaches. We also propose a new cluster-merging criterion for allowing more than two clusters to be merged in each step and a new measure of similarity for taking into consideration not only the inter-connectivity between clusters but also the internal connectivity within the clusters. These result in reducing the average complexity for creating the final hierarchical structure of clusters from O(n/sup 2/) to O(n). The hierarchical structure represents a semantic structure between concepts of clusters and is directly applicable to the future of semantic net.