Towards network-aware data mining

It is argued that existing network monitoring and quality of service mechanisms are insufficient to support the needs of distributed data mining algorithms that are network-aware and can adapt to network-resource constraints.

Abstract

Distributed data mining algorithms executing on a shared network of workstations often suffer from unpredictable performance problems due to limited network resources that are being shared. We show that data mining algorithms, which have an approximate nature, can adapt to network-resource constraints. We argue that existing network monitoring and quality of service mechanisms are insufficient to support the needs of such applications. We then discuss the mechanisms (systems support) needed to support such network-aware applications. Finally, we describe the current status (work-in-progress) of the system under development and discuss some general issues involved in developing resource-aware data mining algorithms.