A Simple MPI Process Swapping Architecture for Iterative Applications

This paper presents MPI process swapping, a simple performance enhancing add-on to the MPI programming paradigm that can be added to iterative MPI applications and requires as few as three lines of source code change.

Abstract

Parallel computing is now popular and mainstream, but performance and ease of use remain elusive to many end-users. There exists a need for performance improvements that can be easily retrofitted to existing parallel applications. In this paper we present MPI process swapping, a simple performance enhancing add-on to the MPI programming paradigm. MPI process swapping improves performance by dynamically choosing the best available resources throughout application execution, using MPI process over-allocation and real-time performance measurement. Swapping provides fully automated performance monitoring and process management, and a rich set of primitives to control execution behavior manually or through an external tool. Swapping, as defined in this implementation, can be added to iterative MPI applications and requires as few as three lines of source code change. We verify our design for a particle dynamics application on desktop resources within a production commercial environment.