Empowering R with High Performance Computing Resources for Big Data Analytics

The chapter shows the potential benefits of exploiting high performance computing with R and recommendations for end users of applying R to big data problems and discusses a list of challenges in improving R performance for the end users.

Abstract

The software package R is a free, powerful, open source software package with extensive statistical computing and graphics capabilities. Due to its high-level expressiveness and multitude of domain specific packages, R has become the lingua franca for many areas of data analysis, drawing power from its high-level expressiveness and its multitude of domain specific, community-developed packages. While R is clearly a "high productivity" language, it has not necessarily been a "high performance" language. Challenges still remain in developing methods to effectively scale R to the power of supercomputers, and in deploying support and enabling access for the end users. In this chapter, we focus on approaches that are available in R that can adopt high performance computing resources for providing solutions to Big Data problems. Here we first present an overview of current approaches and support in R that can enable parallel and distributed computations in order to improve computation scalability and performance. We categorize those approaches into two on the basis of the hardware requirement: single-node parallelism that requires multiple processing cores within a computer system and multi-node parallelism that requires access to computing cluster. We present a detail study on performance benefit of using Intel® Xeon Phi coprocessors (Xeon Phi) with R for improved performance in the case of single-node parallelism. The performance is also compared with using general-purpose graphic processing unit through HiPLAR package and other parallel packages enabling multi-node parallelism including SNOW and pbdR. The results show advantages and limitations of those approaches. We further provide two use cases to demonstrate parallel computations with R in practice. We also discuss a list of challenges in improving R performance for the end users. Nevertheless, the chapter shows the potential benefits of exploiting high performance computing with R and recommendations for end users of applying R to big data problems.