Predictable quality of service atop degradable distributed systems
Cluster ComputingPublished 16 January 2009
Lavanya Ramakrishnan, Daniel A. Reed
Citations10
SJR quartileQ1
SJR score1.04
SNIP1.59
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A resource performability model is presented to estimate lost performance and corresponding cost considerations with varying availability levels and is used in a multi-phase planning approach for scheduling a set of deadline-sensitive meteorological workflows atop grid and cloud resources to trade-off performance, reliability and cost.
Abstract
High performance and distributed computing systems such as peta-scale, grid and cloud infrastructure are increasingly used for running scientific models and business services. These systems experience large availability variations through hardware and software failures. Resource providers need to account for these variations while providing the required QoS at appropriate costs in dynamic resource and application environments. Although the performance and reliability of these systems have been studied separately, there has been little analysis of the lost Quality of Service (QoS) experienced with varying availability levels. In this paper, we present a resource performability model to estimate lost performance and corresponding cost considerations with varying availability levels. We use the resulting model in a multi-phase planning approach for scheduling a set of deadline-sensitive meteorological workflows atop grid and cloud resources to trade-off performance, reliability and cost. We use simulation results driven by failure data collected over the lifetime of high performance systems to demonstrate how the proposed scheme better accounts for resource availability.
Keywords
Computer Science
Journal of Parallel and Distributed ComputingA Comparison of Eleven Static Heuristics for Mapping a Class of Independent Tasks onto Heterogeneous Distributed Computing Systems
1,715 Citations2001Tracy D. Braun, Howard Jay Siegel +9 more
It is shown that for the cases studied here, the relatively simple Min?min heuristic performs well in comparison to the other techniques, and one even basis for comparison and insights into circumstances where one technique will out-perform another.
IEEE Transactions on ComputersOn Evaluating the Performability of Degradable Computing Systems
714 Citations1980Meyer
A hierarchical modeling scheme is used to formulate the capability function and capability is used, in turn, to evaluate performability, and techniques are illustrated for a specific application: the performability evaluation of an aircraft computer in the environment of an air transport mission.
A large-scale study of failures in high-performance computing systems
586 Citations2006Bianca Schroeder, Garth A. Gibson
IEEE Transactions on ReliabilityPerformance And Reliability Analysis Of Computer Systems (an Example-based Approach Using The Sharpe Software
444 Citations1997Robin Sahner, Kishor S. Trivedi +1 more
The SHARPE (Symbolic Hierarchical Automated Reliability and Performance Evaluator) package is the `toolchest' that allows the authors to specify stochastic models easily and solve them quickly, adopting model hierarchies and very efficient solution techniques.
Scientific ProgrammingScheduling Scientific Workflow Applications with Deadline and Budget Constraints Using Genetic Algorithms
388 Citations2006Jia Yu, Rajkumar Buyya
A genetic algorithm approach is presented to address scheduling optimization problems in workflow applications, based on two QoS constraints, deadline and budget, which are presented in this paper.
Task scheduling strategies for workflow-based applications in grids
365 Citations2005James M. Blythe, Shweta Jain +5 more
This work identifies two families of resource allocation algorithms: task-based algorithms, that greedily allocate tasks to resources, and workflow-based algorithms, that search for an efficient allocation for the entire workflow.
Scheduling Workflows with Budget Constraints
248 Citations2007Rizos Sakellariou, Henan Zhao +2 more
This paper considers a basic model for workflow applications modelled as Directed Acyclic Graphs (DAGs) and investigates heuristics that allow to schedule the nodes of the DAG (or tasks of a workflow) onto resources in a way that satisfies a budget constraint and is still optimized for overall time.
Journal of Grid ComputingA Flexible Framework for Fault Tolerance in the Grid
138 Citations2003Soonwook Hwang, Carl Kesselman
How to achieve the flexibility by the use of workflow structure as a high-level recovery policy specification, which enables support for multiple failure recovery techniques, the separation of failure handling strategies from the application code, and user-defined exception handlings.
Toward a framework for preparing and executing adaptive grid programs
113 Citations2002Ken Kennedy, M. Mazina +11 more
The goal of this framework is to provide good resource allocation for Grid applications and to support adaptive reallocation if performance degrades because of changes in the availability of Grid resources.
Computing in Science & EngineeringService-oriented environments for dynamically interacting with mesoscale weather
100 Citations2005
Research that is enabling a major shift toward dynamically adaptive responses to rapidly changing environmental conditions is described.
Performance prediction in production environments
80 Citations2002Jennifer M. Schopf, Francine Berman
Stochastic values are presented, showing their usefulness for a distributed SOR application, and shown to be more effective than single (point) values in predicting the range of application behavior that can occur during execution in production environments.
Future Generation Computer SystemsReliability challenges in large systems
60 Citations2005Daniel A. Reed, Charng‐Da Lu +1 more
This paper presents techniques for detecting imminent failures in the environment and that allow an application to run successfully despite such failures and shows how intelligent and adaptive software can lead to failure resilience and efficient system usage.
Assessing Fault Sensitivity in MPI Applications
52 Citations2005Charng‐Da Lu, Daniel A. Reed
These experiments showed that most applications are very sensitive to even single errors, and the errors were often undetected, yielding erroneous output with no user indicators.
IEEE ConcurrencyEnhancing the fault tolerance of workflow management systems
51 Citations2000Gustavo Alonso, Claus Hagen +3 more
The authors discuss current research directions and potential future extensions that might enable workflow services to meet the needs of mission-critical applications.
Lecture notes in computer sciencePerformance Variability of Highly Parallel Architectures
45 Citations2003William Kramer, Clint Ryan
The amount of variation seen across four large distributed memory systems in a systematic manner is explored and the causes for the variations seen are analyzed and what can be done to decrease the variation without impacting performance is discussed.
Measuring the Performance and Reliability of Production Computational Grids
40 Citations2006Omid Khalili, Jiahua He +3 more
This work reports on data gathered via a deployment of a monitoring and benchmarking infrastructure on two production grid platforms, TeraGrid and Geon, and investigates whether the execution time of a typical grid application can be predicated based on previous runs of simple benchmarks.
Parallel scheduling of complex dags under uncertainty
31 Citations2005Grzegorz Malewicz
The goal is to find a regimen Ε, that dictates how workers get assigned to tasks (possibly in parallel and redundantly) throughout execution, so as to minimize expected completion time.
Scalable Grid Application Scheduling via Decoupled Resource Selection and Scheduling
25 Citations2006Yang Zhang, Anirban Mandal +5 more
Leveraging the Virtual Grid abstraction, it is demonstrated that the decoupled approach is indeed both scalable and effective in large-scale and highly heterogeneous resource environments.
Performability modeling for scheduling and fault tolerance strategies for scientific workflows
25 Citations2008Lavanya Ramakrishnan, Daniel A. Reed
The concept of performability is used to capture the degraded performance that might result from varying resource availability and the resulting model is applied to workflow planning and fault tolerance strategies.
Fault tolerant computing on the grid: what are my options?
23 Citations2003Jon Weissman
This paper examined fault tolerance options for a common class of high-performance parallel applications, single-program-multiple-data (SPMD), and performance models for two fault tolerance methods, checkpoint-recovery (CR) and wide-area replication (WR) have been developed.
ProceedingsMinimizing the Network Overhead of Checkpointing in Cycle-harvesting Cluster Environments
16 Citations2005Daniel Nurmi, John Brevik +1 more
A new system for computing efficient checkpoint schedules in cycle-harvesting environments and the choice of model distribution has a relatively small but positive effect on time efficiency, it has a substantial impact on network utilization.