MDCube

Published 1 December 2009

Haitao Wu, Guohan Lu, Dan Li, Chuanxiong Guo, Yongguang Zhang

Citations166

TL;DR

MDCube, a high performance interconnection structure to scale BCube-based containers to mega-data centers, uses the high-speed uplink interfaces of the commodity switches in BCube containers to build the inter-container structure, reducing the cabling complexity greatly.

Abstract

Shipping-container-based data centers have been introduced as building blocks for constructing mega-data centers. However, it is a challenge on how to interconnect those containers together with reasonable cost and cabling complexity, due to the fact that a mega-data center can have hundreds or even thousands of containers and the aggregate bandwidth among containers can easily reach tera-bit per second. As a new inner-container server-centric network architecture, BCube [9] interconnects thousands of servers inside a container and provides high bandwidth support for typical traffic patterns. It naturally serves as a building block for mega-data center.

Keywords

Computer Science

Communications of the ACMMapReduce

18,538 Citations2008Jay B. Dean, Sanjay Ghemawat

This presentation explains how the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks.

View PDF

ACM SIGOPS Operating Systems ReviewThe Google file system

5,003 Citations2003Sanjay Ghemawat, Howard Gobioff +1 more

The Hadoop Distributed File System

4,780 Citations2010Konstantin V. Shvachko, Hairong Kuang +2 more

The architecture of HDFS is described and experience using HDFS to manage 25 petabytes of enterprise data at Yahoo! is reported on.

ACM SIGCOMM Computer Communication ReviewA scalable, commodity data center network architecture

3,161 Citations2008Mohammad Al-Fares, Alexander Loukissas +1 more

Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes

2,474 Citations1991Frank Thomson Leighton

This chapter discusses sorting on a Linear Array with a Systolic and Semisystolic Model of Computation, which automates the very labor-intensive and therefore time-heavy and expensive process of manually sorting arrays.

Dryad

2,460 Citations2007Michael Isard, Mihai Budiu +3 more

The Dryad execution engine handles all the difficult problems of creating a large distributed, concurrent application: scheduling the use of computers and their CPUs, recovering from communication or computer failures, and transporting data between vertices.

VL2

2,089 Citations2009Albert Greenberg, James R. Hamilton +7 more

VL2 is a practical network architecture that scales to support huge data centers with uniform high capacity between servers, performance isolation between services, and Ethernet layer-2 semantics, and is built on a working prototype.

A scalable, commodity data center network architecture

1,655 Citations2008Mohammad Al-Fares, Alexander Loukissas +1 more

This paper shows how to leverage largely commodity Ethernet switches to support the full aggregate bandwidth of clusters consisting of tens of thousands of elements and argues that appropriately architected and interconnected commodity switches may deliver more performance at less cost than available from today's higher-end solutions.

View PDF

The Google file system

1,366 Citations2003Sanjay Ghemawat, Howard Gobioff +1 more

This paper presents file system interface extensions designed to support distributed applications, discusses many aspects of the design, and reports measurements from both micro-benchmarks and real world use.

BCube

1,349 Citations2009Chuanxiong Guo, Guohan Lu +7 more

Experiments in the testbed demonstrate that BCube is fault tolerant and load balancing and it significantly accelerates representative bandwidth-intensive applications.

View PDF