Data center TCP (DCTCP)
Published 30 August 2010Open access
Mohammad Alizadeh, Albert Greenberg, David A. Maltz, Jitendra Padhye, Parveen Patel, Balaji Prabhakar
Citations1,771
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
DCTCP enables the applications to handle 10X the current background traffic, without impacting foreground traffic, thus largely eliminating incast problems, and delivers the same or better throughput than TCP, while using 90% less buffer space.
Abstract
Cloud data centers host diverse applications, mixing workloads that require small predictable latency with others requiring large sustained throughput. In this environment, today's state-of-the-art TCP protocol falls short. We present measurements of a 6000 server production cluster and reveal impairments that lead to high application latencies, rooted in TCP's demands on the limited buffer space available in data center switches. For example, bandwidth hungry "background" flows build up queues at the switches, and thus impact the performance of latency sensitive "foreground" traffic.
Keywords
Computer ScienceEngineering
IEEE/ACM Transactions on NetworkingRandom early detection gateways for congestion avoidance
6,255 Citations1993Sally Floyd, Van Jacobson
Red gateways are designed to accompany a transport-layer congestion control protocol such as TCP and have no bias against bursty traffic and avoids the global synchronization of many connections decreasing their window at the same time.
ACM SIGCOMM Computer Communication ReviewA scalable, commodity data center network architecture
3,161 Citations2008Mohammad Al-Fares, Alexander Loukissas +1 more
ACM SIGOPS Operating Systems ReviewCUBIC
2,222 Citations2008Sangtae Ha, Injong Rhee +1 more
The CUBIC protocol modifies the linear window growth function of existing TCP standards to be a cubic function in order to improve the scalability of TCP over fast and long distance networks.
VL2
2,089 Citations2009Albert Greenberg, James R. Hamilton +7 more
VL2 is a practical network architecture that scales to support huge data centers with uniform high capacity between servers, performance isolation between services, and Ethernet layer-2 semantics, and is built on a working prototype.
A scalable, commodity data center network architecture
1,655 Citations2008Mohammad Al-Fares, Alexander Loukissas +1 more
This paper shows how to leverage largely commodity Ethernet switches to support the full aggregate bandwidth of clusters consisting of tens of thousands of elements and argues that appropriately architected and interconnected commodity switches may deliver more performance at less cost than available from today's higher-end solutions.
The Addition of Explicit Congestion Notification (ECN) to IP
1,183 Citations2001K. Ramakrishnan, Sally Floyd +1 more
This memo specifies the incorporation of ECN (Explicit Congestion Notification) to TCP and IP, including ECN's use of two bits in the IP header.
HighSpeed TCP for Large Congestion Windows
1,157 Citations2003Sally Floyd
The proposals in this document are experimental and while they may be deployed in the current Internet, they do not represent a consensus that this is the best method for high-speed congestion control.
The nature of data center traffic
1,031 Citations2009Srikanth Kandula, Sudipta Sengupta +3 more
The nature of traffic in data centers is explored, designed to support the mining of massive data sets, and a petabyte of measurements over two months are obtained, from which detailed views of traffic and congestion conditions and patterns are obtained.
On designing improved controllers for AQM routers supporting TCP flows
959 Citations2002C.V. Hollot, Vishal Misra +2 more
A previously developed linearized model of TCP and active queue management (AQM) is studied, and the proportional integral (PI) controller is shown to outperform RED significantly.
IEEE/ACM Transactions on NetworkingFAST TCP: Motivation, Architecture, Algorithms, Performance
956 Citations2006David X. Wei, Cheng Jin +2 more
FAST TCP is described, a new TCP congestion control algorithm for high-speed long-latency networks, from design to implementation, and its equilibrium and stability properties are characterized.
TCP Vegas
683 Citations1994Lawrence S. Brakmo, Sean O’Malley +1 more
This paper motivates and describes the three key techniques employed by Vegas, and presents the results of a comprehensive experimental performance study—using both simulations and measurements on the Internet— of the Vegas and Reno implementations of TCP.
Sizing router buffers
682 Citations2004Guido Appenzeller, Isaac Keslassy +1 more
It is shown that a link with n flows requires no more than B = (overlineRTT x C) √n, for long-lived or short-lived TCP flows, because of the large number of flows multiplexed together on a single backbone link.
Adaptive RED: An Algorithm for Increasing the Robustness of RED's Active Queue Management
673 Citations2001Sally Floyd, Ramakrishna Gummadi +1 more
This revised version of Adaptive RED, which can be implemented as a simple extension within RED routers, removes the sensitivity to parameters that affect RED’s performance and can reliably achieve a specified target average queue length in a wide variety of traffic scenarios.
A Compound TCP Approach for High-Speed and Long Distance Networks
578 Citations2006Kun Tan, Jingmin Song +2 more
A novel Compound TCP (CTCP) approach, which is a synergy of delay-based and loss-based approach, and provides very good bandwidth scalability with improved RTT fairness, and at the same time achieves good TCP-fairness, irrelevant to the windows size is proposed.
Safe and effective fine-grained TCP retransmissions for datacenter communication
431 Citations2009Vijay Vasudevan, Amar Phanishayee +6 more
This paper uses high-resolution timers to enable microsecond-granularity TCP timeouts and shows that eliminating the minimum retransmission timeout bound is safe for all environments, including the wide-area.
Understanding TCP incast throughput collapse in datacenter networks
402 Citations2009Yanpei Chen, Rean Griffith +3 more
Practical guide to controlled experiments on the web
383 Citations2007Ron Kohavi, Randal M. Henne +1 more
This work provides a practical guide to conducting online experiments, and shares key lessons that will help practitioners in running trustworthy controlled experiments, including statistical power, sample size, and techniques for variance reduction.
ACM Transactions on Computer SystemsA binary feedback scheme for congestion avoidance in computer networks
367 Citations1990K. K. Ramakrishnan, Raj Jain
The scheme is distributed, adapts to the dynamic state of the network, converges to the optimal operating point, is quite simple to implement, and has low overhead.
Measurement and analysis of TCP throughput collapse in cluster-based storage systems
261 Citations2008Amar Phanishayee, Elie Krevat +5 more
This paper analyzes this Incast problem, explores its sensitivity to various system parameters, and examines the effectiveness of alternative TCP- and Ethernet-level strategies in mitigating the TCP throughput collapse.
IEEE/ACM Transactions on NetworkingExperimental Evaluation of TCP Protocols for High-Speed Networks
247 Citations2007Yee-Ting Li, Douglas J. Leith +1 more
It is found that both scalable-TCP and FAST- TCP consistently exhibit substantial unfairness, even when competing flows share identical network path characteristics.
IEEE/ACM Transactions on NetworkingThe synchronization of periodic routing messages
245 Citations1994Sally Floyd, Van Jacobson
The authors study the synchronization of periodic routing messages, and offer guidelines on how to avoid inadvertent synchronization, and show that synchronization can be avoided by the addition of randomization to the traffic sources and quantify how much randomization is necessary.
Lecture notes in computer scienceProcessor Sharing Flows in the Internet
229 Citations2005Nandita Dukkipati, Masayoshi Kobayashi +2 more
This paper explores how a new congestion control algorithm — Rate Control Protocol (RCP) — comes much closer to emulating PS over a broad range of operating conditions, and shows that under a wide range of traffic characteristics and network conditions, RCP’s performance is very close to ideal processor sharing.
The Nature of Datacenter Traffic: Measurements & Analysis
170 Citations2009Srikanth Kandula, Sudipta Sengupta +3 more
ACM SIGCOMM Computer Communication ReviewPart II
150 Citations2005Gaurav Raina, Don Towsley +1 more
Control theory is used to address the question of how to size the buffers in core Internet routers and shows that small buffers actually promote desynchronization--a virtuous circle.
One more bit is enough
140 Citations2005Yong Xia, Lakshminarayanan Subramanian +2 more
A simple, low-complexity protocol, called Variable-structure congestion Control Protocol (VCP), is designed and implemented that leverages only the existing two ECN bits for network congestion feedback, and yet achieves comparable performance to XCP, ie high utilization, low persistent queue length, negligible packet loss rate, and reasonable fairness.
On designing and deploying internet-scale services
104 Citations2007James R. Hamilton
This paper summarizes the best practices accumulated over many years in scaling some of the largest services at MSN and Windows Live.
AF-QCN: Approximate Fairness with Quantized Congestion Notification for Multi-tenanted Data Centers
103 Citations2010Abdul Kabbani, Mohammad Alizadeh +3 more
An algorithm is developed, called AF-QCN (for Approximately Fair QCN), which ensures a faster convergence to fairness than QCN, maintains this fairness at fine-grained time scales, and provides programmable weighted fair bandwidth shares to flows/flow-classes.
Congestion Control for Small Buffer High Speed Networks
57 Citations2007Yue Gu, Don Towsley +2 more
This paper presents a new congestion controller that allows a TCP connection to achieve arbitrarily large bandwidths without demanding the loss probability go to zero, and shows its performance to be superior to TCP NewReno in a variety of environments.
Experimental evaluation of Cubic-TCP
53 Citations2008Douglas J. Leith, Robert Shorten +1 more
An initial experimental evaluation of the recently proposed Cubic-TCP algorithm is presented using a suite of benchmark tests that have been recently proposed in the literature.
ACM SIGCOMM Computer Communication ReviewStability and fairness of explicit congestion control with small buffers
50 Citations2008F. P. Kelly, Gaurav Raina +1 more
This paper develops a variant of RCP (rate control protocol) that achieves α-fairness when buffers are small, including proportional fairness as the case α = 1, and establishes a simple decentralized sufficient condition for local stability.
Congestion Control using Efficient Explicit Feedback
35 Citations2009Ihsan Ayyub Qazi, Lachlan L. H. Andrew +1 more
Using extensive packet-level simulations, the efficacy of BMCC is assessed and comparisons with several proposed schemes are performed and analytical models that predict and provide insights into the convergence properties of the protocol are presented.