Windows Azure Storage
Published 23 October 2011
Brad Calder, Wang Ju, Aaron Ogus, Niranjan Nilakantan, Arild Skjolsvold, Sam McKelvie
Citations799
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The WAS architecture, global namespace, and data model is described, as well as its resource provisioning, load balancing, and replication systems.
Abstract
Windows Azure Storage (WAS) is a cloud storage system that provides customers the ability to store seemingly limitless amounts of data for any duration of time. WAS customers have access to their data from anywhere at any time and only pay for what they use and store. In WAS, data is stored durably using both local and geographic replication to facilitate disaster recovery. Currently, WAS storage comes in the form of Blobs (files), Tables (structured storage), and Queues (message delivery). In this paper, we describe the WAS architecture, global namespace, and data model, as well as its resource provisioning, load balancing, and replication systems.
Keywords
Computer Science
ACM SIGOPS Operating Systems ReviewThe Google file system
5,003 Citations2003Sanjay Ghemawat, Howard Gobioff +1 more
Journal of the Society for Industrial and Applied MathematicsPolynomial Codes Over Certain Finite Fields
3,564 Citations1960I.S. Reed, G. Solomon
A mapping of m symbols into 2 symbols will be shown to be (2 m)/2 or ( 2 m 1)/2 symbol correcting, depending on whether m is even or odd.
Dynamo
3,454 Citations2007Giuseppe DeCandia, Deniz Hastorun +7 more
D Dynamo is presented, a highly available key-value storage system that some of Amazon's core services use to provide an "always-on" experience and makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use.
ACM Transactions on Computer SystemsBigtable
3,432 Citations2008Fay W. Chang, Jay B. Dean +7 more
The simple data model provided by Bigtable is described, which gives clients dynamic control over data layout and format, and the design and implementation of Bigtable are described.
ACM Transactions on Computer SystemsThe part-time parliament
2,714 Citations1998Leslie Lamport
The Paxon parliament's protocol provides a new way of implementing the state machine approach to the design of distributed systems.
ACM SIGOPS Operating Systems ReviewCassandra
2,630 Citations2010Avinash Lakshman, Prashant Malik
Cassandra is a distributed storage system for managing very large amounts of structured data spread out across many commodity servers, while providing highly available service with no single point of failure.
Bigtable: a distributed storage system for structured data
1,955 Citations2006Fay W. Chang, Jay B. Dean +7 more
ACM Transactions on Database SystemsOn optimistic methods for concurrency control
1,439 Citations1981H. T. Kung, John T. Robinson
In this paper, two families of nonlocking concurrency controls are presented and the methods used are "optimistic" in the sense that they rely mainly on transaction backup as a control mechanism, “hoping” that conflicts between transactions will not occur.
The Google file system
1,366 Citations2003Sanjay Ghemawat, Howard Gobioff +1 more
This paper presents file system interface extensions designed to support distributed applications, discusses many aspects of the design, and reports measurements from both micro-benchmarks and real world use.
Acta InformaticaThe log-structured merge-tree (LSM-tree)
1,276 Citations1996Patrick E. O׳Neil, E.T. Cheng +2 more
The log-structured mergetree (LSM-tree) is a disk-based data structure designed to provide low-cost indexing for a file experiencing a high rate of record inserts (and deletes) over an extended period.
The dangers of replication and a solution
1,027 Citations1996Jim Gray, Pat Helland +2 more
A new two-tier replication algorithm is proposed that allows mobile (disconnected) applications to propose tentative update transactions that are later applied to a master copy, to avoid the instability of other replication schemes.
Proceedings of the VLDB EndowmentPNUTS
960 Citations2008Brian F. Cooper, Raghu Ramakrishnan +7 more
PNUTS provides data storage organized as hashed or ordered tables, low latency for large numbers of concurrent requests including updates and queries, and novel per-record consistency guarantees and utilizes automated load-balancing and failover to reduce operational complexity.
Operating Systems Design and ImplementationThe Chubby lock service for loosely-coupled distributed systems
899 Citations2006Mike Burrows
The paper describes the initial design and expected use, compares it with actual use, and explains how the design had to be modified to accommodate the differences.
Managing update conflicts in Bayou, a weakly connected replicated storage system
814 Citations1995Doug Terry, Marvin Theimer +4 more
The motivation for and design of these mechanisms for conflict detection and per -write conflict resolution based on client-provid ed procedures are presented and the experiences gained with an initial implementation of the system are described.
Towards robust distributed systems (abstract)
784 Citations2000Eric A. Brewer
Several issues in an attempt to clean up the way the authors think about distributed systems, including the fault model, high availability, graceful degradation, data consistency, evolution, composition, and autonomy are looked at.
New directions in traffic measurement and accounting
742 Citations2001Cristian Estan, George Varghese
This paper describes two novel and scalable schemes for measuring only "heavy" flows whose traffic is above some threshold such as 1% of the link and proposes a new form of accounting called threshold accounting which generalizes the familiar notions of usage-based and duration based pricing.
Megastore: Providing Scalable, Highly Available Storage for Interactive Services
654 Citations2011Jason D. Baker, Chris T. Bond +8 more
Megastore provides fully serializable ACID semantics within ne-grained partitions of data, which allows us to synchronously replicate each write across a wide area network with reasonable latency and support seamless failover between datacenters.
New directions in traffic measurement and accounting
414 Citations2002Cristian Estan, George Varghese
This paper introduces a paradigm shift for measurement by concentrating only on large flows --- those above some threshold such as 0.1% of the link capacity, and proposes two novel and scalable algorithms for identifying the large flows: sample and hold and multistage filters, which take a constant number of memory references per packet and use a small amount of memory.
Chain replication for supporting high throughput and availability
374 Citations2004Robbert van Renesse, Fred B. Schneider
Besides outlining the chain replication protocols themselves, simulation experiments explore the performance characteristics of a prototype implementation and several object-placement strategies (including schemes based on distributed hash table routing) are discussed.
<i>DCD</i> —disk caching disk
143 Citations1996Yiming Hu, Qing Yang
This paper presents a novel disk storage architecture called DCD, Disk Caching Disk, for the purpose of optimizing I/O performance, which can be applied directly to current file systems without the need of changing the operating system.
File and Storage TechnologiesSnapMirror: File-System-Based Asynchronous Mirroring for Disaster Recovery
138 Citations2002Russel H. Patterson, Stephen Manley +4 more
Snap-Mirror is presented, an asynchronous mirroring technology that leverages file system snapshots to ensure the consistency of the remote mirror and optimize data transfer, and exploiting file system knowledge of deletions is critical to achieving any reduction for no-overwrite file systems such as WAFL and LFS.
QueueGFS: Evolution on Fast-forward
71 Citations2009Marshall Kirk McKusick, Sean Quinlan
During the early stages of development at Google, the initial thinking did not include plans for building a new file system, but while work was still being done on one of the earliest versions of the company’s crawl and indexing system, however, it became quite clear to the core engineers that they really had no other choice, and GFS (Google File System) was born.
Object storage on CRAQ: high-throughput chain replication for read-mostly workloads
61 Citations2009Jeff Terrace, Michael J. Freedman
Additional design and implementation considerations for geo-replicated CRAQ storage across multiple datacenters to provide locality-optimized operations are explored and multi-object atomic updates and multicast optimizations for large-object updates are discussed.
Profiling over Adaptive Ranges
7 Citations2006Shashidhar Mysore, Banit Agrawal +3 more