The Google file system

Published 19 October 2003

Sanjay Ghemawat, Howard Gobioff, Shun-Tak A. Leung

Citations1,366

TL;DR

This paper presents file system interface extensions designed to support distributed applications, discusses many aspects of the design, and reports measurements from both micro-benchmarks and real world use.

Abstract

article Share on The Google file system Authors: Sanjay Ghemawat Google GoogleView Profile , Howard Gobioff Google GoogleView Profile , Shun-Tak Leung Google GoogleView Profile Authors Info & Claims ACM SIGOPS Operating Systems ReviewVolume 37Issue 5December 2003 pp 29–43https://doi.org/10.1145/1165389.945450Online:19 October 2003Publication History 2,957citation60,150DownloadsMetricsTotal Citations2,957Total Downloads60,150Last 12 Months3,038Last 6 weeks280 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Keywords

Computer Science

A case for redundant arrays of inexpensive disks (RAID)

1,427 Citations1988David A. Patterson, Garth A. Gibson +1 more

Five levels of RAIDs are introduced, giving their relative cost/performance, and a comparison to an IBM 3380 and a Fujitsu Super Eagle is compared.

ACM Transactions on Computer SystemsScale and performance in a distributed file system

1,412 Citations1988John H. Howard, Michael L. Kazar +5 more

This paper examines the consequences of the design decision to transfer whole files between servers and workstations rather than some smaller unit such as records or blocks, as almost all other distributed file systems do, and compares the whole file transfer strategy with that of a block-oriented file system, Sun Microsystems' NFS.

GPFS: A Shared-Disk File System for Large Computing Clusters

1,224 Citations2002Frank Schmuck, Roger Haskin

Frangipani

380 Citations1997Chandramohan A. Thekkath, Timothy Mann +1 more

Initial measurements indicate that Frangipani has excellent single-server performance and scales well as servers are added, and can be exported to untrusted machines using ordinary network file access protocols.

Replication in the harp file system

274 Citations1991Barbara Liskov, Sanjay Ghemawat +4 more

Preliminary performance results indicate that Harp provides equal or better response time and system capacity than an unreplicated implementation of NFS that uses Unix files directly.

View PDF

Serverless network file systems

198 Citations1995Thomas E. Anderson, Michael Dahlin +4 more

A new paradigm for network file system design, serverless network file systems that utilizes workstations cooperating as peers to provide all file system services to provide better performance and scalability than traditional file systems.

A cost-effective, high-bandwidth storage architecture

178 Citations1998Garth A. Gibson, David F. Nagle +8 more

Measurements of the prototype NASD system show that these services can be cost-effectively integrated into a next generation disk drive ASK, and show scaluble bandwidth for NASD-specialized filesystems.

Cluster I/O with River

175 Citations1999Remzi H. Arpaci-Dusseau, Eric Anderson +5 more

This work introduces River, a data-flow programming environment and I/O substrate for clusters of computers based on two simple design features: a high-performance distributed queue, and a storage redundancy mechanism called graduated declustering.

View PDF

SWIFT: USING DISTRIBUTED DISK STRIPING TO PROVIDE HIGH I/O DATA RATES

128 Citations1991Luis Valencia–Cabrera, Darrell D. E. Long

It is shown that the data rates scale well in the number of storage devices, and that by replacing the most highly stressed components by more powerful ones the data rate of the entire system increase significantly.