Big Data: Issues and Challenges Moving Forward

Published 1 January 2013

Stephen H. Kaisler, Frank Armour, J. Alberto Espinosa, William H. Money

DOI10.1109/hicss.2013.645

Citations925

TL;DR

The issues and challenges are analyzed as a collaborative research program into methodologies for big data analysis and design are begun.

Abstract

Big data refers to data volumes in the range of exabytes (1018) and beyond. Such volumes exceed the capacity of current on-line storage systems and processing systems. Data, information, and knowledge are being created and collected at a rate that is rapidly approaching the exabyte/year range. But, its creation and aggregation are accelerating and will approach the zettabyte/year range within a few years. Volume is only one aspect of big data; other attributes are variety, velocity, value, and complexity. Storage and data transport are technology issues, which seem to be solvable in the near-term, but represent longterm challenges that require research and new paradigms. We analyze the issues and challenges as we begin a collaborative research program into methodologies for big data analysis and design.

Keywords

Computer Science

Policy SciencesDilemmas in a general theory of planning

15,300 Citations1973Horst W. J. Rittel, Melvin M. Webber

View PDF

Choice Reviews OnlineThe black swan: the impact of the highly improbable

3,505 Citations2007

Communications of the ACMThe pathologies of big data

646 Citations2009Adam Jacobs

Scale up your datasets enough and your apps come undone; scale up too much and they come undone.

View PDF

SSRN Electronic JournalSix Provocations for Big Data

417 Citations2011danah boyd, Kate Crawford

Given the rise of Big Data as both a phenomenon and a methodological persuasion, it is time to start critically interrogating this phenomenon, its assumptions, and its biases.

View PDF

ScienceAgencies Rally to Tackle Big Data

95 Citations2012Jeffrey Mervis

A federal effort to improve the nation's ability to manage, understand, and act upon the 1.2 zettabytes of electronic data generated each year and computer scientists welcome the spotlight that the White House is shining on big-data research.

A Decision Framework for Cloud Computing

52 Citations2012Stephen H. Kaisler, William H. Money +1 more

A preliminary decision framework is developed to assist managers who are determining which cloud solution matches their specific requirements and evaluating the numerous commercial claims (in many cases unsubstantiated) of a cloud's value.

Communications of the ACMResearchers' big data crisis; understanding design and functionality

25 Citations2012Michael Stonebraker, Jason Hong

Michael Stonebraker issues a call to arms about research groups' data-management problems, and Jason Hong discusses the nature of functionality with respect to design.

Data Analysis Challenges

14 Citations2008D. Meiron, S. Cazares +11 more

JASON finds that DOD/IC data requirements are certainly significant, but not unmanageable given the capabilities of current and projected storage technology.

Assessing Nation-State Instability and Failure

9 Citations2006Robert L. Popp, Stephen H. Kaisler +7 more

The motivation and rationale for the program, the Q/CSS models and mechanisms, and results from some of the models are described, and the utility of the results can provide added value for decision-making problems around planning, intelligence analysis, information operations and training is presented.