Comprehensive analysis of big data variety landscape
International Journal of Parallel Emergent and Distributed SystemsPublished 7 July 2014
Jemal Abawajy
Citations69
SJR quartileQ3
SJR score0.28
SNIP0.43
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This article presents a comprehensive analysis of big data variety and its adverse effects on the network performance and presents taxonomy of bigData variety and discusses various dimensions of thebig data variety features.
Abstract
Big data presents a remarkable opportunity for organisations to obtain critical intelligence to drive decisions and obtain insights as never before. However, big data generates high network traffic. Moreover, the continuous growth in the variety of network traffic due to big data variety has rendered the network to be one of the key big data challenges. In this article, we present a comprehensive analysis of big data variety and its adverse effects on the network performance. We present taxonomy of big data variety and discuss various dimensions of the big data variety features. We also discuss how the features influence the interconnection network requirements. Finally, we discuss some of the challenges each big data variety dimension presents and possible approach to address them.
Keywords
Computer Science
Big data: The next frontier for innovation, competition, and productivity
5,228 Citations2011James Manyika
The amount of data in the authors' world has been exploding, and analyzing large data sets will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus, according to research by MGI and McKinsey.
Spark: cluster computing with working sets
4,236 Citations2010Matei Zaharia, Mosharaf Chowdhury +3 more
Spark can outperform Hadoop by 10x in iterative machine learning jobs, and can be used to interactively query a 39 GB dataset with sub-second response time.
Pregel
3,534 Citations2010Grzegorz Malewicz, Matthew H. Austern +5 more
A model for processing large graphs that has been designed for efficient, scalable and fault-tolerant implementation on clusters of thousands of commodity computers, and its implied synchronicity makes reasoning about programs easier.
Dynamo
3,454 Citations2007Giuseppe DeCandia, Deniz Hastorun +7 more
D Dynamo is presented, a highly available key-value storage system that some of Amazon's core services use to provide an "always-on" experience and makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use.
ACM SIGOPS Operating Systems ReviewCassandra
2,630 Citations2010Avinash Lakshman, Prashant Malik
Cassandra is a distributed storage system for managing very large amounts of structured data spread out across many commodity servers, while providing highly available service with no single point of failure.
Bigtable: a distributed storage system for structured data
1,955 Citations2006Fay W. Chang, Jay B. Dean +7 more
ACM Computing SurveysProcessing flows of information
814 Citations2012Gianpaolo Cugola, Alessandro Margara
A general, unifying model is proposed to capture the different aspects of an IFP system and use it to provide a complete and precise classification of the systems and mechanisms proposed so far.
Managing data transfers in computer clusters with orchestra
559 Citations2011Mosharaf Chowdhury, Matei Zaharia +3 more
This work proposes a global management architecture and a set of algorithms that improve the transfer times of common communication patterns, such as broadcast and shuffle, and allow scheduling policies at the transfer level,such as prioritizing a transfer over other transfers.
CERN Document Server (European Organization for Nuclear Research)MongoDB: The Definitive Guide
456 Citations2010Kristina Chodorow, Michael Dirolf
This authoritative introduction to MongoDB will learn the many advantages of using document-oriented databases, and discover why MongoDB is a reliable, high-performance system that allows for almost infinite horizontal scalability.
Coflow
337 Citations2012Mosharaf Chowdhury, Ion Stoica
ComputerBig Data: New Opportunities and New Challenges [Guest editors' introduction]
272 Citations2013Katina Michael, Keith Miller
The authors can live with many of the uncertainties of big data for now, with the hope that its benefits will outweigh its harms, but they shouldnit blind ourselves to the possible irreversibility of changes-whether good or bad-to society.
Programming your network at run-time for big data applications
172 Citations2012Guohui Wang, T. S. Eugene Ng +1 more
This study uses Hadoop as an example to discuss the integrated network control architecture, job scheduling, topology and routing configuration mechanisms for Hadoops, and suggests that such an integrated control has great potential to improve application performance with relatively small configuration overhead.
Processing flows of information
120 Citations2011Alessandro Margara, Gianpaolo Cugola
This tutorial draws a general framework to analyze and compare the results achieved so far in the area of IFP systems, offering a systematic overview of the topic and highlighting a number of open issue that still need to be addressed in research.
IEEE Computer Graphics and ApplicationsThe Top 10 Challenges in Extreme-Scale Visual Analytics
110 Citations2012Pak Chung Wong, Han‐Wei Shen +3 more
A team of scientists and researchers discusses the top 10 challenges in extreme-scale visual analytics, evaluating the problems and challenges from both technical and social perspectives.
Computer law & security reviewLocation and tracking of mobile devices: Überveillance stalks the streets
81 Citations2013Katina Michael, Roger Clarke
This article investigates the dimensions of the problem of people-tracking through the devices that they carry, and calls on legislatures to address these problems, through both domestic laws and multilateral processes.
Institution of Engineering and Technology eBooksBig data analytics for security intelligence
76 Citations2021I. Sumaiya Thaseen, Aswani Kumar Cherukuri +2 more
This chapter presents a system called seabed, which facilitates efficient analytics on huge encrypted datasets, and outlines research directions for security intelligence by applying security analytics.
ACM SIGCOMM Computer Communication ReviewRethinking the physical layer of data center networks of the next decade
52 Citations2013Howard Wang, Yiting Xia +4 more
This work proposes an unconventional approach that leverages physical layer photonic technologies to build custom communication devices for accelerating each *-cast pattern, and integrates such devices into an application-driven, dynamically configurable photonics accelerated data center network.
Journal of the Association for Information SystemsBig Data – A State-of-the-Art
41 Citations2012Marco Pospiech, Carsten Felden
It is of interest how and whether the data provisioning is able to support enterprises in the handling and value creation of this emerging issue.
Riding the multimedia big data wave
30 Citations2013John R. Smith
A multi-layer architecture that incorporates capabilities for audio-visual feature extraction, machine learning and semantic modeling and provides a powerful framework for learning and classifying contents of multimedia data is described.
IEEE SpectrumThe gene machine and me
9 Citations2013Eliza Strickland
I want to learn my own biological secrets, so I'm not going to see just any man, but the mad scientist of genomics himself, the arrogant upstart of biotechnology, an inventor and entrepreneur who has upended the business of genetic sequencing once before-and now appears to be doing it again.
Store, schedule and switch - A new data delivery model in the big data era
8 Citations2013Weiqiang Sun, Fengqin Li +3 more
In the new data transfer model, a service level requirement is assigned (by the data source) to each big data transfer request, and the network stores the data until it find necessary, or enough network resource is available for that transfer.
Big Data AnalyticsUsing Graph Analytics for Big Data
2 Citations2013David Loshin
This chapter looks at business problems suited for graph analytics, what differentiates the problems from traditional approaches, and considerations for discovery vs. search analyses.
Theoretical study of dilute nitride 1.3 µm quantum well semiconductor lasers for short pulse generation: Effect of incorporation of N compositional fluctuations
1 Citations2013Xiao Sun, Nikolaos Vogiatzis +1 more