SAMOA

Published 13 May 2013

Gianmarco De Francisci Morales

Citations65

TL;DR

This talk offers a sneak preview of SAMOA, an upcoming platform for mining dig data streams in a cluster/cloud environment that features a pluggable architecture that allows it to run on several distributed stream processing engines such as S4 and Storm.

Abstract

Social media and user generated content are causing an ever growing data deluge. The rate at which we produce data is growing steadily, thus creating larger and larger streams of continuously evolving data. Online news, micro-blogs, search queries are just a few examples of these continuous streams of user activities. The value of these streams relies in their freshness and relatedness to ongoing events. However, current (de-facto standard) solutions for big data analysis are not designed to deal with evolving streams.

Keywords

Computer ScienceDecision Sciences

Communications of the ACMMapReduce

18,538 Citations2008Jay B. Dean, Sanjay Ghemawat

This presentation explains how the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks.

View PDF

ACM SIGKDD Explorations NewsletterThe WEKA data mining software

17,849 Citations2009Mark Hall, Eibe Frank +4 more

This paper provides an introduction to the WEKA workbench, reviews the history of the project, and, in light of the recent 3.6 stable release, briefly discusses what has been added since the last stable version (Weka 3.4) released in 2003.

The Design of the Borealis Stream Processing Engine

1,232 Citations2005Daniel J. Abadi, Yanif Ahmad +10 more

This paper outlines the basic design and functionality of Borealis, and presents a highly flexible and scalable QoS-based optimization model that operates across server and sensor networks and a new fault-tolerance model with flexible consistency-availability trade-offs.

Research Commons (University of Waikato)MOA: Massive Online Analysis

1,049 Citations2010Albert Bifet, Geoffrey Holmes +2 more

View PDF

S4: Distributed Stream Computing Platform

905 Citations2010Leonardo Neumeyer, Bruce Robbins +2 more

The architecture resembles the Actors model, providing semantics of encapsulation and location transparency, thus allowing applications to be massively concurrent while exposing a simple programming interface to application developers.

Journal of Machine Learning ResearchMOA: Massive Online Analysis

774 Citations2010BifetAlbert, HolmesGeoff +2 more

MOA includes a collection of offline and online methods as well as tools for evaluation that implements boosting, bagging, and Hoeffding Trees, all with and without Naive Bayes classifiers at the leaves.

Proceedings of the VLDB EndowmentHaLoop

688 Citations2010Yingyi Bu, Bill Howe +2 more

HaLoop is presented, a modified version of the Hadoop MapReduce framework that is designed to serve iterative applications and dramatically improves their efficiency by making the task scheduler loop-aware and by adding various caching mechanisms.

MapReduce online

682 Citations2010Tyson Condie, Neil Conway +4 more

A modified version of the Hadoop MapReduce framework that supports online aggregation, which allows users to see "early returns" from a job as it is being computed and also supports continuous queries, which enable MapReduce programs to be written for applications such as event monitoring and stream processing.

ACM Transactions on AlgorithmsOn distributing symmetric streaming computations

66 Citations2010Jon Feldman, S. Muthukrishnan +3 more

It is shown that in principle, mud algorithms are equivalent in power to symmetric streaming algorithms, and any symmetric (order-invariant) function that can be computed by a streaming algorithm can be computed by a mud algorithm, with comparable space and communication complexity.

DEDUCE

48 Citations2010Vibhore Kumar, Henrique Andrade +2 more

Deduce is presented, which extends IBM's System S stream processing middleware with support for MapReduce by providing language and runtime support for easily specifying and embedding Map Reduce jobs as elements of a larger data-flow.