Parallel User Profiling Based on Folksonomy for Large Scaled Recommender Systems: An Implimentation of Cascading MapReduce

Published 1 December 2010

Huizhi Liang, Jim Hogan, Yue Xu

DOI10.1109/icdmw.2010.161

Citations20

TL;DR

A parallel user profiling approach and a scalable recommender system to solve the scalability problem of the current user profiling and recommender systems is proposed.

Abstract

The Large scaled emerging user created information in web 2.0 such as tags, reviews, comments and blogs can be used to profile users' interests and preferences to make personalized recommendations. To solve the scalability problem of the current user profiling and recommender systems, this paper proposes a parallel user profiling approach and a scalable recommender system. The current advanced cloud computing techniques including Hadoop, MapReduce and Cascading are employed to implement the proposed approaches. The experiments were conducted on Amazon EC2 Elastic MapReduce and S3 with a real world large scaled dataset from Delicious website.

Keywords

Computer Science

IEEE Transactions on Knowledge and Data EngineeringToward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions

10,115 Citations2005Gediminas Adomavičius, Alexander Tuzhilin

This paper presents an overview of the field of recommender systems and describes the current generation of recommendation methods that are usually classified into the following three main categories: content-based, collaborative, and hybrid recommendation approaches.

The Google file system

1,348 Citations2003Sanjay Ghemawat, Howard Gobioff +1 more

Tag-based social interest discovery

335 Citations2008Xin Li, Lei Guo +1 more

An Internet Social Interest Discovery system, ISID, to discover the common user interests and cluster users and their saved URLs by different interest topics, and shows that ISID can effectively cluster similar documents by interest topics and discover user communities with common interests no matter if they have any online connections.

Pairwise document similarity in large collections with MapReduce

204 Citations2008Tamer Elsayed, Jimmy Lin +1 more

This paper presents a MapReduce algorithm for computing pairwise document similarity in large document collections that exhibits linear growth in running time and space in terms of the number of documents.

View PDF

Connecting users and items with weighted tags for personalized item recommendations

85 Citations2010Huizhi Liang, Yue Xu +3 more

This paper proposes to use the multiple relationships among users, items and tags to find the semantic meaning of each tag for each user individually, and explores the user and item-based collaborative filtering combined with the content filtering approach.

View PDF

Extracting user profiles from large scale data

34 Citations2010Michal Shmueli-Scheuer, Haggai Roitman +3 more

This work first describes an efficient user profiling framework with high user profiling quality guarantees, then describes a scalable implementation of the proposed framework in Apache Hadoop and discusses its challenges.

Beyond online aggregation

33 Citations2010Joos-Hendrik Böse, Artur Andrzejak +1 more

It is shown how an online version of the Map-Reduce programming model can be used to implement such algorithms, and a solution for the "hardest" class of these algorithms - those requiring multiple Map- Reduce phases is proposed.

Parallelizing Random Walk with Restart for large-scale query recommendation

4 Citations2010Meng-Fen Chiang, Tsung‐Wei Wang +1 more

A parallel approach for the implementation of RWR in a cloud computing environment is designed and implemented for query recommendation in Yahoo! Asia Knowledge Plus, which contains a huge amount of Question and Answer Web pages (abbreviated as QA).