Extracting user profiles from large scale data
Published 26 April 2010
Michal Shmueli-Scheuer, Haggai Roitman, David Carmel, Yosi Mass, David Konopnicki
Citations34
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work first describes an efficient user profiling framework with high user profiling quality guarantees, then describes a scalable implementation of the proposed framework in Apache Hadoop and discusses its challenges.
Abstract
In this work we present the details of a large scale user profiling framework that we developed here in IBM on top of Apache Hadoop. We address the problem of extracting and maintaining a very large number of user profiles from large scale data. We first describe an efficient user profiling framework with high user profiling quality guarantees. We then describe a scalable implementation of the proposed framework in Apache Hadoop and discuss its challenges.
Keywords
Computer SciencePhysics and AstronomySocial Sciences
Lecture notes in computer scienceLarge-Scale Parallel Collaborative Filtering for the Netflix Prize
696 Citations2008Y. Zhou, Dennis M. Wilkinson +2 more
This paper describes a CF algorithm alternating-least-squares with weighted-?-regularization(ALS-WR), which is implemented on a parallel Matlab platform and shows empirically that the performance of ALS-WR monotonically improves with both the number of features and thenumber of ALS iterations.
Adaptive web search based on user profile constructed without any effort from users
640 Citations2004Kazunari Sugiyama, Kenji Hatano +1 more
Experimental results show that search systems that adapt to each user's preferences can be achieved by constructing user profiles based on modified collaborative filtering with detailed analysis of user's browsing history in one day.
Lecture notes in computer scienceUser Profiles for Personalized Information Access
493 Citations2007Susan Gauch, Mirco Speretta +2 more
This chapter surveys some of the most popular techniques for collecting information about users, representing, and building user profiles and discusses in detail user profiles represented as weighted keywords, semantic networks, and weighted concepts.
WebMate
458 Citations1998Liren Chen, Katia Sycara
z Searching enhancement y Parallel search, searching keywords refinement, relevant feedback z Browsing assistant y Learning, Recommending, Alias, Monitoring bookmarks, More like, Sending page to friends, Prefetching hiperlinks z Offline browsing y downloading following pages, getting references and printing it out.
What makes a query difficult?
203 Citations2006David Carmel, Elad Yom‐Tov +2 more
This work addresses a novel model that captures the main components of a topic and the relationship between those components and topic difficulty and demonstrates the applicability of the difficulty model for several uses such as predicting query difficulty, predicting the number of topic aspects expected to be covered by the search results, and analyzing the findability of a specific domain.
Large-scale behavioral targeting
201 Citations2009Ye Chen, Dmitry Pavlov +1 more
This work designed and implemented a highly scalable and efficient solution to behavioral targeting using Hadoop MapReduce framework, and believes that this work makes significant contributions to solving large-scale machine learning problems of industrial relevance in general.
Learning implicit user interest hierarchy for context in personalization
159 Citations2003Hyoung R. Kim, Philip K. Chan
Enhancing cluster labeling using wikipedia
142 Citations2009David Carmel, Haggai Roitman +1 more
The experimental results reveal that the Wikipedia labels agree with manual labels associated by humans to a cluster, much more than with significant terms that are extracted directly from the text.
Self-adaptive user profiles for large-scale data delivery
83 Citations2002Uğur Çetintemel, Michael J. Franklin +1 more
This work proposes a novel incremental algorithm for constructing user profiles based on monitoring and user feedback that can be tuned to trade-off profile complexity and quality, suitable for use in large scale information filtering applications such as push based WWW page dissemination.
Lecture notes in computer scienceDynamic Adaptation Strategies for Long-Term and Short-Term User Profile to Personalize Search
73 Citations2007Lin Li, Zhenglu Yang +2 more
An adaptive scheme to learn the changes of user preferences from click-history data, and a novel rank mechanism to bias the search results of each user are introduced.
Proceedings of the VLDB EndowmentMaintaining dynamic channel profiles on the web
18 Citations2008Haggai Roitman, David Carmel +1 more
This work proposes a novel monitoring scheme that learns the channels' monitoring rates and describes a novelty detection filter that refines the monitoring rate according to the expected rate of novel content published on the channels.
Annotation of URLs
7 Citations2009Max Hinne, Wessel Kraaij +4 more
This work hypothesized that the queries associated with a URL could also be presented as useful URL metadata in a search engine result list, e.g. for helping to determine the semantic category of a URL.
IEEJ Transactions on Electronics Information and SystemsUser Profile Creation Using Genetic Algorithm with Kullback Leibler Divergence
1 Citations2006Hidekazu Yanagimoto, Sigeru Omatu
A user profile creation method using the Kullback Leibler divergence as a discriminant function which denotes to user's interest in the information filtering system and uses the real-coded genetic algorithm.