Document representation based on semantic smoothed topic model
Published 1 May 2016
Ying Liu, Wei Song, Lizhen Liu, Hanshi Wang
Citations4
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The experimental results suggest that the semantic smoothed topic model proposed is useful for capturing the semantic of text to alleviating polysemy and synonyms problem and data sparseness problem.
Abstract
The goal of document representation is to capture certain feature of the document. Many existing document representation methods are based on bag-of-words and ignore semantic relevance between words in the document. There we proposed a semantic smoothed topic model to represent document. It takes semantic similarity into consideration for topic of document. We conducted two experiments utilizing this method for text classification and information retrieval task. The experimental results suggest that our method is useful for capturing the semantic of text to alleviating polysemy and synonyms problem and data sparseness problem.
Keywords
Computer Science
Journal of Machine Learning ResearchLatent dirichlet allocation
27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more
arXiv (Cornell University)Distributed Representations of Words and Phrases and their Compositionality
18,086 Citations2013Tomáš Mikolov, Ilya Sutskever +3 more
This paper presents a simple method for finding phrases in text, and shows that learning good vector representations for millions of phrases is possible and describes a simple alternative to the hierarchical softmax called negative sampling.
arXiv (Cornell University)Efficient Estimation of Word Representations in Vector Space
18,050 Citations2013Tomáš Mikolov, Kai Chen +2 more
Two novel model architectures for computing continuous vector representations of words from very large data sets are proposed and it is shown that these vectors provide state-of-the-art performance on the authors' test set for measuring syntactic and semantic word similarities.
Computer Networks and ISDN SystemsThe anatomy of a large-scale hypertextual Web search engine
15,828 Citations1998Sergey Brin, Lawrence M. Page
This paper provides an in-depth description of Google, a prototype of a large-scale search engine which makes heavy use of the structure present in hypertext and looks at the problem of how to effectively deal with uncontrolled hypertext collections where anyone can publish anything they want.
Journal of the American Society for Information ScienceIndexing by latent semantic analysis
12,677 Citations1990Scott Deerwester, Susan Dumais +3 more
The PageRank Citation Ranking : Bringing Order to the Web
12,645 Citations1999Lawrence M. Page, Sergey Brin +2 more
This paper describes PageRank, a mathod for rating Web pages objectively and mechanically, effectively measuring the human interest and attention devoted to them, and shows how to efficiently compute PageRank for large numbers of pages.
arXiv (Cornell University)Efficient Estimation of Word Representations in Vector Space
11,710 Citations2013Tomáš Mikolov, Kai Chen +2 more
Information Processing & ManagementTerm-weighting approaches in automatic text retrieval
9,532 Citations1988Gerard Salton, Chris Buckley
This paper summarizes the insights gained in automatic term weighting, and provides baseline single term indexing models with which other more elaborate content analysis procedures can be compared.
Journal of the ACMAuthoritative sources in a hyperlinked environment
9,060 Citations1999Jon Kleinberg
This work proposes and test an algorithmic formulation of the notion of authority, based on the relationship between a set of relevant authoritative pages and the set of “hub pages” that join them together in the link structure, and has connections to the eigenvectors of certain matrices associated with the link graph.
Empirical Methods in Natural Language ProcessingTextRank: Bringing Order into Text
3,347 Citations2004Rada Mihalcea, Paul Tarau
TextRank, a graph-based ranking model for text processing, is introduced and it is shown how this model can be successfully used in natural language applications.
Linguistic Regularities in Continuous Space Word Representations
2,882 Citations2013Tomáš Mikolov, Wen-tau Yih +1 more
The vector-space word representations that are implicitly learned by the input-layer weights are found to be surprisingly good at capturing syntactic and semantic regularities in language, and that each relationship is characterized by a relation-specific vector offset.
Journal of Machine Learning ResearchA neural probabilistic language model
2,663 Citations2003BengioYoshua, DucharmeRéjean +2 more
This work reports on experiments using neural networks for the probability function, showing on two text corpora that the proposed approach very significantly improves on a state-of-the-art trigram model.
Elsevier eBooksNewsWeeder: Learning to Filter Netnews
2,022 Citations1995Ken Lang
The results show that a learning algorithm based on the Minimum Description Length (MDL) principle was able to raise the percentage of interesting articles to be shown to users from 14% to 52% on average.
Topic-sensitive PageRank
1,524 Citations2002Taher H. Haveliwala
A set of PageRank vectors are proposed, biased using a set of representative topics, to capture more accurately the notion of importance with respect to a particular topic, and are shown to generate more accurate rankings than with a single, generic PageRank vector.
Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors
1,379 Citations2014Marco Baroni, Georgiana Dinu +1 more
An extensive evaluation of context-predicting models with classic, count-vector-based distributional semantic approaches, on a wide range of lexical semantics tasks and across many parameter settings shows that the buzz around these models is fully justified.
University of North Texas Digital Library (University of North Texas)TextRank: Bringing Order into Texts
633 Citations2004Mihalcea, Rada, 1974-, Tarau, Paul
A Hierarchical Neural Autoencoder for Paragraphs and Documents
518 Citations2015Jiwei Li, Thang Luong +1 more
This paper introduces an LSTM model that hierarchically builds an embedding for a paragraph from embeddings for sentences and words, then decodes this embedding to reconstruct the original paragraph and evaluates the reconstructed paragraph using standard metrics to show that neural models are able to encode texts in a way that preserve syntactic, semantic, and discourse coherence.
Semi-supervised learning of compact document representations with deep networks
223 Citations2008Marc ' Aurelio Ranzato, Martin Szummer
An algorithm to learn text document representations based on semi-supervised autoencoders that are stacked to form a deep network that can be trained efficiently on partially labeled corpora, producing very compact representations of documents, while retaining as much class information and joint word statistics as possible.
An Analytical Comparison of Approaches to Personalizing PageRank
184 Citations2003Taher H. Haveliwala, Sepandar Kamvar +1 more
This work analytically compares three recent approaches to personalizing PageRank and discusses the tradeoffs of each one.
Supervised semantic indexing
95 Citations2009Bing Bai, Jason Weston +6 more
This article proposes Supervised Semantic Indexing (SSI), an algorithm that is trained on (query, document) pairs of text documents to predict the quality of their match and proposes several improvements to the basic model, including low rank (but diagonal preserving) representations, and correlated feature hashing (CFH).
Neural Information Processing SystemsLarge Margin Taxonomy Embedding for Document Categorization
62 Citations2008Kilian Q. Weinberger, Olivier Chapelle
This work presents a novel algorithm that goes beyond hierarchical classification and estimates the latent semantic space that underlies the class hierarchy and shows that the optimization is convex and can be solved efficiently for large data sets.
From sBoW to dCoT marginalized encoders for text representation
26 Citations2012Zhixiang Xu, Minmin Chen +2 more
This paper proposes Dense Cohort of Terms (dCoT), an unsupervised algorithm to learn improved sBoW document features and demonstrates empirically, on several benchmark datasets, that dCoT features significantly improve the classification accuracy across several document classification tasks.
Topic Modeling in Semantic Space with Keywords
18 Citations2015Xiaojia Pu, Rong Jin +3 more
This paper proposes a novel method called TDCS (Topic Distilling with Compressive Sensing) for explicit and accurate modeling the topic implied by several keywords, which is efficient for subsequent text mining tasks.
IEEE/ACM Transactions on Audio Speech and Language ProcessingCompact Multiview Representation of Documents Based on the Total Variability Space
10 Citations2015Mohamed Morchid, Mohamed Bouallegue +4 more
This paper proposes a method based on a two-step process that expands the representation space by using a set of topic spaces and compacts the represented space by removing poorly relevant dimensions, which provides a view-independent representation of documents while extracting complementary information from a massive multi-view representation.