10.1162/153244303322533223
Applied Physics LettersPublished 1 January 2000Open access
Citations1,771
SJR quartileQ1
SJR score0.90
SNIP1.00
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work proposes to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences.
Abstract
This DOI is not currently attached to any metadata records. DOIs can’t actually ever be deleted (they’re persistent), but sometimes our members create DOIs in error. We do have a process to approximate deletion which we follow only in rare cases where the DOI has been genuinely created in error, and most crucially, if the DOI has never been published anywhere online or in print and never otherwise distributed to or communicated with anyone (authors, readers, reviewers, etc.
Keywords
Arts and HumanitiesHealth Professions
Journal of the American Society for Information ScienceIndexing by latent semantic analysis
12,677 Citations1990Scott Deerwester, Susan Dumais +3 more
Neural ComputationTraining Products of Experts by Minimizing Contrastive Divergence
5,017 Citations2002Geoffrey E. Hinton
A product of experts (PoE) is an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary because it is hard even to approximate the derivatives of the renormalization term in the combination rule.
Computer Speech & LanguageAn empirical study of smoothing techniques for language modeling
2,077 Citations1999Stanley F. Chen, Joshua Goodman
Improved backing-off for M-gram language modeling
1,490 Citations2002Reinhard Kneser, Hermann Ney
This paper proposes to use distributions which are especially optimized for the task of back-off, which are quite different from the probability distributions that are usually used for backing-off.
Birkhäuser Basel eBooksThe MPI Message Passing Interface Standard
1,300 Citations1994L. J. Clarke, I. Glendinning +1 more
An overview of the MPI initiative and the standard interface is presented, in particular those aspects which merge demonstrated research with common practice.
Distributional clustering of English words
994 Citations1993Fernando Pereira, Naftali Tishby +1 more
Lecture notes in computer scienceEfficient BackProp
867 Citations1998Yann LeCun, Léon Bottou +2 more
Many undesirable behaviors of backprop can be avoided with tricks that are rarely exposed in serious technical publications, and this paper gives some of those tricks, and offers explanations of why they work.
Distributional clustering of words for text classification
683 Citations1998Lee D. Baker, Andrew Kachites McCallum
This paper describes the application of Distributional Clustering to document classi(cid:12)cation and shows that it can reduce the feature dimensionality by three orders of magnitude and lose only 2% accuracy, better than Latent Semantic In-dexing, class-based clustering, feature selection by mutual information, or Markov-blanket-based feature selection.
Computer Speech & LanguageA bit of progress in language modeling
463 Citations2001Joshua Goodman
A combination of all techniques together to a Katz smoothed trigram model with no count cutoffs achieves perplexity reductions between 38 and 50% (1 bit of entropy), depending on training data size, as well as a word error rate reduction of 8.9%.
Journal of Computational BiologyImproving Prediction of Protein Secondary Structure Using Structured Neural Networks and Multiple Sequence Alignments
158 Citations1996Søren Riis, Anders Krogh
It is shown that the network outputs can be interpreted as estimated probabilities of correct prediction, and, therefore, these numbers indicate which residues are predicted with high confidence.
Cognitive ScienceNatural language processing with modular pdp networks and distributed lexicon
156 Citations1991Risto Miikkulainen, Michael A. Dyer
Quick Training of Probabilistic Neural Nets by Importance Sampling
149 Citations2003Yoshua Bengio, Jean-Sébastien Senécal
Inspired by the contrastive divergence model, sampling-based methods which require network passes only for the observed “positive example” and a few sampled negative example words are proposed and evaluated.
IEEE Transactions on Neural NetworksTaking on the curse of dimensionality in joint distributions using neural networks
120 Citations2000Samy Bengio, Yoshua Bengio
Modeling High-Dimensional Discrete Data with Multi-Layer Neural Networks
110 Citations1999Yoshua Bengio, Samy Bengio
A new architecture for modeling high-dimensional data that requires resources that grow only at most as the square of the number of variables is proposed, using a multi-layer neural network to represent the joint distribution of the variables as the product of conditional distributions.
Can artificial neural networks learn language models?
91 Citations2000Wei Xu, Alexander I. Rudnicky
This paper investigated an alternative way to build language models, i.e., using artificial neural networks to learn the language model, and shows that the neural network can learn a language model that has performance even better than standard statistical methods.
IEEE Transactions on Neural NetworksSequential neural text compression
84 Citations1996Jürgen Schmidhuber, Selina Heil
It is shown that neural networks may be promising tools for data compression without loss of information and compression ratios exceeding those of the widely used Lempel-Ziv algorithms.
Comparison of part-of-speech and automatically derived category-based language models for speech recognition
67 Citations2002Thomas Niesler, E. W. D. Whittaker +1 more
This paper compares various category-based language models when used in conjunction with a word-based trigram by means of linear interpolation to find the largest improvement with a model using automatically determined categories.
Self-organizing letter code-book for text-to-phoneme neural network model
30 Citations2000Kåre Jean Jensen, Søren Riis
An improved input coding method for a textto-phoneme (TTP) neural network model for speaker independent speech recognition systems that is jointly optimized with the TTP model ensuring that the coding is optimal in terms of overall performance.
Extracting distributed representations of concepts and relations from positive and negative propositions
17 Citations2000Alberto Paccanaro, Geoffrey E. Hinton
This paper presents results in two simple domains, which show that learning leads to good generalization in linear relational embedding and an extended formulation of LRE that solves both these problems.