Local distance-based classification
Knowledge-Based SystemsPublished 8 April 2008
Manuel Laguı ́a, Juan Luis Castro
Citations15
SJR quartileQ1
SJR score1.93
SNIP2.34
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The bands distance has obtained the best overall results with 1-NN and k-NN classifiers when compared with other distances and confirms the good behavior of the new proposed classification method.
Abstract
In this paper, we have introduced a new method in which every training point learns what is happening in its neighborhood. So, a hyperplane is learned and associated to each point. With this hyperplane we can define the bands distance, a distance measure that bring closer or move away points depending on its classes. We have used this new distance in classification tasks and have performed tests over 68 datasets: 18 well-known UCI-Repository datasets, one private dataset, and 49 ad hoc synthetic datasets. We have used 10-fold cross-validation and, in order to compare the results of the classifiers, we have considered the mean accuracy and have also performed a paired two-tailored t-Student's test with a significance level of 95%. The results are encouraging and confirm the good behavior of the new proposed classification method. The bands distance has obtained the best overall results with 1-NN and k-NN classifiers when compared with other distances. Finally, we extract conclusions and outline some lines of future work.
Keywords
Computer Science
IEEE Transactions on Information TheoryNearest neighbor pattern classification
16,134 Citations1967Thomas M. Cover, Peter E. Hart
The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points, so it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.
Machine LearningInstance-Based Learning Algorithms
4,117 Citations1991David W. Aha, Dennis Kibler +1 more
This paper describes how storage requirements can be significantly reduced with, at most, minor sacrifices in learning rate and classification accuracy and extends the nearest neighbor algorithm, which has large storage requirements.
Machine LearningInstance-based learning algorithms
2,915 Citations1991David W. Aha, Dennis Kibler +1 more
This paper describes how storage requirements can be significantly reduced with, at most, minor sacrifices in learning rate and classification accuracy and extends the nearest neighbor algorithm, which has large storage requirements.
International Statistical ReviewDiscriminatory Analysis. Nonparametric Discrimination: Consistency Properties
2,473 Citations1989Evelyn Fix, J. L. Hodges
Journal of Artificial Intelligence ResearchImproved Heterogeneous Distance Functions
1,344 Citations1997D.R. Wilson, Tony Martinez
These new distance functions are designed to handle applications with nominal attributes, continuous attributes, or both and achieve higher classification accuracy on average than three previous distance functions on those datasets that have both nominal and continuous attributes.
Communications of the ACMToward memory-based reasoning
1,232 Citations1986Craig Stanfill, David L. Waltz
The intensive use of memory to recall specific episodes from the past—rather than rules—should be the foundation of machine reasoning.
Machine LearningA Weighted Nearest Neighbor Algorithm for Learning with Symbolic Features
663 Citations1993Scott Cost, Steven L. Salzberg
A nearest neighbor algorithm for learning in domains with symbolic features, which produces excellent classification accuracy on three problems that have been studied by machine learning researchers: predicting protein secondary structure, identifying DNA promoter sequences, and pronouncing English text.
International Journal of Man-Machine StudiesTolerating noisy, irrelevant and novel attributes in instance-based learning algorithms
406 Citations1992David W. Aha
This paper presents a comprehensive sequence of three incremental, edited nearest neighbor algorithms that tolerate attribute noise, determine relative attribute relevances, and accept instances described by novel attributes.
Machine LearningA weighted nearest neighbor algorithm for learning with symbolic features
371 Citations1993Scott Cost, Steven L. Salzberg
A nearest neighbor algorithm for learning in domains with symbolic features, which produces excellent classification accuracy on three problems that have been studied by machine learning researchers: predicting protein secondary structure, identifying DNA promoter sequences, and pronouncing English text.
Machine LearningA Nearest Hyperrectangle Learning Method
348 Citations1991Steven L. Salzberg
The results in these domains support the claim that NGE theory can be used to create compact representations with excellent predictive accuracy.
Machine LearningAn Experimental Comparison of the Nearest-Neighbor and Nearest-Hyperrectangle Algorithms
203 Citations1995Dietrich Wettschereck, Thomas G. Dietterich
In domains where the decision boundaries are axis-parallel, the NGE approach can produce excellent generalization with interpretable hypotheses, and in all domains tested, NGE algorithms require much less memory to store generalized exemplars than is required by NN algorithms.
Machine LearningA nearest hyperrectangle learning method
181 Citations1991Steven L. Salzberg
The results in these domains support the claim that NGE theory can be used to create compact representations with excellent predictive accuracy.
Machine LearningAn experimental comparison of the nearest-neighbor and nearest-hyperrectangle algorithms
137 Citations1995Dietrich Wettschereck, Thomas G. Dietterich
In domains where the decision boundaries are axis-parallel, the NGE approach can produce excellent generalization with interpretable hypotheses, and in all domains tested, NGE algorithms require much less memory to store generalized exemplars than is required by NN algorithms.
A study of distance-based machine learning algorithms
124 Citations1994Thomas G. Dietterich, Dietrich Wettschereck
It is shown that the k-nearest neighbor algorithm (kNN) outperforms the first nearest neighbor algorithm only under certain conditions, and methods for choosing the value of k for kNN are investigated, and two methods for learning feature weights for a weighted Euclidean distance metric are proposed.
Context-Sensitive Feature Selection for Lazy Learners
98 Citations1997Pedro Domingos
Lecture notes in computer scienceLearning a local similarity metric for case-based reasoning
73 Citations1995Francesco Ricci⋆, Paolo Avesani
A new class of local similarity metrics, called AASM, is presented that are not symmetric and that can be adopted as the basic retrieval method in a CBR system and can achieve significant data compression while maintainig the same accuracy as NN.
Artificial Intelligence ReviewControl-Sensitive Feature Selection for Lazy Learners
63 Citations1997Fedro Domingos
Experiments show that RC almost always improves accuracy with respect to FSS and BSS, and a study using artificial domains confirms the hypothesis that this difference in performance is due to RC's context sensitivity, and suggests conditions where this sensitivity will and will not be an advantage.
Studies in classification, data analysis, and knowledge organizationClassification and Learning of Similarity Measures
62 Citations1993Michael M. Richter
IEEE Transactions on Pattern Analysis and Machine IntelligenceData compression and local metrics for nearest neighbor classification
41 Citations1999Francesco Ricci⋆, Paolo Avesani
A local distance measure for the nearest neighbor classification rule is shown to achieve high compression rates and high accuracy on real data sets, and how compression can be traded for accuracy by exploiting voting techniques is shown.
On the Notion of Similarity in Case-Based Reasoning
27 Citations1995Michael M. Richter
The semantics of similarity measures is studied and reduced to the evidence theory of Dempster and Shafer and applications are given for classification and configuration.
Lecture notes in computer scienceOn the importance of similitude: An entropy-based assessment
14 Citations1996Enric Plaza, Ramón López de Mántaras +1 more
This paper argues that the issue to be addressed is estimating the relevance of similitude terms with regard to the task at hand, and proposes a way of using the Case Base to estimate the relevance to be called the discriminating base.
Similarity relations based on distances as fuzzy concepts.
2 Citations2001Manuel Laguía, Juan Luis Castro
This paper analyzes some of the natural kinds of similarity relations and finds non-usual distances in order to obtain these ones.
Algorithms for Classification based on k-NN
1 Citations2007Manuel Laguía, Juan Luis Castro
This paper focuses on methods that solve classification tasks based on distances, and introduces some variants of the basic k-NN method adding up to 3 characteristics, and makes an exhaustive comparison using six different distance functions and 68 data sets, including UCI--Repository and artificial data sets.