CBSA: content-based soft annotation for multimodal image retrieval using bayes point machines

This work proposes a content-based soft annotation procedure for providing images with semantical labels, and experiments with two learning methods, support vector machines (SVMs) and Bayes point machines (BPMs), to select a base binary-classifier for CBSA.

Abstract

We propose a content-based soft annotation (CBSA) procedure for providing images with semantical labels. The annotation procedure starts with labeling a small set of training images, each with one single semantical label (e.g., forest, animal, or sky). An ensemble of binary classifiers is then trained for predicting label membership for images. The trained ensemble is applied to each individual image to give the image multiple soft labels, and each label is associated with a label membership factor. To select a base binary-classifier for CBSA, we experiment with two learning methods, support vector machines (SVMs) and Bayes point machines (BPMs), and compare their class-prediction accuracy. Our empirical study on a 116-category 25K-image set shows that the BPM-based ensemble provides better annotation quality than the SVM-based ensemble for supporting multimodal image retrievals.