Supervised representation learning: transfer learning with deep autoencoders

This paper proposes a supervised representation learning method based on deep autoencoders for transfer learning that consists of an embedding layer and a label encoding layer that minimize the difference between domains explicitly and encode label information in learning the representation.

Abstract

Transfer learning has attracted a lot of attention in the past decade. One crucial research issue in transfer learning is how to find a good representa-tion for instances of different domains such that the divergence between domains can be reduced with the new representation. Recently, deep learning has been proposed to learn more robust or higher-level features for transfer learning. However, to the best of our knowledge, most of the previous ap-proaches neither minimize the difference between domains explicitly nor encode label information in learning the representation. In this paper, we pro-pose a supervised representation learning method based on deep autoencoders for transfer learning. The proposed deep autoencoder consists of two encoding layers: an embedding layer and a label encoding layer. In the embedding layer, the dis-tance in distributions of the embedded instances be-tween the source and target domains is minimized in terms of KL-Divergence. In the label encoding layer, label information of the source domain is en-coded using a softmax regression model. Extensive experiments conducted on three real-world image datasets demonstrate the effectiveness of our pro-posed method compared with several state-of-the-art baseline methods. 1