Biometric Feature Extraction and Encoding for Face-Voice Association Learning using Spiking Neural Network

Abstract

In most biometric neural network-based recognitions, learning have been implemented using supervised approaches. These approaches require a set of learning targets with particular encoding rules to train a network to recognise the biometric features. However, in some applications, the exact teacher signals are not easy to be formulated. For this study, we propose an encoding of biometric features for association learning of face-voice using a recurrent spatio-temporal neural network. The association learning rules are implemented using reward-modulated spike-time dependent plasticity without any particular spike templates as the learning targets. Prior to learning, the face features are extracted using principal component analysis (PCA). We have also experimented on the face feature extraction using singular value decomposition (SVD). The network learns better with the features obtained from PCA. Meanwhile the voice features are extracted using wavelet packet decomposition (WPD). The face and voice features are then encoded into spikes. We have run a series of experiment to find the optimal set of learning parameters. The best performance achieved was 77.26% and 82.66% for training and testing, respectively.