Development of affective state recognition model based on thermal imaging

Abstract

In social interaction, the explicit and implicit communication plays a significant role in an effective interaction. However, the typical modalities of interaction such as verbal and body language, sometimes, may be deterred causing meaningful communication could not be achieved especially when dealing with an emotionally-challenged subject. Hence, the apprehension of their emotional states is highly intrinsic. Progress has been made in affective computing using the Autonomic Nervous System (ANS) parameters for affect detection. Nevertheless, while a significant number of findings have been reported, most of the experimentations employed the invasive approaches where direct contact between subject and sensor was required. Even though the existence of research that utilised the non-invasive approach for affect detection is irrefutable, yet, the universality of such approach remains a much-debated question as it is believed to be varied based on gender, culture and age. All the previously mentioned methods suffer from a number of serious drawbacks when dealing with the subjects who are unable to express their emotions explicitly. Henceforth, the thermal imaging based affect recognition was devoted in this thesis to classify six prototypical emotions. The ability of thermal imaging to quantify the ANS parameters through contactless, non-invasive and non-intrusive manner is believed could circumvent the limitation of other approaches. In the proposed framework, the first stage involves the image acquisition and enhancement using Contrast Limited Adaptive Histogram Equalization (CLAHE). In the second phase, the second order statistical features were extracted using Gray Level Co-occurrence Matrix (GLCM) from four regions of interest (ROI); periorbital, supraorbital, mouth and nose. Lastly, the third phase classifies the respective emotions using the k-nearest neighbour (k-NN) algorithm with 10-folds cross-validation routine. The proposed model was found to outperform the existing models with 86.7% accuracy (mean accuracy of existing models = 73.63%).