COMPOUND FACIAL EXPRESSION RECOGNITION BASED ON MOBILENET WITH DATA AUGMENTATION

Abstract

Abstract—Emotion identification through facial expressions holds significant importance as a natural means to convey emotional states and intentions. Automatic facial expression recognition (FER) plays a crucial role in various human endeavours, including medical treatments, evaluations, human-computer interactions, and human-robot communication. While this area has garnered substantial attention in computer vision, there remains a notable research gap, particularly in addressing the limitations of existing work primarily focused on basic facial expressions. One key challenge was the coexistence of compound and basic facial expression images within the same dataset folder, resulting in an imbalance among different expression classes. This dataset imbalance can introduce biases towards more prevalent classes, affecting overall accuracy. Additionally, a considerable volume of data is required to enhance the effectiveness of deep learning models. The primary objective of our research is to develop a robust system for recognising compound facial expressions, with supplementary investigations into basic facial expressions. Leveraging Convolutional Neural Networks (CNNs) and data augmentation techniques, we have organised the dataset into distinct folders for each expression class. This separation allows us to assess the quantity of images for each class and implement data augmentation methods to achieve a balanced dataset. Furthermore, we address the need for an ample dataset by employing a data regeneration approach based on Generative Adversarial Networks (GANs). We subsequently employ the MobileNet deep learning model to evaluate classification accuracy for basic and compound facial expressions. In-depth analysis of results includes precision, recall, and F1-score metrics. Our findings reveal a classification accuracy of 81% for basic facial expressions, a substantial improvement over previous methodologies. Moreover, our approach achieves a classification accuracy of 56% for compound facial expressions. These results signify significant progress in the field, particularly in addressing the challenges of recognising compound facial expressions.