Smart Class Analytics for Assessing Group Understandingvia Facial Expressions

Abstract

Understanding the comprehension level of students in a communicating classroom is critical. As technology becomes increasingly integral to our lives, advanced facial emotion recognition becomes a powerful tool for assessing comprehension levels in a real-time scenario. We created a system to measure cognizance levels and analyze facial emotions to ascertain comprehension. In this research, we presented a new system for assessing the comprehension level of a group setting through facial expressions in a real-time. Our techniques, supported by advanced DL techniques like OpenCV and Deep Face provide us with different backend options that are very effective for facial emotion recognition and analysis such as MTCNN and Retina face.

Initially, we used OpenCV for real-time capturing video by computer webcam and capture the frames at regular intervals. The frames are stored in a pre-defined directory. After that, we defined a function that iterates over each frame for detecting emotions with facial landmarks and also shows the dominant emotion with each face in the frame. A pretrained MTCNN model (one of the state-of-the-art models provided by Deep Face) has been utilized for detecting the face present in each frame and facial landmark localization. MTCNN is a lightweight model that detects faces with high accuracy and speed.

Finally, The Deep Face analyze () function has been utilized for emotion analysis in the wild with the MTCNN backend which provides insights into the dominant emotion related to each face. Comprehension levels were measured as per emotional states detected, with positive affect [‘happy’, ‘neutral’ (when confidence is >70), and ‘surprised’ (when confidence is <= 60%)] implying higher comprehension, negative affect [‘sad’, ‘disgust’, ‘fear’, and ‘surprised when the confidence level is above 60%’] hinting at lower comprehension, and when [‘happy’, ‘neutral’] emotions having confidence <=70% hinting to moderate affect. The measured comprehension level is then disseminated into high, moderate, and low.

The finding exposed the complex interrelation among facial expressions, emotions, and comprehension levels in intragroup interactions. This research supports the development of understanding human behavior and interaction dynamics, with connection to domains such as psychology, human-computer interaction, and education.