Student Engagement Detection in Classrooms through Computer Vision and Deep Learning: A Novel Approach Using YOLOv4

Abstract

Identifying early signs of disengagement in students is critical for reducing dropout rates in educational institutions. This study introduces a novel model that utilizes computer vision and deep learning to monitor student engagement levels within the classroom environment. The proposed model employs a camera system to capture images within the classroom at random intervals, which are then associated with student IDs and analyzed to assess engagement. This analysis categorizes the images into 'engaged' and 'not engaged', from which a weekly average engagement score is derived and proposed to be communicated to each student. The differentiation between student images and general classroom activity is achieved using the YOLOv4 algorithm, trained on a custom dataset. YOLOv4, known for its balance between accuracy and processing speed, was selected as the primary tool for one-stage object detection. The dataset comprised 893 training images, 191 validation images, and 192 testing images. The study also explored a comparative analysis with a model trained on YOLOv3, examining performance metrics before and after augmentation with a Generative Adversarial Network (GAN)-enhanced dataset. Performance evaluation revealed improvements across all metrics after integrating GAN images, with YOLOv4 showing superior performance over YOLOv3. Notably, Mean Average Precision (mAP50) and Average Intersection over Union (IoU) saw significant gains, along with increases in Precision, Recall, and F1-score for both engaged and not engaged classifications. Privacy and security considerations are thoroughly addressed in the concluding section of the paper.  The proposed model of this research shows the promise of these technologies in fostering a supportive learning environment by identifying and mitigating student disengagement.