Facial expressions and emotional wellbeing: a deep learning prototype

La Paz. Deep Learning. 2021

INTRODUCTION

This educational project from 2021 explored facial-expression classification with deep learning. The team proposed recording a model’s labels while a person viewed images. The original idea was linked to iconic therapy, but the work presented does not establish therapeutic effectiveness or the ability to diagnose a disorder.

A facial expression alone does not reliably establish a person’s emotional state. Dataset labels are classification categories, not diagnoses. This distinction is particularly important when considering use with children or in healthcare.

Scientific review of facial expressions and emotions

PROBLEM DESCRIPTION

The technical challenge was to process camera images, classify expressions and record the results over time. The team explored automated logging; that did not validate a psychological interpretation of the labels.

OBJECTIVE

Build a prototype that records the facial-expression categories estimated by the model. Any potential clinical application would require a dedicated study, consent and professional oversight; the prototype described does not establish those requirements.

DATASET

The team used a dataset published by Jonathan Oheix on Kaggle, containing 35,900 facial-expression images. These show faces in greyscale at 48 × 48 pixels. The dataset has training and validation folders and seven labels: anger, disgust, fear, happiness, neutrality, sadness and surprise.

SELECTION OF THE MODEL(S)

We selected ResNet50 after comparing several architectures on results, training time and optimisation options. It best suited the prototype’s resources and goals.

ResNet50 is a 50-layer convolutional neural network. It can use pretrained weights; for this project, we worked with the image dataset described above.

Other tools used in the project:

Keras: we used this Python library for its accessible interface for experimenting with neural networks, with TensorFlow as the underlying framework.

We also used OpenCV and NumPy.

MODEL EVALUATION

We tested the following models:

-AlexNet: This architecture has relatively few layers and produced results the team considered sufficient for an initial test.

-Face Recognizer: We trained a model using this method, but running it in real time required too many resources for the lower-powered devices we had in mind.

-EfficientNet: We attempted to train this network for three epochs but could not complete the process within the time available.

-ResNet2: This variant ran into training difficulties: after three hours, it had not completed its first epoch.

-ResNet50: We selected this network for the prototype, completing 100 epochs in one hour and 45 minutes using a Google Colab GPU.

ANALYSIS OF RESULTS

The chart shows how accuracy evolved over 100 epochs of ResNet50 training.

This second chart shows the model’s loss.

CONCLUSION AND RECOMMENDATIONS

We recommend checking images and labels before training. A local GPU or one available through Google Colab can help. Adding relevant, representative images may improve performance, but more data alone does not guarantee better results: improvements must be checked on independent evaluation data.

The comparison helped the team select a model for the classification prototype. The next step would be to evaluate errors, data representativeness and interpretation limits before considering use beyond the educational exercise.