Facial expressions and emotional wellbeing: a deep learning prototype

La Paz. Deep Learning. 2021

INTRODUCTION

This educational project from 2021 explored facial-expression classification with deep learning. The team proposed recording a model’s labels while a person viewed images. The original idea was linked to iconic therapy, but the work presented does not establish therapeutic effectiveness or the ability to diagnose a disorder.

A facial expression alone does not reliably establish a person’s emotional state. Dataset labels are classification categories, not diagnoses. This distinction is particularly important when considering use with children or in healthcare.

Scientific review of facial expressions and emotions

PROBLEM DESCRIPTION

The technical challenge was to process camera images, classify expressions and record the results over time. The team explored automated logging; that did not validate a psychological interpretation of the labels.

OBJECTIVE

Build a prototype that records the facial-expression categories estimated by the model. Any potential clinical application would require a dedicated study, consent and professional oversight; the prototype described does not establish those requirements.

DATASET

The team used a dataset published by Jonathan Oheix on Kaggle, containing 35,900 facial-expression images. These show faces in greyscale at 48 × 48 pixels. The dataset has training and validation folders and seven labels: anger, disgust, fear, happiness, neutrality, sadness and surprise.

SELECTION OF THE MODEL(S)

Because there are various models in the field of Deep Learning, the ResNet50 model was chosen, this is because after a model analysis in which the time it took to train them was taken into account, its optimization, among other aspects, was the one that had the best results among all the variants that were taken into account.

ResNet50 was used because it is a convolutional neural network that is 50 layers deep. This can load a pre-trained version of the network, in the case of this project the database mentioned above was used.

Other tools used in the project:

Keras: This open source library written in Python was used, since it is mainly based on facilitating a rapid experimentation process, and since it is an intuitive interface, it allowed us to access machine learning frameworks, in this case TensorFlow was used.

Additionally, OpenCv and Numpy were used to complete all aspects of the project.

MODEL EVALUATION

Tests were carried out with the following models:

-AlexNet: This network is one of the most popular, but it has few layers and sufficient precision was still obtained.

-Face Recognizer: A model was entered with this method, but when it came time to create the model to evaluate it in real time, it consumed too many computer resources, so it was not appropriate if one wanted to use it on simpler devices.

-EfficientNet: An attempt was made to train this network with three epochs but it was not achieved due to the long training period it required.

-ResNet2: This network presented difficulties at the beginning of its training, being the case that it did not pass stage number one, after 3 hours had passed.

-ResNet50: This network was the one we used in the project since we were able to train 100 epochs using the collaborative GPU and the time used was 1 hour 45 minutes.

ANALYSIS OF RESULTS

The graph shown corresponds to the accuracy that was achieved after having trained it with 100 stages, due to this and the ResNet, these results can be achieved

Unlike the previous graph, this one corresponds to the loss of the model

CONCLUSION AND RECOMMENDATIONS

It is recommended to verify the dataset correctly before entering the network, since many of them come with images that do not correspond. Likewise, to the extent possible, it is recommended to train with a physical GPU, if this is not the case, it is recommended to use the collaborative GPU. If this is the case, use a dataset with more images, since this will increase the precision of the training.

The comparison helped the team select a model for the classification prototype. The next step would be to evaluate errors, data representativeness and interpretation limits before considering use beyond the educational exercise.

Project presentation: DemoDay

More artificial intelligence!

The mission of Saturdays.ai is to make artificial intelligence more accessible (#ai4all) while carrying out projects for good (#ai4good). The workshops we carry out are part of the program AI 4 Schools so that anyone “learn by doing” AI regardless of their background or starting point.

Explore our AI master’s programme at https://saturdays.ai/master-ia-online/

If you want to learn more about artificial intelligence join our community through this link or visit us on our website www.saturdays.ai Join us!

WRITTEN BY

Alejandra Rodrigo