MOODY.AI

Editorial note: this project classifies facial expressions using its dataset’s labels. That task does not, by itself, establish a person’s emotions, attention or wellbeing. The results described should be read as those of an educational prototype.

Evidence on facial expressions and emotions

Guadalajara. Third Edition. 2021

Our Saturday.AI experience

Our team is made up of 5 people from various professional areas and levels of experience, as well as different places of origin, which further enriched our project due to the different points of view. Our project was named Moody.AI

team logo

Members:

   Raúl Sandoval

   Karla Ramírez

   Aldo Álvarez

   Israel Castillo

   Jaime Chacoff

When the team was created, each member gave their project proposal, because we were all very interested in image processing, the project that involved this topic was selected. Which was about facial emotion recognition, based on the scoop that even we were victims of virtual classes, we decided that a good approach would be education; Since children are a more vulnerable sector, it was decided to focus on them.

Initially, the project established creating a tool that would allow detecting, at an early stage, children's negative emotions during virtual classes since it would be of great help to school authorities and the children's parents.

However, reviewing the topic in more depth we came across certain barriers, such as the fact that we did not have a dataset specialized in children, which would require explicit consent from parents to be able to capture images of their faces, among other things.

By continuing to investigate the effects of the pandemic and virtual communications, we discovered that the impact is also significant on adults working remotely (home office).

Considering that we could also have a significant social impact by focusing the project on adults, it was decided to change the focus of the project and orient it to the detection of emotions in adults who are experiencing the modality of remote work (home office).

The fact that the datasets available on the Internet are mostly from adults also had an influence. Likewise, an important aspect was the possibility of testing the faces of the team members (all of legal age) and whose participation was voluntary.

In conclusion, we worked on a model for the detection of emotions through facial expression, for adults working remotely (home office). There was a focus on negative emotions, so that this tool helps the human resources areas of companies to detect the state of mind in the work environment and with this information they can take the necessary actions.

Introduction

NOM-035-STPS-2018: has the objective of establishing the elements to identify, analyze and prevent psychosocial risk factors, as well as to promote a favorable organizational environment in the workplace.

Today, what emotional health needs is more attention, visibility and less stigma. To this end, the use of Artificial Intelligence in projects with positive social impact was proposed, considering living in a time of pandemic as a factor.

Initially, the project established having a tool that would allow detecting, at an early stage, children's negative emotions during virtual classes since it would be of great help to school authorities and the children's parents.

Considering that we can have a broader social impact by including adults in this project, and taking into account that to use the faces of minors it is necessary to have consent from parents or guardians, it was decided to change the focus of the project and direct it to the detection of emotions in adults who are experiencing the modality of remote work (home office).

Problematic

Due to the global COVID-19 pandemic, the population has had to remain in quarantine for more than a year. A significant percentage of companies have implemented remote work so as not to slow down operations during the pandemic. Now work is carried out in a “home office” mode, which has generated problems that have affected remote working collaborators. Among these problems, the lack of coexistence with work colleagues, the lack of distance between personal life and work life, and anxiety due to uncertainty and confinement stand out. All of these problems end up reducing the emotional stability of an individual, which has great repercussions on their performance and general well-being.

There are psychological and social studies that highlight the enormous negative impact that the pandemic can have on an individual's mental stability. In this way, it is considered to be of great help for companies to be alerted to visualize in real time the negative emotions detected in their collaborators, which can be: sadness, anger, fear or disgust. With this, a preliminary detection of negative emotional behaviors would be made, which will serve as a first guide to follow up the case with a professional consultation.

Dataset

An existing, freely available online dataset was used, which was created by Pierre-Luc Carrier and Aaron Courville as part of a research project [3]. Facial expressions from 35,685 images are classified in this file. Each image is 48x48 grayscale pixels in size and is cropped so that only a single person's face is included. Each face in the data set is classified based on facial expressions. The classes into which expressions are divided are: happiness, neutral, sadness, anger, surprise, disgust and fear.

Model

All layers have been organized sequentially using TensorFlow 2.5 and Keras. The network resembles a VGG-16 type convolutional network with several CONV convolutional layers, maintaining the padding and always with a 3×3 kernel except in the first 2 convolutional layers that have a 5×5 kernel.

Additionally, MaxPooling layers between the CONVs for spatial reduction but maintaining relevant image information. To prevent overfitting, BatchNorm and Dropout layers have been added as regularizers.

moodyNet: VGG-16 type convolutional network

Each convolutional layer has a ReLU (Rectified Linear Unit) activation so that negative values are not transferred to the next layer.

After the convolutional layers, come the dense layers, for this, the resulting matrix of the convolutional layers is flattened to continue towards 4 dense layers with a ReLU activation and a last dense layer, with the number of parameters equal to the number of classes, with a softmax activation. The last softmax layer will deliver values ​​between 0 and 1, which correspond to the probability of each class.

The optimizer that has been used is ADAM.

Results

Loss function and precision across epochs

Conclusions

With the final product, an algorithm is obtained capable of analyzing the type of emotions that predominate in each user through the visual material provided, placing special emphasis on alerting the prevalence of negative emotions. With the specialized analysis of the emotional profile of each user, company members in charge of monitoring the well-being of workers will have the opportunity to follow up on alerts from users who could be suffering from an emotional problem during their work days. Providing this type of personalized attention to company members can be crucial to improving each individual's performance and job satisfaction, as well as their quality of life.

We believe that we can help Human Resources departments a lot to ensure that employees have the best possible disposition, as well as the greatest emotional health that we can give them. It is a tool that allows you to improve the attention of people who work instead of just focusing on their performance labor. It is a way of accepting that people's emotional well-being produces a change in how they work and how the space in which they operate professionally is developed.

Future projection

We are sure that this first version of Moody.ai can be perfected and expandable to more sophisticated versions with different approaches. Among the ideas that have emerged in the team for subsequent versions of Moody.ai are the following applications:

• Automate emotion detection and trigger events based on these detections. For example, identifying, in real time, faces of tired transport drivers, blinking more frequently than normal, yawning, etc.
• Detection of employees' emotions on specific topics discussed in remote work environments and for companies to identify them early.
• Identify, in real time, early people with negative emotions carrying out potentially risky activities: machinery operators, production line personnel, construction workers, etc.

We believe that continuing to develop Moody.ai could achieve unlimited social impact. This tool is easy to adapt to the context where it needs to be implemented and as the network grows, an increasingly robust model will be obtained that will be able to make more accurate predictions.

Project presentation: DemoDay

Repository

In the following repository you will find the code used to develop this application: https://github.com/SaturdaysAI/Projects/tree/master/Guadalajara/March2021/EmotionsDetector-main

More artificial intelligence!

The mission of Saturdays.ai is to make artificial intelligence more accessible (#ai4all) while carrying out projects for good (#ai4good). The workshops we carry out are part of the program AI 4 Schools so that anyone “learn by doing” AI regardless of their background or starting point.

Explore our AI master’s programme at https://saturdays.ai/master-ia-online/

If you want to learn more about artificial intelligence join our community through this link or visit us on our website www.saturdays.ai Join us!

WRITTEN BY

I. C.H. Smith.

WRITTEN BY

SophieB

WRITTEN BY

SophieB