Machine Learning to prevent online harassment of children

Helping children navigate the internet more safely

When your children surf the internet, do you apply or configure any parental control filters?

Latam online. Second Edition. 2021

Currently, online sexual harassment or harassment is known in English as online grooming; where an adult manages to have contact through a technological means against a minor, the objective of the abuser being to attack through interactions such as: harassment of their morals, talking about sex, obtaining intimate material or agreeing to a sexual encounter.

Online grooming has increased as a result of the COVID-19 confinement. In Argentina it increased more than 30% in 2020(2), INEGI data from 2019 shows that in Mexico it grew 27%(3), while the Spanish newspaper ABC reported an increase of 410% in recent years (4).

Context… please!

Parental-control tools can block apps or sites, restrict purchases and manage device use. Our project explores a different task: analysing conversation text to flag possible risks for a person to review.

Derived from the confinement due to COVID-19, children and adolescents have been more vulnerable due to the need to use mobile devices as part of their daily activities. Much of this was to take virtual classes, complete homework and on other occasions as a means of entertainment and communication, as they were limited by not being able to go out and share time with family and friends.

Lack of supervision has allowed minors to experience unwanted situations without the knowledge of adults.

The article cited from Salud con Lupa reports harassment on Facebook (39%), Instagram (23%) and WhatsApp (14%) among the cases it discusses (5).

The proposal

Totem means protector

Tótem: “Let them browse with peace of mind”

Without much fuss, this is what we did.

We currently have a dataset of conversations in English, which are analyzed and classified as normal or dangerous. Once this result is obtained, a notification will be sent to the parent or guardian if an inappropriate conversation is detected.

How did they do that?!

Here we explain what we applied to obtain the results that we will show you later...

Phase 1 (Here we go…)

Obtaining the data

We obtained the data from the PAN Lab 2012 project, cited in reference 6, after requesting and receiving permission to use it. The corpus contained conversations of varying lengths and writing styles in XML format. We converted it to CSV (comma-separated values) for further processing.

Phase 2 (Let's analyze this...)

Data Cleaning

The second task presented a high degree of difficulty, due to the noise of the data. Therefore the following steps were followed.

Any type of punctuation was eliminated.
Numbers were converted to words using the library num2words.
We searched for abbreviations and texts with cyber slang such as: “u” for “you”, “ur” for “your”, “what ‘s” for “what is”, among others. In addition, a dictionary was created with these new words.
We removed common stop words such as the, and, that, a, any, an, be and with.
Removed emojis, URLs, hashtags and any type of alphanumeric value.
The lemmatization technique was used to bring all the verbs to their infinitive form. In order to create embeddings from simpler words.
The data was explored, analyzing the 100 most common words in the predators' conversations.

Phase 3 (Let's train this...)

Vectorization

The extraction of characteristics represented a particular challenge, in the first instance it was decided to do it based on the most frequent words present in the conversations of predators; However, several of these were in common use, so it was necessary to analyze alternatives to achieve optimal performance. Thus, it was decided to use the TF-IDF method (Term Frequency — Inverse Document Frequency), one of its characteristics being to highlight the importance of a word in a set of documents (corpus). In this sense, to transform the sequence of words (from conversations) into feature vectors with numerical representations, the TF-IDF vectorizer (TfidfVectorizer) from the scikit-learn library was used. This transformer made it possible to generate a matrix of characteristics, with an adequate representation to train the model. For this purpose, the dataset was divided into:

Training data: 80%
Testing data: 20%

Training

To train the model, the scikit-learn library was used and the Support Vector Machines (SVM) model was chosen to use a binary classifier.

The hyper parameters were:

Regularisation parameter (C): 10
Kernel: RBF
Kernel coefficient (gamma): Scale

Evaluation

During the evaluation of the model the following results were obtained:

The confusion matrix being the following:

The model results show a notable improvement compared to those corresponding to the Baseline:

We deployed the model in a Streamlit web application. A user enters a conversation and the app displays the predicted category. The examples below illustrate how it works; an alert requires review and does not by itself establish abuse.

Analysis result in a normal conversation

Result of the analysis of a dangerous conversation

Does our proposal make sense to you?

The prototype classifies conversations from the dataset studied and explores alerts for review. Conversation-monitoring tools also affect privacy: their development must address consent, data access, classification errors and children’s wellbeing. They do not replace support from trusted adults.

In the future, it would be great to develop these alerts available to parents as part of the restrictions of some applications, which are concerned with applying parental control rules for vulnerable users who use their services, such as: https://www.facebook.com, https://www.instagram.com, https://www.tiktok.com, https://www.whatsapp.com, etc.

References

Here you can see a video that describes our proposal

Open video or resource

Members

Aarón Arguello
Edson Segales
Ingrid Velecela
Karla Avilés
Viridiana Vergara

Project presentation: DemoDay

Open video or resource

Repository

And if you want to see the data source and the code we applied, you can find it on GitHub:

https://github.com/SaturdaysAI/Projects/tree/master/LATAM_remote/TotemIA

More artificial intelligence!

Saturdays.AI makes AI more accessible through shared learning and projects with a social purpose. Our motto is AI4ALL IS AI4GOOD.

Meet the Saturdays.AI community