Classification of languages originating from Bolivia with Machine Learning

La Paz. 2021

We use Machine Learning techniques for language classification.

Bolivia fights to ensure that indigenous languages do not disappear, however, it is still very difficult to access resources that help their assimilation and learning. That is why we propose to create a tool with Machine Learning for language classification, which, although it is a simple task, is essential to perform more complex tasks such as automatic translation, sentiment analysis, speech-to-text conversion, text-to-speech, etc. This classification model was created using NLP (natural language processing) and ML (machine learning) tools, obtaining an accuracy of over 99%.

Since 2006, Bolivia has been a leader in the defense and vindication of indigenous peoples and cultures in its territory and in the world.

"Today, the 36 native languages in Bolivia are official languages. In Bolivia, an indigenous language must initially be spoken and taught" [1].

But does all this work really have positive results in the assimilation and learning of the native languages in Bolivia?

Problem description

Although in our country we fight to ensure that these indigenous languages do not disappear, it is still very difficult to access resources that help their assimilation and learning.
However, technology can currently be used as an ally to solve this problem and language detection is an initial and essential point to create tools for automatic text translation, text-to-speech conversion, voice-to-text, voice-to-speech, among many other applications.

Objective

Create a tool with Machine Learning capable of identifying and classifying languages originating from Bolivia, to speed up related tasks such as translation, information retrieval, etc.

Limits and scope

LIMITS: Due to the difficulty of obtaining a sufficiently large data set of the most spoken languages in Bolivia (Quechua, Guaraní, Aymara), we only focus on the Quechua language.

SCOPES: The identification tool, in a first stage, will be able to classify the language of phrases as either Quechua or Spanish.

Methodology

For language classification using Machine Learning, an incremental iterative methodology was used, which involves the following phases:

Figure 1. Methodology (Dataiku)

Data capture

After creating the dataset, we realized that it was unbalanced because the number of phrases in Quechua doubled that of Spanish, for that reason we decided to balance the data by adding Spanish phrases obtained from a Kaggle dataset.

Figure 2. Data capture (Image extracted from a website)

Pre-processing

At this stage, various forms of pre-processing were carried out, from feature engineering to vectorization. Which are described below.

Cleaning irrelevant characters

The Spanish phrases from Kaggle had characters from other languages and irrelevant for the classification, which is why before joining with the dataset that was available, all those characters from the Kaggle phrases were cleaned. Once the dataset was joined, there were still characters that did not provide information such as: digits, punctuation marks, etc. and therefore a cleaning of these characters was carried out.

Techniques implemented

Based on the information in the dataset, it was noted that there was incorrectly tabulated data and data imputation was carried out using null values. On the other hand, since the dataset has only one feature and the target, there was no need to reduce the dimensions.

Analysis of the features

Each language has its own grammatical rules and the Quechua language is no stranger to that, therefore the rules of this language were investigated and certain interesting characteristics were noticed that differentiate it from other languages, to mention a few:

1. The Quechua alphabet has 28 consonants (some consonants are different from Spanish, such as: ch', chh, qh, p') and 3 vowels (a, i, u)
2. Quechua consonants are classified according to the mode of articulation, some of these are:

• Plosives (p, t, k, q)
• Aspirated (ph, th, chh, kh, qh)
• Glottalized (p', t', ch', k', q')
• Semiconsonants (w, y)

3. To differentiate the gender of a person, the words are used: warmi and qhari

4. The interrogation in Quechua is made by adding the suffix -chu to the word.

These and many more characteristics of Quechua grammar, as well as Spanish grammar, were taken into account to create the graphs, which allow us to corroborate these differences between the grammatical rules in the dataset.

One of the graphs we made was the frequency of vowels by language in the dataset (Figure 3). The frequencies of the vowels were calculated according to the number of characters in each sentence.

Figure 3. Vowel frequency (Own elaboration)

Other graphs we made were the frequency of consonants such as: K, H, M, R (Figure 4) and special characters such as: á, é, í, ó, ú, ä, ü, ' (Figure 4), depending on the language.

Figure 4. Character histogram (Own creation)

These graphs allowed us to clarify some doubts about the grammatical differences between the Spanish and Quechua languages, and represented a key point to carry out the pre-processing of the data.

Due to the differences in some letters and characters used for each language, in addition to certain suffixes or prefixes, numbering and other characteristics, we decided to vectorize the phrases according to the n-gram character model. To capture important characteristics in both languages, we delimit the n-gram model from 1 to 5, this for performance reasons and also because we consider that this number allows us to abstract those grammatical characteristics that we mentioned above.

Figure 5: Pre-processing and vectorization of the features (Own creation)

In addition to vectorizing the phrases according to the frequency of unique characters that each phrase has, the TF-IDF frequency was applied, a statistical measure that evaluates how relevant a term is for a document in a collection. In this case each term is represented by each character and the document is represented by the language phrase in the dataset (collection of phrases).

Model selection and evaluation

With the data ready, the prediction models were built. Classification models were used because we have a binary classification problem, there are only two possible labels, “Quechua” and “Spanish”.

Therefore, the following supervised learning algorithms were used:

• Naïve Bayes
• Support vector Machine
• Logistic regression

To find the best combination of hyperparameters, GridSearch from the Sklearn library was used.

The confusion matrices for the 3 models are:

Figure 6. Confusion matrices, balanced accuracy and execution time of the 3 models (Own elaboration)

Classification of new data

Once we have selected our best model, it is necessary to test it with some phrases never seen before, so we test phrases that only contain words from one language, phrases that contain words from both languages that is common in the colloquial speech of Quechua speakers and finally a meaningless phrase. Our selected model behaves well with the first two types of phrases, however, when entering meaningless phrases that do not belong to either Spanish or Quechua, these phrases are directly classified as Quechua, this is because as we handle a binary model, the entered text must be tagged with one label or another.

Figure 7. Model results (own elaboration)

Conclusion

Overall, the Machine Learning model for language classification based on Support Vector Machine offers the best predictive result with a balanced accuracy score of over 99%.

Specifically, the model works well for classifying Spanish and Quechua given the high precision, and f1 scores for these two languages.

Although the best precision was obtained with SVM, considering the precision and performance variables, the best model would be the one based on Logistic Regression, since it offers a shorter execution time than SVM and has an accuracy greater than 99%, which is an important factor in real-time applications.

It was possible to abstract some characteristics of the Quechua language, making it possible to carry out the same analysis with other languages originating from Bolivia.

Future jobs

Although this problem appears to be simple, it is a necessary step to:

• Automatic translation
• Language detection for the use of boots
• Sentiment analysis, etc.

The source code for this project can be found at: github

References

UN – Bolivia, at the forefront in the protection and promotion of indigenous languagesSaturdays.AI

• NLP
• Machine Learning
• Supervised Learning
• Tf Idf

WRITTEN BY

EVELYN CUSI LOPEZ

Repository

In the following repository you will find the code used to develop that application: https://github.com/SaturdaysAI/Projects/tree/master/Lapaz/clasificacion-idiomas-machine-learning-main

More Artificial Intelligence!

The mission of Saturdays.ai is to make artificial intelligence more accessible (#ai4all) through intensive courses and programs where projects for good are carried out (#ai4good).

Explore our AI master’s programme at https://saturdays.ai/master-ia-online/

If you want to learn more about artificial intelligence join our community at community.saturdays.ai or visit us on our website www.saturdays.ai Join us!