CiclopeIA: Artificial intelligence to help people with visual disabilities

Latam online. Second Edition. 2021

Many people are motivated by challenges at work, in studies and in life in general, but I ask, can you imagine a life where every day there are different challenges, starting in the morning when when you get up and take a refreshing bath in the morning, you must do what is necessary to not trip on the way to find your clothes and combine them appropriately for the occasion, because today you must go to the center of the city to complete some bureaucratic procedures that you require, in addition to doing the supermarket shopping for the week. After making a coffee and ensuring that it does not spill over the edge of the cup, you are ready to start your day, you leave your house and hope that you do not have to trip over obstacles that are common on the street: garbage that did not reach its respective bin, the improvements of the city characterized by its constant planning changes at the cost of leaving unfinished work in the streets, the not so friendly dogs on the street that could test your instinct and reason, but hey, it is something that one already knows and it What you should do is get to the bus station and wait for some kind-hearted passerby to help you identify the bus line that brings you closer to your destination. There will be time to go to the supermarket later, get what you need and trust that you will receive the right amount in change or have the security of paying what is appropriate.

These situations describe a life full of challenges that blind people must face on a daily basis, and that often prevent them from socially integrating into work, commercial, and leisure environments and even increase their dependence on third parties, which also has a psychological impact that eventually and little by little undermines their self-esteem.

Depending on the age at which blind people lose their sense of sight, it can become a more or less traumatic experience that can be bearable if they have emotional support, especially in the initial stage, and work hard to educate themselves and adapt to their new condition and quality of life, which will require going through a tortuous path of mourning for the loss of sight.

Despite these not very encouraging situations, there are many people who do not let themselves be defeated by adversity and successfully adapt both emotionally and socially to lead a dignified life and with reasons to continue challenging adversity day by day.

According to the WHO [1] “Globally, it is estimated that approximately 1.3 billion people live with some form of vision impairment.

Globally, the main causes of poor vision are uncorrected refractive errors and cataracts.

“Most people with low vision are over 50 years old.”

In addition, vision impairment or loss has impacts on the quality of life of people who suffer from it, according to WHO, “Young children with severe vision impairment at an early age may suffer delays in motor, linguistic, emotional, social and cognitive development.”

“For older adults, vision impairment can contribute to social isolation, difficulty walking, increased risk of falls and fractures, and increased likelihood of early admission to nursing homes.”

The Lancet Global Health [ 2] states that:

“In 2020, an estimated 596 million people worldwide have distance vision deficiency, and of these 43 million are blind.”

“A large proportion of these (90%) live in low- or middle-income countries.”

“By 2050, population aging, growth and urbanization will lead to an estimated 895 million people with distance vision impairment and 61 million blind.”

Given this problem, and as part of the practical application of a Machine Learning course [ 3] and Artificial Intelligence (AI) [4], The idea of using technology to identify ideas that can address the problems of people with visual disabilities and design a tool that supports them in achieving their daily activities was born. The group's idea was to think of a project that could contribute socially and have an impact on the community. From the point of view of the Sustainable Development Goals (SDGs) [5] would be related_:_ Health and well-being, Industry, innovation and infrastructure, reduction of inequalities and alliances to achieve the objectives.

The project Cyclops.AI, As we have called it, it seeks to integrate into a cell phone application, different options (skills) aimed at solving limitations that people with visual disabilities experience in their daily activities such as: Recognition of bills and coins quickly and effectively, identification of the bus line, identification of colors, knowing the fill level of a container, finding lost objects, etc.

As you can see, the options that are intended to be integrated are ambitious and will require extensive work, however, it is necessary to start somewhere, it is often said that a “cake is eaten in pieces” and that is why the application initially has the functionality that allows the user to recognize the amount of cash in dollars (bills), using the camera of their cell phone. After detection, the app plays a voice message with the result of the recognized amount. The use of Ciclope.AI provides the user with security and autonomy when making cash transactions and reduces the risk of being a victim of deception. Interaction with the application can be done in Spanish under the Android operating system.

Initial Project Scope

When we started the project and after an approach with a group of blind people, we identified some options (skills) that we should include in the application, so we took the most frequently requested option, which is the identification of banknotes when carrying out monetary transactions with third parties. Additionally, it was known that the best way to interact with blind people is through audio, so we decided that the user's interaction with the application should be done through voice, both input and output.

To support our configuration we reviewed some statistics on the site yiminshum.com, “there are currently 5.19 billion unique users on mobile devices, where it does not divide the type of phone, this covers 67% of the population.”

"73% of people are connected and share their time from a smartphone. 23.5% are associated with a common telephone, where its functions are the basic and limited ones that a telephone must fulfill, which is to call and send messages, and 3.6% are associated with a router, tablet or mobile PC."

“OS, better known as an operating system, is important for the operation of computers in the world, 74% of users are on Android devices, 25% is iOS, 0.4% is KAI, 0.2% is Samsung OS and 0.6% is other operating systems.”

With this information, the scope of the project was limited to Android smartphones, which cover a large part of the market, especially in places other than the United States, to Spanish-speaking countries that have the dollar as their currency.

Applied AI

Faced with the identified problem, it was determined that the best way to support blind people is to develop a cell phone application that acts as their eyes in situations that are required. For this, from a technical point of view, different image recognition models that could be applied were explored, and in the end it was decided to use YoloV5 due to its versatility when identifying objects and based on tests carried out by Towards Data Science [6] that recommend the model over another also well-known one. Other elements that we considered were the possibility of using the model without the need to have an active internet connection but rather a pre-trained model that is copied to the cell phone in a Pytorch Lite version that takes up less size and resources.

To train the model, available dataset options were explored. Unfortunately, one was not available, so we decided to create our own dataset.

One of the big challenges was precisely fine-tuning the dataset so that it includes the appropriate images, considering different scenarios in which the bills could be, considering the reverse and adverse, and sufficient number of images. To obtain the images, a Web scrapping tool was used and later with the online tool. https://labelflow.ai/ Labels were assigned to each image to identify each bill in the different images.

After training the model in Google's colab tool, a file was obtained with the best model generated and used in the cell phone application.

Without a doubt, AI will provide solutions to many of today's problems and will allow its application to spread massively in different areas of knowledge and daily life. Our work is a small sample of the potential to be exploited with AI and a contribution for those interested in supporting groups such as the blind who must overcome adversity with little or limited support from society.

Conclusions

Artificial Intelligence tools can be used for multiple purposes, however, developing products that provide solutions to social needs represents an enormous opportunity that results in well-being and improvement in the quality of life of minority groups in society.

From a technical point of view, the project represented an opportunity to continue learning from this exciting world of AI and understand, among other things, that to create effective models it is important to work exhaustively on:

• Create or have quality datasets.
• Apply different models to evaluate performance.
• Make users participate in product development.
• Work with multidisciplinary teams.
• Maintain a permanent search for new solutions.

Demo video

Open video or resource

References

[1] World Health Organization. (2021). Blindness and visual impairment. Retrieved January 12, 2022, from https://www.who.int/es/news-room/fact-sheets/detail/blindness-and-visual-impairment.

[2] The Lancet Global Health. (2021). The Lancet Global Health Commission On Global Eye Health: Vision Beyond 2020. Retrieved January 12, 2022, from https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(20)30488-5/fulltext

[3] BBVA (2019). What is Machine Learning and how does it work? Retrieved February 4, 2022 from https://www.bbva.com/es/machine-learning-que-es-y-como-funciona/

[4] Juan Antonio Pascual Estapé(2019). Artificial intelligence: what it is, how it works and what it is currently used for. Retrieved February 4, 2022 from https://computerhoy.com/reportajes/tecnologia/inteligencia-artificial-469917

[5] United Nations (2021). The agenda for sustainable development. Retrieved February 4, 2022. https://www.un.org/sustainabledevelopment/es/development-agenda/

[6] Towards Data Science (2020). YOLOv5 compared to Faster RCNN. Who wins?. Retrieved February 4, 2022. https://towardsdatascience.com/yolov5-compared-to-faster-rcnn-who-wins-a771cd6c9fb4

Members

• Alexander Cortes
• Ariosto Olmedo Cabrera
• Antonio Paucar
• Carlos Sesma
• Miriam Quimi
• Santiago Yunes
• Viviana Marquez

Project presentation: DemoDay

Open video or resource

Repository

In the following repository you will find the code used to develop this application: https://github.com/SaturdaysAI/Projects/tree/master/LATAM_remote/Ciclopeia

More artificial intelligence!

The mission of Saturdays.ai is to make artificial intelligence more accessible (#ai4all) through intensive courses and programs where projects for good are carried out (#ai4good).

If you want to learn more about artificial intelligence join our community at community.saturdays.ai or visit us on our website www.saturdays.ai Join us!