CiclopeIA: Artificial intelligence to help people with visual disabilities

Latam online. Second Edition. 2021

Imagine a morning of errands: choosing clothes, making coffee, reaching the bus stop and buying groceries. For someone with a visual impairment, each task can present barriers when the environment is inaccessible. An obstruction on the pavement, missing audio information about a bus route or difficulty identifying a banknote can complicate everyday activities. These situations are the starting point for our project.

Accessibility barriers can restrict participation in work, shopping and leisure. The aim is to reduce those barriers and support independence, without assuming that everyone with a visual impairment has the same experiences or needs.

When sight loss occurs later in life, adjustment may involve emotional support, training and accessible tools. Each person experiences that process differently.

Designing with users and listening to their priorities helps focus technology on specific needs.

In the WHO material consulted for the project [1] “Globally, it is estimated that approximately 1.3 billion people live with some form of vision impairment.

Globally, the main causes of poor vision are uncorrected refractive errors and cataracts.

“Most people with low vision are over 50 years old.”

In addition, vision impairment or loss has impacts on the quality of life of people who suffer from it, according to WHO, “Young children with severe vision impairment at an early age may suffer delays in motor, linguistic, emotional, social and cognitive development.”

“For older adults, vision impairment can contribute to social isolation, difficulty walking, increased risk of falls and fractures, and increased likelihood of early admission to nursing homes.”

The Lancet Global Health [ 2] states that:

“In 2020, an estimated 596 million people worldwide have distance vision deficiency, and of these 43 million are blind.”

“A large proportion of these (90%) live in low- or middle-income countries.”

“By 2050, population aging, growth and urbanization will lead to an estimated 895 million people with distance vision impairment and 61 million blind.”

Given this problem, and as part of the practical application of a Machine Learning course [ 3] and Artificial Intelligence (AI) [4], The idea of using technology to identify ideas that can address the problems of people with visual disabilities and design a tool that supports them in achieving their daily activities was born. The group's idea was to think of a project that could contribute socially and have an impact on the community. From the point of view of the Sustainable Development Goals (SDGs) [5] it relates to health and wellbeing, industry and innovation, reduced inequalities and partnerships for the goals.

The project Ciclope.IA, As we have called it, it seeks to integrate into a cell phone application, different options (skills) aimed at solving limitations that people with visual disabilities experience in their daily activities such as: Recognition of bills and coins quickly and effectively, identification of the bus line, identification of colors, knowing the fill level of a container, finding lost objects, etc.

The proposed scope is broad, so we started with one specific function: identifying US dollar banknotes through a phone’s camera. After detection, the app plays a voice message with the result of the recognized amount. The use of Ciclope.IA provides the user with security and autonomy when making cash transactions and reduces the risk of being a victim of deception. Interaction with the application can be done in Spanish under the Android operating system.

Initial Project Scope

Conversations with a group of people with visual impairments identified several useful features. We prioritised banknote identification, the feature most requested by that group, and chose voice input and output for interaction.

To support our configuration we reviewed some statistics on the site yiminshum.com, “there are currently 5.19 billion unique users on mobile devices, where it does not divide the type of phone, this covers 67% of the population.”

"73% of people are connected and share their time from a smartphone. 23.5% are associated with a common telephone, where its functions are the basic and limited ones that a telephone must fulfill, which is to call and send messages, and 3.6% are associated with a router, tablet or mobile PC."

“OS, better known as an operating system, is important for the operation of computers in the world, 74% of users are on Android devices, 25% is iOS, 0.4% is KAI, 0.2% is Samsung OS and 0.6% is other operating systems.”

With this information, the scope of the project was limited to Android smartphones, which cover a large part of the market, especially in places other than the United States, to Spanish-speaking countries that have the dollar as their currency.

Applied AI

Faced with the identified problem, it was determined that the best way to support blind people is to develop a cell phone application that acts as their eyes in situations that are required. For this, from a technical point of view, different image recognition models that could be applied were explored, and in the end it was decided to use YoloV5 due to its versatility when identifying objects and based on tests carried out by Towards Data Science [6] that recommend the model over another also well-known one. Other elements that we considered were the possibility of using the model without the need to have an active internet connection but rather a pre-trained model that is copied to the cell phone in a Pytorch Lite version that takes up less size and resources.

To train the model, available dataset options were explored. Unfortunately, one was not available, so we decided to create our own dataset.

One challenge was to gather enough banknote images in different settings, including both front and back. We collected images through web scraping, then labelled them with the online tool https://labelflow.ai/ Labels were assigned to each image to identify each bill in the different images.

After training the model in Google's colab tool, a file was obtained with the best model generated and used in the cell phone application.

Without a doubt, AI will provide solutions to many of today's problems and will allow its application to spread massively in different areas of knowledge and daily life. Our work is a small sample of the potential to be exploited with AI and a contribution for those interested in supporting groups such as the blind who must overcome adversity with little or limited support from society.

Conclusions

Artificial Intelligence tools can be used for multiple purposes, however, developing products that provide solutions to social needs represents an enormous opportunity that results in well-being and improvement in the quality of life of minority groups in society.

From a technical point of view, the project represented an opportunity to continue learning from this exciting world of AI and understand, among other things, that to create effective models it is important to work exhaustively on:

• Create or have quality datasets.
• Apply different models to evaluate performance.
• Make users participate in product development.
• Work with multidisciplinary teams.
• Maintain a permanent search for new solutions.

Demo video

Open video or resource

References

[1] World Health Organization. (2021). Blindness and visual impairment. Retrieved January 12, 2022, from https://www.who.int/es/news-room/fact-sheets/detail/blindness-and-visual-impairment.

[2] The Lancet Global Health. (2021). The Lancet Global Health Commission On Global Eye Health: Vision Beyond 2020. Retrieved January 12, 2022, from https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(20)30488-5/fulltext

[3] BBVA (2019). What is Machine Learning and how does it work? Retrieved February 4, 2022 from https://www.bbva.com/es/machine-learning-que-es-y-como-funciona/

[4] Juan Antonio Pascual Estapé(2019). Artificial intelligence: what it is, how it works and what it is currently used for. Retrieved February 4, 2022 from https://computerhoy.com/reportajes/tecnologia/inteligencia-artificial-469917

[5] United Nations (2021). The agenda for sustainable development. Retrieved February 4, 2022. https://www.un.org/sustainabledevelopment/es/development-agenda/

[6] Towards Data Science (2020). YOLOv5 compared to Faster RCNN. Who wins?. Retrieved February 4, 2022. https://towardsdatascience.com/yolov5-compared-to-faster-rcnn-who-wins-a771cd6c9fb4

Members

• Alexander Cortes
• Ariosto Olmedo Cabrera
• Antonio Paucar
• Carlos Sesma
• Miriam Quimi
• Santiago Yunes
• Viviana Márquez

Project presentation: DemoDay

Open video or resource

Repository

In the following repository you will find the code used to develop this application: https://github.com/SaturdaysAI/Projects/tree/master/LATAM_remote/Ciclopeia

More artificial intelligence!

Saturdays.AI makes AI more accessible through shared learning and projects with a social purpose. Our motto is AI4ALL IS AI4GOOD.

Meet the Saturdays.AI community