Tech for Good: How We're Making Medical Tests Accessible for People with Visual Impairments

What is a quick glance at a plastic strip for most of us represents a massive barrier for people with visual impairments: reading medical rapid tests. Whether COVID-19, influenza, or pregnancy tests, without help from others, the result is often impossible to determine independently.

In collaboration with the non-profit organization MI4People, Leo Back, a full-stack developer at AOE, has developed a solution that addresses this problem using modern AI and native accessibility features.

Who is MI4People?

Behind the name MI4People (Machine Intelligence for People) is a non-profit organization (gGmbH) dedicated to bridging the gap between cutting-edge AI research and charitable purposes. While AI most often ends up in commercial products, MI4People identifies socially relevant problems in areas such as healthcare, environmental protection, and education, and develops AI-based software solutions for them.

The connection to MI4People came through a networking event hosted by AOE called Sustainable Future .digital. That's where the first conversations began, and before long, Dr. Denis Dalic (CTO of MI4People) had a project for us where we could contribute our development expertise. Following an internal call for volunteers, Leo stepped forward. He took ownership of the project from that point on, including putting in time on weekends.

Making Test Results Accessible with an "Audio UI"

The goal: an app that allows users to take a photo of their test kit. An AI analyzes the image and reads the result aloud via voice output.

"The big challenge at the beginning was rethinking the approach," Leo explains. "As a developer, you usually build graphical interfaces. Here, the GUI was really only there as a feedback channel for us. For the end users, we designed a purely audio-based UI."

To do this, Leo relied on proven standards:

• React Native as the app framework
• VoiceOver (iOS) and TalkBack (Android) as interfaces

AI Power Makes It Possible

The project originally started in 2020 with a custom-trained neural network. But when Leo took it over, the code was difficult to maintain and limited to COVID tests. The breakthrough came with the rapid advancement of Large Language Models (LLMs). After experimenting with GPT-4o, Leo found the "sweet spot" with GPT-4o mini, a model that offers high precision at low cost. In addition to COVID tests, the app can now also output results for pregnancy tests and so-called "5-in-1 virus tests."

"Since MI4People is a non-profit, the cost-benefit calculation was crucial," Leo explains. "Every request costs tokens. We had to find a model precise enough to detect the lines on a test kit without blowing the NGO's budget."

How Does the AI Handle Edge Cases?

With medical results, there is no room for error. AI models, however, tend to hallucinate an answer rather than admit they cannot recognize something. Through intensive prompt engineering and testing with over 50 different image scenarios, Leo built in "guardrails":

• The AI is instructed to report "Invalid" when in doubt, rather than risk a false positive or negative result.
• If the image is too blurry or out of focus, users are prompted to take a new photo.

A Win for Everyone Involved

This commitment pays off for all parties. "I see this as an absolute win-win-win situation," Leo reflects. By independently guiding the project, he was able to dive deep into forward-looking technologies and significantly expand his knowledge in app development. For AOE, the collaboration adds a tangible social impact project to the portfolio, one that demonstrates how AI expertise can be applied to corporate social responsibility.

The most lasting success, however, belongs to MI4People and society: a concrete gain in accessibility has been created, giving people with visual impairments back an important degree of autonomy and privacy in their daily lives. "AI unfolds its true value not in the maximization of profit, but in its ability to create social benefit," says Dr. Denis Dalic.

What's Next?

The app is close to release. It is currently in a testing phase with the German Federation of the Blind and Partially Sighted in Hesse to ensure smooth day-to-day usability. Once the feedback has been incorporated, the app will be published for free in the App Store under MI4People's account, under the name CoVision.

We are proud that our developers are using their expertise to make the world a little more accessible for everyone!