Seeing AI: Microsoft and Haleon launch-AI driven labeling app for the visually impaired

17 Oct 2022 --- Last week, on World Sight Day, Microsoft and Haleon launched a new feature to Microsoft’s Seeing AI app that reads health product labels aloud for people who are blind or have impaired vision. The app is currently available in the US and UK and compatible with over 1,500 consumer health products. 

The Seeing AI app provides an audio read-out of important information, such as product name, ingredients and usage instructions by scanning the barcode of Haleon products. The app already had functionalities designed by Microsoft to “help people who have trouble seeing by narrating the world around them.” 

“We believe everyone should have access to self-care products, services and the information needed to make informed, proactive choices about their health needs. Haleon initiated this collaboration with Microsoft via its Seeing AI app to make consumer health more accessible, achievable and inclusive,” says Katie Williams, US chief marketing officer at Haleon.

Microsoft partners with Haleon to expand its SeeingAI app to read out labels for all of Haleon's healthcare products. Seeing AI
When users hold their phone camera over the packaging barcode the app will read out the product name and all text on the package. Users can skip ahead or move back to the relevant section they want to listen to. 

According to the Center for Disease Control and Prevention (CDC), 12 million American adults over the age of 40 are blind or have low vision. 

The read-aloud component is beneficial not only to the visually impaired but also can be utilized by those who have difficulty reading product labels due to low literacy.

The features were designed with “advancing inclusivity and improving accessibility” at the top of mind for Microsoft engineers. The lead engineer on the project, Saqib Shaikh, lost his sight at the age of seven, giving special insight into what functionalities are critical to users. 

“AI’s intelligent barcode scanner plays audio cues to help you find the barcode, and now the information displayed for Haleon products is coming straight from the manufacturer, providing richer information including usage instructions and ingredients. This can be invaluable for someone who cannot read the label, leading to greater independence,” he says. 

The AI app technology is an extension of the companies’ commitment to improving healthcare inclusivity. The companies have stated they hope their innovation encourages others in the industry 

The partnership is planning to expand the availability of the barcodes globally and add additional languages in the future. Seeing AI is operative on the Apple App Store and will be available on Android in the future without a declared date.Haleon commits to striving for a more inclusive global healthcare system. 

Some of Haleon’s popular products the barcode will be accessible on includes Sensodyne, Centrum, Emergen-C, ChapStick and Aquafresh.

Health Inclusivity Index
Haleon recently released a Health Inclusivity Index that measures how successful countries are in creating an inclusive healthcare system. 

Haleon conducted a global study of 40 countries regarding how inclusive their policies are to remove the personal, social, cultural and political barriers that could prevent people from achieving good physical and mental health.

The Index finds that the UK has the most inclusive approach to healthcare at a national policy and program level, followed by Australia, France, Germany and Sweden. Other countries in the top ten include South Korea, Canada, Israel, Thailand and Switzerland. 

By Sabine Waldeck