III develops AI-powered audiovisual technology for dementia care

Taipei, Oct. 4 (CNA) The government-funded Institute for Information Industry (III) and industry partners have developed an artificial intelligence-powered technology that turns old photographs, voice recordings and family stories into personalized videos for use in dementia care, the organization said Sunday.

Developed by the III's Institute of AI Innovation (iAI), the "Multimodal Real-time Digital Scene Generation Technology Application Service" uses generative AI to recreate familiar scenes from patients' photos and life stories, replacing standardized care materials with personalized digital content.

The technology has completed a proof-of-concept trial at the dementia care home of Mennonite Christian Hospital in Hualien, where it was used for reminiscence therapy, cognitive stimulation and rehabilitation.

Carrie Huang (黃婉茹), deputy division head at the iAI, said familiar people, places and experiences can help trigger memories in people with dementia.

Edward Hong (洪毓良), senior strategy director at the iAI, said the multimodal system integrates photos, voice recordings and text descriptions with AI technologies including semantic analysis, image and video generation, and lip synchronization to create customized digital content.

Hong said the hospital monitors patients' brainwave activity while they watch the videos and changes observed can help refine future content.

Huang said patients at the Mennonite Christian Hospital facility wear location-enabled wristbands that allow the system to identify them when they enter an audiovisual room, where personalized videos are automatically projected.

According to feedback from frontline caregivers, patients who watched the personalized videos were more willing to communicate, participate in activities and, in some cases, engage more actively in rehabilitation, Hong said.