Facing Dementia with MR

Company: Microsoft
Role: Interaction Designer
Platform: Hololens, Sketch
Date: 02/2017 - 05/2017
Link: Project blog


What is Microsoft Design Expo?

Microsoft Design Expo is a design challenge competition every year, this year's topic is using MR technology to think about current reality's potential possibilities. Our team inspired by team member's personal family story, we hope to do something helpful for people with dementia.


Our project goal is to come up with some concepts about using MR technology to face against dementia, based on solid and thoughtful user research to deliver a workable prototype.


In order to know more, we did researches about market, people with dementia, relatives, caregivers and so on. There were 48.6 million people live dementia around the world, but as we may know, there is no cure so far.

Target Users

At this stage, based on some pre research, our target users will be people with early stage dementia because of feasibility consideration. However, we also hope to expand target groups to large population of people with dementia further.


Lack of professional knowledge: even the stories about dementia have been sharing around all the time, we found ourselves so lacking of understanding of this 6th disease.
Tight Timeline: this project lasts 3 months, which we are sure not enough to build every functional features. Instead, we can only focus on the concept ideation and pitching.

My Role & Team

My Role: I played the role of interaction designer, designer and prototyper, took participation in the process of ideation, market research, user interviews, video editing, prototyping.

Team: I worked on a cross functional team of two technologists, two UX/UI designers.


User Interviews

As our limited understanding of dementia, the first thing we did is to do interviews with people. We asked people from NYU program ITP, whose family members were suffering from dementia or Alzheimer's Disease, to ask more first hand information.

Cognitive Training

We found several valuable insights, but the most interesting profound one we found is that people with early stage dementia often use cognitive trainings to keep their brains active.


In detail, there are lots of methods people can choose from in relation to different symptoms and training purposes. One type of the Most commonly used methods is paper-based games, however, while doing these training, users focus on what’s in their hands, so they are disconnected from the environment they lived in.

Our Solution

Our solution is to make a Mixed reality application to help people with early stage dementia to receive self-treatment any time in their homes. This application will be customized by doctors for each individual user, and there will be web system for doctors to track training process and make adjustments accordingly.

Key Features

- The app can be used at home because with a familiar environment, users might feel calm and less stressed.

- The app has a self-training therapy quality. Instead of asking users to perform treatment in front of people, the self-training method will ease users’ anxiety. In this way, the users train themselves instead of go to a clinic, this saves families money and time.

- To provide more specific therapy, the app can also be customized by doctors and caregivers according to a user's personal interests, background, and their recent conditions. In addition, user’s family or caregiver can record instructions to make it easier for users to follow the therapy.

Doctor System

Another important part of our system is customization. In our interviews with caregivers and professors, we value this direction most as it has been proven that, doctors's pain points are:
• not able to track progress
• not able to change cognitive training tasks easily

Video Demo


NYU Alzheimer's Disease Center

We found several valuable insights, but the most interesting profound one we found is that people with early stage dementia often use cognitive trainings to keep their brains active. Which in many ways, the cost and time behind cognitive trainings are burdens of a family.

- “It is valuable to allow people maintain their daily interactions and routines at home."
- “It is important to keep the person doing what they can for as long as possible.”
- “Training process should include caregivers for smoothest transitions.”

User Testing

We brought these prototypes for people to test them. We approach family members of people with dementia. We gained some really positive feedbacks:
- They want to see more use of the spatial space
- They want more intuitive gestures for people with dementia


We want to explore the social aspect in cognitive training and offer the option for caregivers to conduct the training with people with dementia. We believe it would better satisfy the emotional needs of people with dementia and allow caregivers to better empathize with people with dementia.


We realize much Alzheimer's Disease and dementia are such tough and global issues that so far no cure, but we are also very excited about the accomplishments we got so far, we will keep on exploring this technology. We are open to any questions and look forward to share more details. 

Reach us through emails:

Michael Dory, Nai-Chen Yang, Yun Li, Yiting Liu, Xingjian An
All Copyrights Reserved