What is Holli?

Holli is a software for a wearable system, Google Glass, that coaches children with ASD in everyday social interactions. It was designed by a team of scientists, led by University of Toronto assistant professor Azadeh Kushki.

How does it work?

The user and the interlocutor’s speech is captured by Google Glass’s microphone. The speech recognition engine  converts the spoken input into text which is displayed on the user’s glasses. Then this text is processed and an appropriate set of responses is generated. At that point, the user can select one of the responses and say it to the interlocutor. The prompts disappear from the display and the app listens to the next speech input of the conversation.

The study involved testing the app on 15 children with ASD. The system was almost 90 per cent accurate at detecting the researcher’s speech, and the average user response time 2.5 seconds.

https://www.youtube.com/watch?v=XLf5MIx1mRk

How did children react?

Children were able to use this new technology and enjoyed interacting with it. They were encouraged to make eye-contact and were excited about the rewards the system provided them with.

https://www.youtube.com/watch?v=7GDiI7Oi8NE

What do the researchers say?

Researchers highlight the real-world element of their app and its potential to teach transferable skills.

Dr Azadeh Kushki notes: “The interesting thing about our new technology is that we […] coach children who are communicating with people in real-world situations. […] Children can practice their skills outside of their normal therapy sessions and it can provide them with increased independence in everyday interactions.”

Google Glass

Brain Power ‘s  Autism System