Persons with motor impairments such as cerebral palsy, or neurological impairments such as autism, are often disadvantaged by an inability to communicate independently. Embedded systems, such as tablet computers, can be used as AAC devices and speak on behalf of the individual. However, most young users are first generation learners, unfamiliar with English. Hence, this project envisages the creation of an embedded system that can support multiple Indian languages, and comes in a ruggedized child-friendly package with access switches. The device will also be optimized for use in a classroom, with headphone/external speakers, enabling its use as an independent communication device. We aim to transform the tablets into advanced AAC devices/device platform that are robust, easy to use, hold and carry, can be produced on a large scale, and can be easily adapted across different regions of the country.

Through this project, we will prototype multi-lingual AAC devices, specifically for people who are non-verbal such as those with motor impairments, or those who are autistic. Re-packaging commercial tablets as a child-friendly device, along with access switches, external speakers and local language support will enable us to deliver solutions compatible with the needs of users. The software will be ported onto the tablets with required specialized peripherals such as contact-less switches (ADITI) and intelligent gesture recognition (iGest) for persons with cerebral palsy. The devices will be supported with specialized training for the educators. We will design the specialized curriculum and teaching methods to closely integrate the use of AAC devices in the teaching sessions, and provide specialized training to the educators.

Inadequate languages, leading to cognitive and social difficulties, are problems faced by young adults on the autism spectrum.  Given that typical studies so far have focused on language loss of verbal children on the autistic spectrum, there is little attention given to concept loss, which incorporates language as well as behavioral issues. We will research this loss of language as well as concepts in the early years of a child on the autism spectrum and find strategies that can be created through audio-visual means to bridge the gaps as they reach adulthood. This research will be directed especially to address the Indian context.

Objectives

  1. Design, prototype and collect field data on embedded systems for use as alternative and augmentative communication (AAC) devices.
  2. Provide access switches with tablet-type personal computing devices suitably modified for use by first generation learners and persons with motor and neurological impairments.
  3. Develop software, with support for multiple Indian languages, to mitigate the effects of autism spectrum disorders among young adults.

Current Status

IIT Madras has been at the forefront of developing AAC solutions, and had prototyped KAVI (communicating audio-visual interface) as early as 2008. Efforts have been ongoing to develop and improve access switches such as ADITI and iGest, and software such as HOPE, KAVI-PTS (picture to speech) and Mozhi (word builder in Tamil). IIT Madras also has support for text to speech in Indian languages, with a small footprint, suitable for use on tablet computers and phones.

KAVI is envisaged as an embedded system capable of running picture based text to speech software such as Hope or Sanyog. The key innovation is a two speaker system that would allow the user to create a sentence on a headphone and then play the completed sentence over a speaker. This arrangement would permit the use of alternative and augmentative communication (AAC) software in a public setting such as a classroom, or a playground. KAVI-PTS is a picture to speech software developed in collaboration with Mindtree Foundation for the Android platform. KAVI-PTS is used in conjunction with a local language text to speech engine that has been ported to the Android platform.

Need

There is no device that addresses the needs of first generation learners in low resource settings. A limited number of Android tablets and iPads are available in a few schools, but the software is cumbersome and there are no access switches that work easily with these devices. The present commercial eco-system for embedded systems is an opportunity for us to deliver an AAC device with

  1. An ergonomic enclosure for a tablet with suitable specifications
  2. Accessories (speakers, switches, battery pack) within the enclosure

Outcomes

The measureable outcomes of this effort will accrue during the user trial and testing phase. We expect to

  1. enable independent communication for persons with motor impairments
  2. understand issues related to cognition, learning and social behavior among young adults with disability
  3. provide a low cost indigenous AAC devices suitable for export to other countries in low resource settings
  4. provide a standardized platform for use with other accessories for persons with disability

We expect to license the technology out to industry for manufacturing, sales, marketing and to NGOs for training and after-sales support. The impact of this effort will be measured through a combination of user studies as well as using internal log files on the tablet. We will document usage patterns, a child’s ability to progress from using pictures of objects to more abstract ideas and finally to word and sentence building. These are typical milestones used to monitor language development, and we aim to incorporate these into our software. In addition, usage patterns in a school, in terms of number of children (if the tablet is shared) and the types of curricula where AAC can be incorporated will be documented, to help us better design hardware interfaces and software functionality.