Open Access Open Access  Restricted Access Subscription or Fee Access

Implementation of Communication between Normal and Differently Abled Person Using Hand Gesture and Voice Recogonition

M. Alfena, K. Bhavatarini Sarika, R. Chithara, S. Deepika, J.A. Dhinesh Joseph


A significant portion of our population has speech and hearing impairment which reduce their ability to communicate with a normal person. These impairments may be acquired at birth or through accident or disease or through aging .This paper  presents  the  development of a mobile phone based assistive aid for the speech and hearing impaired to help them in their daily life when they cannot afford to wear hearing aids .The visual aid takes form  an easy -to-use app on a mobile phone or tablet in which the elderly can capture the image of  their hand gesture ,the image will be processed and a corresponding audio is played  to the normal person. Then the normal person can record the speech using the mobile application which will automatically perform speech-to-text conversion on the mobile phone such that the impaired person can read the text visually. The conversion is done at the backend server via internet .This application is intended to enable the impaired person to lead a fulfilling and independent lifestyle.


Hand Gesture, Speech-to-Text

Full Text:



WHO, “Deafness and Hearing Loss”,, Feb 2013.

Jerger J, Chmiel R, Wilson N, et al. “Hearing impairment in older adults: New concepts”. J Am Geriatric Soc 1995; 43:928-35.

Wall Hagen MI, Strawbridge WJ, Kurata J, et al. “Comparative impact of Hearing and vision impairment on subsequent functioning”. J Am Geriatric Soc 2001; 49:1086-92.

Dr Tan Teck Shi, MS Esther Lim Ching Yee, A/P Low Wong Kein, Dr Tan Ngiap-Chuan, “Effective Communication with Hearing Impaired Patients : An Approach for Family Physicians”, the Singapore Family Physician, vol. 37, No. 1, Jan – Mar 2011, pp. 61-66.

M B Jennings. “Hearing rehabilitation for older adults: an update on Hearing aids, hearing assistive technologies, and rehabilitation services”. Geriatrics and aging 2006; 9(10): 708 –11.

S. Smith, "Android SDK: Using the Text to Speech Engine," 15 November 2011. [Online]. Available: engine/?search_index=13.

CMU Sphinx Group, "A speech recognizer written entirely in the JavaTM programming language," 2013. [Online]. Available:

R. W. Root and S. Draper, "Questionnaires as a software evaluation tool," in Proceedings of CHI 83, New York, 1983.

A. M. Lund, "Measuring Usability with the USE Questionnaire," Usability & User Experience, 2001. [Online]. Available: [Accessed 14 April 2014].

D. K. Farkas, "Evaluation and Usability Testing," Washington, 2013.


  • There are currently no refbacks.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.