Open Access Open Access  Restricted Access Subscription or Fee Access

Multimodal Human Computer Interaction with Midas Intelligent Infokiosk

P. Tripureswari, S. Kshema

Abstract


In this paper, we present an intelligent information kiosk called MIDAS (Multimodal Interactive-Dialogue Automaton for Self-service), including its hardware and software architecture, stages of deployment of speech recognition and synthesis technologies. MIDAS uses the methodology Wizard of Oz (WOZ) that allows an expert to correct speech recognition results and control the dialogue flow. User statistics of the multimodal human computer interaction (HCI) have been analyzed for the operation of the kiosk in the automatic and automated modes. The infokiosk offers information about the structure and staff of laboratories, the location and phones of departments and employees of the institution. The multimodal user interface is provided with a touchscreen, natural speech input and head and manual gestures, both for ordinary and physically handicapped users.


Keywords


Multimodal User Interfaces, Human-Computer Interaction; Automatic Speech Recognition; Speech Synthesis; Artificial Intelligence, Infokiosk

Full Text:

PDF

References


L. McCauley, and S. D’Mello, “MIKI: a speech enabled intelligent kiosk. Intelligent virtual agents,” LNCS, Springer Verlag, vol. 4133, 2006, pp. 132–144.

M. Johnston, and S. Bangalore, “MATCHkiosk: A Multimodal Interactive City Guide,” Proc. Association of Computational Linguistics ACL-2004, Barcelona, Spain, 2004, pp. 223–226.

O. Aran, and L. Akarun, “A Multi-class Classification Strategy for Fisher Scores: Application to Signer Independent Sign Language Recognition,” Pattern Recognition, vol. 43(5), 2010, pp. 1776-1788.

N. Dahlback, A. Jonsson, and L. Ahrenberg, “Wizard of Oz Studies - Why and How,” Knowledge Based Systems, Elsevier, vol. 6(4), 1993, pp. 258–266.

A. Ronzhin, and A. Karpov, “Russian Voice Interface,” Pattern Recognition and Image Analysis, MAIK, vol. 17(2), 2007, pp. 321–336.


Refbacks

  • There are currently no refbacks.