Multimodal Framework Design, Children’s Multimodal Usage Patterns and the Efficacy of Mutual Disambiguation for Children

Proposed Framework API Implementation: Overview

nJava based API

nSpeech Recognition (JSAPI) nGesture Recognition (Customized HHReco)

nMultimodal Integration

n3D Rendering Environment (Perlin Java 3D Renderer)

User Interface

Speech Recognition (JSAPI)

Gesture Recognition (HHreco+)

Multimodal Integration Agent

Application Interpretation

Statistics Collection Engine