Multimodal Framework Design, Children’s Multimodal Usage Patterns and the Efficacy of Mutual Disambiguation for Children

VoiceLog Usage Scenario

nScenario (U: user, VL: VoiceLog)

qU: “Show me the humvee.”

qVL: displays image of HMMWV

qU: “Expand engine.”

qVL: flashes area of the engine and replaces the image with a part diagram for the engine

qU: (while pointing at a region with the pen) “Expand this area.”

qVL: Highlights area, the fuel pump, and displays expanded view.

qU: (points to a specific screw in diagram) “I need 10.”

qVL: brings up order form filled out with the part name, part number, and quantity fields for the item.