Extensible MultiModal Environment Toolkit (EMMET) Geometry Placer Demo


This is a simple demo of my Extensible MultiModal Environment Toolkit (EMMET). In this demo geometric objects can be created, colored, or deleted using speech, gesture, mouse and keyboard


Instructions:

SPEECH: (Speech recognition occurs only while you are holding down the left CTRL key)
   Say "Add a(n) <object type> here" to create an object at the mouse cursor location.
      (<object type> can be "ball", "cube", or "pyramid")
       Note: The same object type can be added repeatedly by saying "and here".
   Say "Erase this" to erase the object which is currently under the mouse cursor
   Say "Color this <color>" or "Paint this <color>" to color the object under the mouse cursor
      (<color> can be "red", "blue", or "green")
      
GESTURE: (Draw gestures by holding down the left mouse button and dragging)
   Draw a circle, square, or triangle gesture to create a ball, cube, or pyramid respectively
   Draw a single stroke from left to right through any object to erase it
   Draw a single stroke capital 'R', 'B', or 'G' to color the object red, blue, or green
   
KEYBOARD:
   Press the 'B', 'C', or 'P' key to create a ball, cube, or pyramid 
   Press the 'R', 'B', or 'G' key to color an onject red, blue, or green
   Press the 'E' key while the mouse cursor is over an objec to erase it
   
MOUSE:
   The current object creation mode is indicated to the lower left. To cycle through 
   the different creation modes, click on the object creation mode indicator button.
      When the mode is "ball", "pyramid", or "cube", RIGHT-clicking on the floor will 
         create the respective object
      When the mode is "erase" RIGHT-clicking will erase the clicked upon object
   The current "paint color" is also indicated to the lower left. To cycle through the 
   different colors, click the "paint color" indicator button           
      RIGHT-click on an existing object to color it the current paint color

    Notes:
  • EMMET requires Java 1.4+ with Webstart. The above link should direct you to the Java download site instead of launching the demo if these requirements were not detected on your machine.
  • Speech recognition assumes a microphone is connected to your machine and a speech recognition engine is installed.
    • Most recent laptops have built in microphones and many PCs are packaged with a sufficient microphone
    • Windows 2000 and WindowsXP include compatible speech recognition engines by default. I recommend that you set up your microphone through the Control Panel->Speech control.
  • The demo will soon display a dialog requesting feedback on your experience with the interface. To aid in my research, please fill out the brief survery and allow the results to be trasmitted. No data other than the multimodal usage statistics will be transmitted.