In this course we look at many things that are often overlooked in computer science - how to push the capabilities of the physical interfaces that connect us with the computers we use.
At this moment in history the possibilities are exploding, as the fruits of Moore's Law are reaching the point where cameras and vision-based analysis, musical instruments as interfaces, robotic interfaces, immersive environments, networked hand-held displays and many other physical modalities are becoming capable of transforming the very idea of using computers for communication and problem solving.
We will explore (and invent) new modalities in input and output, and look at both old and new ways of using physical devices and human senses to allow people to effect communication and expression with computers.
This is a projects course. You will be expected to show initiative and to be able to think "outside of the box".
LINK TO STUDENT PROJECT PAGES
NOTE: THE SECOND CLASS IS MONDAY FEBRUARY 6
INFO ABOUT SHADOW WALL TYPE INTERFACES
John Gerrard's "The Ladder"
You can also start to check out and explore the many cool and exciting ideas by STEPHEN LEWIS
We will learn the basics about the Basic Stamp. You might want to look at Aditi Kapoor's PIC page.
We will also learn the basics about grabbing video images for video processing. One way to do this it through the Processing programming environment, which works with a Video tracking library.
Here are Stephen Lewis' notes about this, andAnother option for a vision library is this helpful page on Computer Vision for Artists.
Stephen Lewis' example source code.
Another example from Stephen Lewis: tracking video blobs and turning them into a tone melody.
By the way, Stephen Lewis's email address is:
SLewis AT trevornet DOT org
Install Processing from http://www.processing.com, and also install the other things you need to run Processing, as per below.
If you have a notebook computer but you don't have a USB video camera, then you should buy yourself a webcam (it doesn't need to be a fancy one - you can get one for less than $50 from many stores likes Staples, RadioShack, etc.).
Here are the other things you need to do to get Video input working in Processing on a Windows machine:
When you install Quicktime 6.5.2, when you get to the "Choose Installation Type" window (about five clicks in) make sure to check "Custom", and then on the next window, scroll down to "QuickTime for Java" and check that box - or you can just press the "Select All" button.
Now you can go ahead and open your version of Processing, and look at the examples in submenu Sketchbook→Examples→Libary-Video
For those who would like to work in C++ (using Visual Studio 7.0), Douglas Summers-Stay has created this helpful web page.
IN OTHER NEWS...
Read README.doc which contains a simple assignment, written by Evan Barba, of what you need to do to get started on PIC programming for this class. You should get to room 1222 where we have the PIC software/hardware set up (the room just before our classroom along the hall) at least once, and do the assignment described in this document before the March 20 class.
Guest lecture by Ramesh Raskar from MERL. Also, Ramesh is looking for potential summer/fall interns. He sent me the following email:
Ken , can you please also forward this to student list for students who maybe interested in summer or fall internships at MERL ? http://www.merl.com/people/raskar/intern.html I will be happy to meet the students to answer any questions about MERL and internships Monday afternoon.
For Monday's class, and all classes thereafter, if you possess a notebook computer, please bring it with you to class.
At next class (April 10) each of you will be giving a short presentation of two things that you will do this week, if you have not already done it:
I will now discuss each of these two projects in turn. I suggest that you do the video project in the first part of the week (plan to be done with this by Wednesday or Thursday) and spend the rest of the time working on the embedded processor project.
To do the Video project, if you don't already have a project in mind and a path to get there, you can follow these instructions:
There are also two extra installable vision libraries in addition to the one that is already built in. You might want to install those as well.
Embedded processor project
If you haven't already dived into using the Basic Stamp processors (or your own board built from a PIC), the Arduino kits have some advantages. You can program them in Processing and they have lots of source code for examples on line.
On Tuesday morning I will be ordering Arduino kits ($30 each) for anyone who wants to use them for your projects. You'll each want your own kit. Optionally, you might also want me to put in an order for you for a Sharp Infrared Proximity sensor ($12) because these are very very cool, and you will be able to astonish friends and family with the projects you can to with them!
If you want to use Arduino to do your Embedded Processing project, then follow these steps:
Guest lecture by NYU professor Nava Rubin about visual perception and the neural basis of human vision, and cool experiments that are being done in that area here at the NYU Center for Neural Science.
Casey Muller gave a guest lecture, describing research with physical media from the student's perspective. He showed some videos and photos of his past work, and showed tools such as rfid readers, VFDs, and input pads. Then he showed a live demo of the GPU stereo and discussed related issues, such as why depth helps toon shading, the tragedy of the commons in wireless networking, and how design by numbers became processing. After this, the students demonstrated the current state of their projects.
Here are this week's class notes.
END OF SEMESTER SHOW
Students present their work for the semester. Please try to show up at 4:30pm, to give some time for set-up. Feel free to invite your friends.
Refreshments will be served.