In this course we look at many things that are often overlooked in computer science - how to push the capabilities of the physical interfaces that connect us with the computers we use.

At this moment in history the possibilities are exploding, as the fruits of Moore's Law are reaching the point where cameras and vision-based analysis, musical instruments as interfaces, robotic interfaces, immersive environments, networked hand-held displays and many other physical modalities are becoming capable of transforming the very idea of using computers for communication and problem solving.

We will explore (and invent) new modalities in input and output, and look at both old and new ways of using physical devices and human senses to allow people to effect communication and expression with computers.

This is a projects course. You will be expected to show initiative and to be able to think "outside of the box".

LINK TO STUDENT PROJECT PAGES

NOTE: THE SECOND CLASS IS MONDAY FEBRUARY 6

INFO ABOUT SHADOW WALL TYPE INTERFACES

John Gerrard's "The Ladder"

You can also start to check out and explore the many cool and exciting ideas by STEPHEN LEWIS

We will learn the basics about the Basic Stamp. You might want to look at Aditi Kapoor's PIC page.

We will also learn the basics about grabbing video images for video processing. One way to do this it through the Processing programming environment, which works with a Video tracking library.

Here are Stephen Lewis' notes about this, and
Stephen Lewis' example source code.

Another example from Stephen Lewis: tracking video blobs and turning them into a tone melody.

By the way, Stephen Lewis's email address is:

SLewis AT trevornet DOT org

Install Processing from http://www.processing.com, and also install the other things you need to run Processing, as per below.

If you have a notebook computer but you don't have a USB video camera, then you should buy yourself a webcam (it doesn't need to be a fancy one - you can get one for less than $50 from many stores likes Staples, RadioShack, etc.).

Here are the other things you need to do to get Video input working in Processing on a Windows machine:

• Go to the release notes page of vdig.com and install Version 1.01. Once you have installed VDIG, you should run its testing program (it's called QTCap - it will be a program from your Start menu) with your camera, just to make sure everything works ok.

• If you have Quicktime 7 installed, you'll need to uninstall that before the next step.

• Install Quicktime 6.5.2, which you can get at: http://www.apple.com/support/downloads/quicktime652forwindows.html. Save the download executable to your harddrive, and then run it.

  Important:

  When you install Quicktime 6.5.2, when you get to the "Choose Installation Type" window (about five clicks in) make sure to check "Custom", and then on the next window, scroll down to "QuickTime for Java" and check that box - or you can just press the "Select All" button.

Now you can go ahead and open your version of Processing, and look at the examples in submenu Sketchbook→Examples→Libary-Video

For those who would like to work in C++ (using Visual Studio 7.0), Douglas Summers-Stay has created this helpful web page.

IN OTHER NEWS...

Read README.doc which contains a simple assignment, written by Evan Barba, of what you need to do to get started on PIC programming for this class. You should get to room 1222 where we have the PIC software/hardware set up (the room just before our classroom along the hall) at least once, and do the assignment described in this document before the March 20 class.

Guest lecture by Ramesh Raskar from MERL. Also, Ramesh is looking for potential summer/fall interns. He sent me the following email:

Ken , can you please also forward this to student list for students who maybe interested in summer or fall internships at MERL ? http://www.merl.com/people/raskar/intern.html I will be happy to meet the students to answer any questions about MERL and internships Monday afternoon.

For Monday's class, and all classes thereafter, if you possess a notebook computer, please bring it with you to class.

At next class (April 10) each of you will be giving a short presentation of two things that you will do this week, if you have not already done it:

• A demonstration of a simple video-tracking project

• A demonstration of a simple embedded processor based project (a PIC-like device, perhaps packaged as Basic Stamp or the Arduino).

I will now discuss each of these two projects in turn. I suggest that you do the video project in the first part of the week (plan to be done with this by Wednesday or Thursday) and spend the rest of the time working on the embedded processor project.

Video project

To do the Video project, if you don't already have a project in mind and a path to get there, you can follow these instructions:

• Make sure you have already installed the video drivers that I referred to in the March 6 class notes.

• Install Processing from http://www.processing.org.

• In the menus at the top, click on Help → Reference, and then click on the "libraries" link (second paragraph on the left column).

• You might want to install the sound library - you'll need to do that in case you're going to do any sound output.

  There are also two extra installable vision libraries in addition to the one that is already built in. You might want to install those as well.

• When you're running Processing, you'll see example programs at menu: File → Sketchbook → examples → Library-Video. For example, the "BrightnessTracking" example finds the brightest pixel in the image captured by a camera. Choose one of those examples, run it, make some cool modification to it by hacking on the source code, and show your cool result at next week's class.

Embedded processor project

If you haven't already dived into using the Basic Stamp processors (or your own board built from a PIC), the Arduino kits have some advantages. You can program them in Processing and they have lots of source code for examples on line.

On Tuesday morning I will be ordering Arduino kits ($30 each) for anyone who wants to use them for your projects. You'll each want your own kit. Optionally, you might also want me to put in an order for you for a Sharp Infrared Proximity sensor ($12) because these are very very cool, and you will be able to astonish friends and family with the projects you can to with them!

If you want to use Arduino to do your Embedded Processing project, then follow these steps:

• When the Arduino board arrives (maybe by Wednesday, certainly by Thursday), go on the web to http://arguino.cc.

• In the menu to the right, click on "TUTORIALS".

• Choose one of the projects listed, and do it. We are providing a a grab bag of input and output devices, including small joysticks, proximity sensors, bend sensors, photocells, microphones, LED lights, buzzers, and so forth, so you will have plenty of flexibility!

Guest lecture by NYU professor Nava Rubin about visual perception and the neural basis of human vision, and cool experiments that are being done in that area here at the NYU Center for Neural Science.

Casey Muller gave a guest lecture, describing research with physical media from the student's perspective. He showed some videos and photos of his past work, and showed tools such as rfid readers, VFDs, and input pads. Then he showed a live demo of the GPU stereo and discussed related issues, such as why depth helps toon shading, the tragedy of the commons in wireless networking, and how design by numbers became processing. After this, the students demonstrated the current state of their projects.

Here are this week's class notes.

END OF SEMESTER SHOW

Students present their work for the semester. Please try to show up at 4:30pm, to give some time for set-up. Feel free to invite your friends.

Refreshments will be served.