Multimedia


Home

Multimedia Project

by Brandon Keely, Cris Ross, Andrea Bianchi, Cheng-Cheng Ku

4 Dimensions Camera Mouse

Introduction
Prototype Demo
Application
Application Demo
Source Code

Introduction

Our primary goal of this project is to use a camera as an enhanced mouse that provides 4 dimensions control(X, Y, Z and Rotation along Z axis). We use the position of the screen in the camera image to determine the X and Y value. Using the size of the screen in the camera image determines the Z value. The rotation of the screen in the camera image determines the Angle value along the Z axis. By holding and manipulating a camera to control the cursor on the screen is like first-person mouse. This interface increases the possibility of more applications taking advantage of the extra control capability of the physical input.
We built our prototype with PROCESSING, which is an open source programming language and environment for people who want to program images, animation and sound. We use a blob detection algorithm to figure out where the computer screen is, and thus where the mouse should be. The accuracy of the detection is affected by the contrast of the luminance between screen and background. Therefore, it will get a better result if we turn off other light sources in the room.

Prototype Demo

Demo Video1
Demo Video2

Application

Based on the interface we have built, we can apply the 4 dimensions controls to applications that we can not achieve in a regular mouse. One of the applications we implemented is a first-person shooting game. The player holds the camera as a gun and aims at the targets that randomly show up in simulated 3 dimension space. The targets are in two kinds of color, green and blue. The player has to fire by rotating the camera clockwise for green targets or counterclockwise for blue targets. We use perspective view and the shadow on the ground to simulate the depth on 2-D screen.
The bigger the size of the luminant screen is, the relatively smaller movement of the cursor responding to the camera movement is. If we hook up computer with a projector, then the camera holder has to not only move the camera around but also move his/her body right, left, forward and backward. In terms of interactive media, it is a good experience for users to actually get involved in the application.

Application Demo

This demo takes this pre-recorded video clip as input to control the shooting gun.
Instruction: Unzip the file and execute mousecam_cube.exe. (Java Runtime Environment is needed)

Snapshot

Source Code

Source code in a zip file, including .pde format for PROCESSING environment and converted .java format. Source code


Introduction
Our primary goal of this project is to use a camera as an enhanced mouse that provides 4 dimensions control(X, Y, Z and Rotation along Z axis). We use the position of the screen in the camera image to determine the X and Y value. Using the size of the screen in the camera image determines the Z value. The rotation of the screen in the camera image determines the Angle value along the Z axis. By holding and manipulating a camera to control the cursor on the screen is like first-person mouse. This interface increases the possibility of more applications taking advantage of the extra control capability of the physical input. We built our prototype with PROCESSING, which is an open source programming language and environment for people who want to program images, animation and sound. We use a blob detection algorithm to figure out where the computer screen is, and thus where the mouse should be. The accuracy of the detection is affected by the contrast of the luminance between screen and background. Therefore, it will get a better result if we turn off other light sources in the room.

Prototype Demo
Demo Video1	Demo Video2


Application
Based on the interface we have built, we can apply the 4 dimensions controls to applications that we can not achieve in a regular mouse. One of the applications we implemented is a first-person shooting game. The player holds the camera as a gun and aims at the targets that randomly show up in simulated 3 dimension space. The targets are in two kinds of color, green and blue. The player has to fire by rotating the camera clockwise for green targets or counterclockwise for blue targets. We use perspective view and the shadow on the ground to simulate the depth on 2-D screen. The bigger the size of the luminant screen is, the relatively smaller movement of the cursor responding to the camera movement is. If we hook up computer with a projector, then the camera holder has to not only move the camera around but also move his/her body right, left, forward and backward. In terms of interactive media, it is a good experience for users to actually get involved in the application.


Application Demo
This demo takes this pre-recorded video clip as input to control the shooting gun. Instruction: Unzip the file and execute mousecam_cube.exe. (Java Runtime Environment is needed)

Snapshot



Source Code
Source code in a zip file, including .pde format for PROCESSING environment and converted .java format. Source code