Enhancing Action Games with a Webcam



This tool enables us to play action games with body gestures.

First, place a virtual sensor and make a connection between the sensor and an appropriate key, with a GUI provided by the tool. And then it invokes the specified key event when an object intrudes into the sensor.

We intend to force conventional games to adapt to EyeToy-like camera system.





Windows + usual japanese keyboard (the other keyboards may not work)


Install the following Python packages: We do not assure that our tool works with the packages whose versions are different from the ones specified above.

Source Code

How to Start

Place the above two files into the same directory, and start with the command:
$ python image.py
You may also be able to start by simply double-clicking image.py .

How to Use

Creating Sensors

Drag with a left button inside the window to create a sensor, and then our tool immediately enters into the "key-binding mode". During this mode, hit a key which you want to bind the sensor to.

Judgement of Intrusion into Sensors

The tool judges each sensor as "positive" if difference between the initial image of the sensor and the current one is larger than a certain threshold.

The sensor is re-initialized with the current image by "reversal" or "move/resize" of the sensor described below.

Reversing Judgement Policy

Left-clicking on a senser brings about the "reversal mode", in which the sensor is judged as "positive" when there are no intruders into the sensor. One more left-click returns the sensor back to the normal mode.

Moving/Resizing Sensors

Dragging a sensor with a left button moves the sensor. Dragging a corner of a sensor with a left button resizes the sensor.

Changing Key Bindings

Right-click on a sensor.

Removing Sensors

Drag a sensor with a right button.

Starting to Play Games

Check the box "Now Playing:". This operation locks each sensor and begins to raise actual key events.

You have to activate the game window which you want to send key events to.


The tool may not work well with rhythm action games, because these games requires to receive key events as soon as possible, while our tool introduces a little timelag due to image analyses.

On the other hand, you will enjoy action games like Mario because even the timelag boosts thrill. We recommend that the sensor which makes Mario jump should be placed far above your head so that we can appreciate Mario's excellent physical ability.

A reversed sensor can be applied to crouching actions by placing the sensor on the location of your chest when you stand up.


Python webcam fun - motion detection
SATOH Hideaki