I’ve been doing a lot of thinking as to the control method for the audio/visuals. I now have the Orbit controller and have tried some experiments with the audio side of things. It works great, but I can see it being potentially an issue for those people that are not musicians or happy to experiment with button presses. There is also the possibility of people breaking the equipment, which would ruin it for others.
An alternative that I’ve been researching is the use of a device called the ‘Leap Motion’. This is a unit that uses motion control similar to things like the kinect camera, but an awful lot more accurate, mainly used for hands as opposed to full body tracking. A great thing about it is that I can install it into a pedestal and it has no working parts. People should be able to experiment and play around with their hand motion to see what different actions will have on both audio and video. It’ll allow for a lot of experimentation from the viewer and even push them to be more immersive.
Hand position, orientation, movement, and speed are all measured with both hands, so the amount of control offered it really quite vast. Using a program called Geco for Leap Motion, it’s possible to turn your hand movements into midi. This midi can then be sent to both Ableton live and Resolume to control the audio and video, respectively.
Craig Winslow created their interactive installation that has incorporated the Leap Motion;
potential issues with this are tracking from the unit itself, until I try some experiments I have to assume there may be problems with lighting. One thing I noticed about the ‘Growth’ exhibit is the pedestals have a small light projecting upward onto the hand which should help the camera pick up the hand movements.