This is me demonstrating the setup for the installation. using Ableton Live to control the music and trigger the video at the right times. The Leap is controlling effects simultaneously on Resolume to reflect the changes with the video.
The player can then control what happens with the remix, essentially creating music and audio without having any real knowledge of how they did it!
I’ve been doing a lot of thinking as to the control method for the audio/visuals. I now have the Orbit controller and have tried some experiments with the audio side of things. It works great, but I can see it being potentially an issue for those people that are not musicians or happy to experiment with button presses. There is also the possibility of people breaking the equipment, which would ruin it for others.
An alternative that I’ve been researching is the use of a device called the ‘Leap Motion’. This is a unit that uses motion control similar to things like the kinect camera, but an awful lot more accurate, mainly used for hands as opposed to full body tracking. A great thing about it is that I can install it into a pedestal and it has no working parts. People should be able to experiment and play around with their hand motion to see what different actions will have on both audio and video. It’ll allow for a lot of experimentation from the viewer and even push them to be more immersive.
Hand position, orientation, movement, and speed are all measured with both hands, so the amount of control offered it really quite vast. Using a program called Geco for Leap Motion, it’s possible to turn your hand movements into midi. This midi can then be sent to both Ableton live and Resolume to control the audio and video, respectively.
Craig Winslow created their interactive installation that has incorporated the Leap Motion;
potential issues with this are tracking from the unit itself, until I try some experiments I have to assume there may be problems with lighting. One thing I noticed about the ‘Growth’ exhibit is the pedestals have a small light projecting upward onto the hand which should help the camera pick up the hand movements.
One issue with a live remix by people who aren’t musicians is timing, chaos, dissonance and really not knowing what to do.
A couple of things that might be worth taking into account
– if using something like the launchpad, a live metronome playing back should give people a visual indication of timing.
– hierarchies of samples. certain samples played back will cut off other samples.