This is me demonstrating the setup for the installation. using Ableton Live to control the music and trigger the video at the right times. The Leap is controlling effects simultaneously on Resolume to reflect the changes with the video.
The player can then control what happens with the remix, essentially creating music and audio without having any real knowledge of how they did it!
My initial plan has been to use Ableton live and Resolume arena together as I wanted to have the viewer actually remix on the fly. Initially looking at it I thought it would be great to trigger video and sections of songs. I attempted to put this together and have found that the Leap Motion only sends through CC messages (a continuous stream of data which has a value between 1-127) rather than piano note data (a single button press if you will, usually used to trigger samples or video clips). There may be ways to get around this, but due to time and ease, I’ve decided to stick to a pre arranged sequence and manipulate the video and audio with effects and plugins.
Attempting to put this all together I’ve encountered some issues. I originally exported my video at 1080p. I wanted it to be as sharp as possible. I had to convert my video into DVX format which is a proprietary format from Resolume and apparently allows it to play better. Previously I have found that a JPG sequence has worked well playing back in Ableton live’s video window. Avoiding H.264 as, although great for streaming, is terrible for programs like Resolume or Ableton. I’ve found though that the 1080p video is a little jerky, so may try it at 720p.
Setting up the controls for both Resolume and Ableton has been tricky as I’ve had to go via Geco, Leap Motion’s Midi controller software. It does work well, but there are so many options open to you, that you don’t know how people are going to use it, especially as they have no proper training and it’s just “wave your hands over this thing”. I’ve considering using a small picture which shows what to do, without stopping people from experimenting. Presently I have various controls mapped to movements, a distortion on the video matching a distortion on the audio. This is proving very tricky though as I’m manipulating several streams of video and audio.
All this said, I think I’ve figured out the problem. the controller would jump channel numbers if I mapped to another effect. I can solve this by mapping to Resolume’s control panel and then matching that to a global effect rather than a specific effect.
My first render went a little wrong. I wanted it to export as a PNG image sequence, due to it’s non lossy nature, but it started as a .MOV PNG sequence, which is the same thing but in a .MOV wrapper. Unfortunately toward the end of the render, 60 hours or so, I noticed a problem with some of the animation not triggering. I clicked further along in the timeline and the whole thing crashed. This meant I lost a good portion of work as I’ve been unable to recover the 9.5 gigs of footage. At this point I remembered the other reason why it’s so useful to render as an image sequence.
Exporting at 4k in H.264 is also a pain in media encoder. You have to stick everything on high or your size is choked to around 2200 pixels. Something to watch out on when using 4K in the future.
OCTANE REFERENCE SETTINGS: http://helloluxx.com/news/octane-render-kernel-settings-reference-library/
When I had my presentation, I was told by the tutors to watch out for it being too floaty feeling. I agree, and thing that the plexus layers will liven things up, but there are fairly good reasons why you don’t do dramatic or jerky camera movements. guidelines for good VR according to Occulus
Here are some of the video loops and sections I’ve created for the backdrop video for ‘Trigger the landslide’. These will be used within the leap motion controlled installation also. Most of them have been made using Sound Keys within After Effects. some may not have sound as they may have only been tracked to a click.
For the last few days I’ve been going crazy attempting to sort out issue after issue. It’s a ll a bit of a blur, so this may not be in any order.
Knowing that I was going to use a render farm and wanting to keep render times down somewhat, I knew that I needed to bake some of my animations. Essentially a few of the plugins I use are mograph type animations, this means that to ensure I have the same animation as I created, once I’ve received my files back from the render, I need to bake them to individual polygons.
I’ve been using Greyscale Gorilla‘s excellent plug-in called ‘Transform’. It breaks apart your models into polygons or chunks, and in nice new inventive ways. I was having issues with baking the ‘poly mode’ animations. GSG has already produced a tutorial for baking ‘chunk mode’, but nothing for poly. After a lot of research I decided that I would get in contact with Greyscale Gorilla themselves and ask how to tackle this seemingly simple but as far as I was concerned, impossible task. Brilliantly, Chris Schmidt sent back a solution for me.
Unhide GSG layers
Make “PolyFXInstance” editable.
Select ALL polygons of this model.
Disconnect (uncheck Preserve Groups)
Add a PointCache tag (Character tags menu)
At this point you can turn off the PolyFX and the Effector!
Part, the second-
Render farms. Ugh. so I know that I am simply not rendering this myself. After some quick calculations I worked out that it would take 3 months of 24/7 rendering to get this thing finished. At least.
Having to produce images at 4X the size of 1080p so that the visuals are high enough resolution and also stretch 360 degrees around you, takes sometime to process. It entails creating an ‘equirectangular’ as explained in this tutorial: Octane 360 in C4D
One of the great things about Octane is it’s use of onboard GPU power. it’s one of the main reasons I use it now, having purchased a GTX970 video card I found that it did speed up rendering a lot. The added viewport window is also brilliant, especially as C4Ds perspective wireframe is slooooooow at times. The biggest issue appears to be support from render farms. OTOY usually don’t give out licenses for render farms as they have aimed their engine at people who want to use their own PC and video cards. It’s totally scalable, which means that multiple cards offer an increase of exactly what that card would be capable of on it’s own. i.e two GTX 970s are twice as effective as one, etc…
This has meant I’ve had to go shop for my render farm requirements. After a LOT of searching I’ve had to settle on a Polish company called ULTRARENDER. to be fair, they’ve been very helpful and even have gone beyond what they needed to do already. The reason Ultrarender is different is because you rent a server filled with high spec cards rather than very fast processors and RAM. The machine I’m using currently has 6 GTX980s and a Tesla, which makes it pretty nippy. It’s not cheap though. I’m spending around £700 for a weeks rental. Hopefully I’ll have enough time to get at least one sequence finished for my presentation. That would be around 1.30mins, which fits in nicely with my 3 min slot.
Mostly so far I’ve been installing programs on the server using a program called TeamViewer. it allows for remote operation of the server and allows me to install everything I would need. I’ve then moved everything from Dropbox where it was stored earlier, to the server to render. The problem is the licenses to use various bits of software have been a total pain.
OTOY allow you to disconnect your program from one computer to use it on another, so no problem (other than the hour wait per deactivation/activation).
Greyscale Gorilla there’s no issue with at all. just install!
Maxon on the other hand don’t allow me to use my student copy of C4D on any other machine (other than the one originally set up with). I can use it, but the resolution is choked at 800×600 and I can’t save PNG sequences (which I need to). So I’m now uploading an older copy I have to try and make that work… continual installing. This sucks.
So filming the space scenes was pretty fun. It went ahead with no errors and seemed to work well.
Set up in my garage/shed I made sure the room was light tight and set up a pane of glass suspended over a black sheet. My camera (Canon 600D) was suspended over the glass facing down.
Then using a collection of ingredients such as hydrogen peroxide, condensed milk, food colourings and compressed air, I started to blob bits around the glass. Using the Hydrogen Peroxide I managed to slowly move the milk and colourings around the surface.
The stars are created by rubbing a paper towel over the glass and showering it in particles.
The light is a small desk lamp set along the glass to capture the ‘starlight’.
Everything was fed directly into a monitor so I could see what was going on better.
The end result has come out really well. I really like some of the results and feel it should work great with the final animation.
Some of the more odd / better music videos I’ve happened across. A lot of these were released within the last 3 years and are a fairly good indication of videos within metal (and industrial and Primus…).
Mastodon – This video sparked a lot of talk because it uses imagery not usually associated with rock and metal, namely twerking.
Devil Wears Prada – Nice use of puppets, also a bit different.
Every time i die – just a big dumb video of a band having fun. kinda cool. not very original.
Behemoth – Lots of dark imagery. More conceptual than full on performance.
Rivers of Nihil – Tech Death metal, a bit of a change from the usual performance video. Although it does still sit in the ‘standard band in wasteland/decrepit house/warehouse’ arena.
Animation this time. Looks like lots of After Effects.
Very typical performance + epic narrative going on here
Slayer – going for a very filmic narrative here. plus the usual performance stuff.
Not a known band at all, but I thought the data mashing was kind of cool.
Marilyn Manson’s comeback. slightly odd, almost pop video, with no musicians really, only writhing women and CG. and Manson.
Sikth – Mikee (the singer) did this video. seems mainly to be after effects, but it’s good to see something a little different.
Clutch – using humour and narrative (as the song does).
Apparently quite a respected video. a lot of the animation is pretty good and obviously a lot of work has gone into it.
Parkway Drive – Actually a very typical video.
Pantera – (1994) considered a bit of a classic performance video. Almost used as a jumping off point for the Martyrdom video.
Slipknot – another classic performance video that sparked millions of ‘play in a house being ripped apart by fans’ style videos.
Really dark and violent imagery suits Dillinger’s style here.
Skinny Puppy – a seminal industrial band. I only like this album though. And this video, although old, is a great example of giving the audience something they didn’t realise they wanted. Goths breakdancing.