Some more thoughts

Here’s Linking Park’s video where they use similar technology. In the Khaidian video of Martyrdom, it looks slightly block because I’ve been forced to only do full body shots of the band (due to the 360 degree view),  on a low resolution camera. The Kinect v.1 is low res compared to the newer Kinect for Xbox one, which would have been great and greatly improved the clarity of image, but the limitations of RGBDToolkit mean I’m limited to the first iteration.


Interestingly, I may end up using the beta for ‘Depthkit’, which uses the Kinect v.2 which would be really interesting to use.

Leap of Faith

This is me demonstrating the setup for the installation. using Ableton Live to control the music and trigger the video at the right times. The Leap is controlling effects simultaneously on Resolume to reflect the changes with the video.

The player can then control what happens with the remix, essentially creating music and audio without having any real knowledge of how they did it!

Resolving Resolume’s reactiveness

My initial plan has been to use Ableton live and Resolume arena together as I wanted to have the viewer actually remix on the fly. Initially looking at it I thought it would be great to trigger video and sections of songs. I attempted to put this together and have found that the Leap Motion only sends through CC messages (a continuous stream of data which has a value between 1-127) rather than piano note data (a single button press if you will, usually used to trigger samples or video clips). There may be ways to get around this, but due to time and ease, I’ve decided to stick to a pre arranged sequence and manipulate the video and audio with effects and plugins.

Attempting to put this all together I’ve encountered some issues. I originally exported my video at 1080p. I wanted it to be as sharp as possible. I had to convert my video into DVX format which is a proprietary format from Resolume and apparently allows it to play better. Previously I have found that a JPG sequence has worked well playing back in Ableton live’s video window. Avoiding H.264 as, although great for streaming, is terrible for programs like Resolume or Ableton. I’ve found though that the 1080p video is a little jerky, so may try it at 720p.

Setting up the controls for both Resolume and Ableton has been tricky as I’ve had to go via Geco, Leap Motion’s Midi controller software. It does work well, but there are so many options open to you, that you don’t know how people are going to use it, especially as they have no proper training and it’s just “wave your hands over this thing”. I’ve considering using a small picture which shows what to do, without stopping people from experimenting. Presently I have various controls mapped to movements, a distortion on the video matching a distortion on the audio. This is proving very tricky though as I’m manipulating several streams of video and audio.

All this said, I think I’ve figured out the problem. the controller would jump channel numbers if I mapped to another effect. I can solve this by mapping to Resolume’s control panel and then matching that to a global effect rather than a specific effect.

Battling technology.

My first render went a little wrong. I wanted it to export as a PNG image sequence, due to it’s non lossy nature, but it started as a .MOV PNG sequence, which is the same thing but in a .MOV wrapper. Unfortunately toward the end of the render, 60 hours or so, I noticed a problem with some of the animation not triggering. I clicked further along in the timeline and the whole thing crashed. This meant I lost a good portion of work as I’ve been unable to recover the 9.5 gigs of footage. At this point I remembered the other reason why it’s so useful to render as an image sequence.

Exporting at 4k in H.264 is also a pain in media encoder. You have to stick everything on high or your size is choked to around 2200 pixels. Something to watch out on when using 4K in the future.

OCTANE REFERENCE SETTINGS: http://helloluxx.com/news/octane-render-kernel-settings-reference-library/

When I had my presentation, I was told by the tutors to watch out for it being too floaty feeling. I agree, and thing that the plexus layers will liven things up, but there are fairly good reasons why you don’t do dramatic or jerky camera movements. guidelines for good VR according to Occulus





And on it goes

For the last few days I’ve been going crazy attempting to sort out issue after issue. It’s a ll a bit of a blur, so this may not be in any order.13239011_10156919808790440_5171223907330343128_n

Knowing that I was going to use a render farm and wanting to keep render times down somewhat, I knew that I needed to bake some of my animations. Essentially a few of the plugins I use are mograph type animations, this means that to ensure I have the same animation as I created, once I’ve received my files back from the render, I need to bake them to individual polygons.

I’ve been using Greyscale Gorilla‘s excellent plug-in called ‘Transform’. It breaks apart your models into polygons or chunks, and in nice new inventive ways. I was having issues with baking the ‘poly mode’ animations. GSG has already produced a tutorial for baking ‘chunk mode’, but nothing for poly. After a lot of research I decided that I would get in contact with Greyscale Gorilla themselves and ask how to tackle this seemingly simple but as far as I was concerned, impossible task. Brilliantly, Chris Schmidt sent back a solution for me.

Unhide GSG layers
Make “PolyFXInstance” editable.
Select ALL polygons of this model.
Disconnect (uncheck Preserve Groups)
Add a PointCache tag (Character tags menu)
Store State
At this point you can turn off the PolyFX and the Effector!

Great stuff!

Part, the second-

Render farms. Ugh. so I know that I am simply not rendering this myself. After some quick calculations I worked out that it would take 3 months of 24/7 rendering to get this thing finished. At least.

Having to produce images at 4X the size of 1080p so that the visuals are high enough resolution and also stretch 360 degrees around you, takes sometime to process. It entails creating an ‘equirectangular’ as explained in this tutorial: Octane 360 in C4D

One of the great things about Octane is it’s use of onboard GPU power. it’s one of the main reasons I use it now, having purchased a GTX970 video card I found that it did speed up rendering a lot. The added viewport window is also brilliant, especially as C4Ds perspective wireframe is slooooooow at times. The biggest issue appears to be support from render farms. OTOY usually don’t give out licenses for render farms as they have aimed their engine at people who want to use their own PC and video cards. It’s totally scalable, which means that multiple cards offer an increase of exactly what that card would be capable of on it’s own. i.e two GTX 970s are twice as effective as one, etc…

This has meant I’ve had to go shop for my render farm requirements. After a LOT of searching I’ve had to settle on a Polish company called ULTRARENDER. to be fair, they’ve been very helpful and even have gone beyond what they needed to do already. The reason Ultrarender is different is because you rent a server filled with high spec cards rather than very fast processors and RAM. The machine I’m using currently has 6 GTX980s and a Tesla, which makes it pretty nippy. It’s not cheap though. I’m spending around £700 for a weeks rental. Hopefully I’ll have enough time to get at least one sequence finished for my presentation. That would be around 1.30mins, which fits in nicely with my 3 min slot.

Mostly so far I’ve been installing programs on the server using a program called TeamViewer. it allows for remote operation of the server and allows me to install everything I would need. I’ve then moved everything from Dropbox where it was stored earlier, to the server to render. The problem is the licenses to use various bits of software have been a total pain.

OTOY allow you to disconnect your program from one computer to use it on another, so no problem (other than the hour wait per deactivation/activation).

Greyscale Gorilla there’s no issue with at all. just install!

Maxon on the other hand don’t allow me to use my student copy of C4D on any other machine (other than the one originally set up with). I can use it, but the resolution is choked at 800×600 and I can’t save PNG sequences (which I need to). So I’m now uploading an older copy I have to try and make that work… continual installing. This sucks.

test 001test 002

The second from last frontier

So filming the space scenes was pretty fun. It went ahead with no errors and seemed to work well.

Set up in my garage/shed I made sure the room was light tight and set up a pane of glass suspended over a black sheet. My camera (Canon 600D) was suspended over the glass facing down.



Then using a collection of ingredients such as hydrogen peroxide, condensed milk, food colourings and compressed air, I started to blob bits around the glass. Using the Hydrogen Peroxide I managed to slowly move the milk and colourings around the surface.

The stars are created by rubbing a paper towel over the glass and showering it in particles.

The light is a small desk lamp set along the glass to capture the ‘starlight’.


Everything was fed directly into a monitor so I could see what was going on better.


The end result has come out really well. I really like some of the results and feel it should work great with the final animation.