Experiments into 3D Video

Now I don't have any fancy 3D TV or projectors, but I do have some old amber/blue 3d glasses lying around, so I thought I'd have  go at 3D  video on no budget.

I have experimented with 3D images before, using Poser and Photoshop, but this time I would be working with real-world video (not CG) so I needed 2 cameras for my DIY 3D rig. I have 2 cameras. I have a very nice Sony-Z5 for high quality video and I have a cheap little JVC Everio for anything else. The way that the amber/blue 3D glasses work (so I am informed) is the the amber image has most of the colour, while the blue provides the depth. So I decided to use the Sony for the amber (left) and the JVC for the right (blue).
The first task was to patch together a 3D rig to hold both cameras. I didn't have much time, or materials, so I came up with an interesting workaround. I set up the Sony on a tripod as usual, then used a Joby Gorillapod to suspend the JVC, upside down, from the microphone mount on the right of the camera. With a bit of trial and error I got both cameras at about the same level and zoom and started recording.

Once I had my footage I imported both into After Effects as separate compositions, created a parent composition containing the two and began the work of lining them both up. Obviously, the first thing to do was to rotate the JVC footage 180 degrees so that it was the right way up. Once I had done that, I created a solid layer in the parent composition and applied the 3d glasses effect. I will not be using this effect for the final result, but it is useful to set it up to interlace the videos while aligning.
While displaying the interlaced view in one window, I scaled, rotated, resized and distorted the JVC footage, until it was a close match to the Sony footage (as the Sony footage was to contain the details I did the work on the JVC footage).

Once the clips were aligned, I hid the solid layer and tinted the JVC composition so that it appeared in greyscale. Making sure the Sony layer was above the JVC layer, I used the Set Channels effect on the Sony layer, setting the blue channel to be sourced from the JVC footage's lightness.

Et Voilà!
(obviously you will need amber/blue 3D glasses to see the effect)

First Impressions: Rotobrush

Following the (relatively quick) install of Production Premium CS 5, I imported the project files I used for producing my latest DVD into the new versions of Premier Pro, Encore and After Effects. All of them loaded fine and I saved out some CS5 versions for later use. I then jumped straight into After Effects to try and implement a shot using the new rotobrush I did not have the time and patience to do with manually controlled masks.
At one point in the show, the 2 leads are singing a duet, with a pretty full stage behind them. The problem is that to make things easier for the cameraman, and due to the camera positioning, the depth of field was pretty big. This was OK for most of the show, but in this one shot as the leads sing to one another, one of the background characters is right in the middle of the shot looking out from between them. Now I could have fixed this in CS4 if I had a lot of time. I could have created an adjustment layer and focus blurred the background, leaving the two leads in focus. This would mean manually masking out the two leads for each frame of the shot.
The shot is about 40 seconds long, that would mean manually drawing the mask for 960 frame!
With the release of CS5 the rotobrush was announced, and for me this was the decider as to whether I upgrade or not. I will be using this tool a lot.

The red will be blurred...

So I loaded up the clip, drew a stick figure over the two leads and "hey presto" they had a nice purple outline around them. It wasn't perfect, but it didn't take long to tidy up. I then hit play and watched as it process each frame in turn. When it went wrong, I corrected it then continued to play. After a while, however, it stopped tracking and simply played back, there was only a purple box around the whole screen. After some trial and error, it seemed that it would calculate 9 frames each time I made a change, then stop. In order to progress, I would make a minor change to the rotobrush area every 9 frames, while reading the manual in between.
It was some time before I realised what I was doing wrong. When the brush is used, it creates a base frame. All the changes are tracked from this frame for a limited time, so as to preserver resources. In the rotobrush view, there is a track under the timeline that highlights the base frame and displays the span (the frames that AE will rotoscope for you). By default the span was 9 frames. I simply had to click and drag the end of the span to where my effect would end and it would do it all for me. (The reason I had missed this before was than my source clip was much longer than the bit I was editing, so the track seemed to be showing a single dot when zoomed out.)
Unfortunately, due to the way it works, all changes are calculated from the base frame. Once that is set and the span extended for 40 seconds, when you click on the last frame of the span, all of the frames before that must be calculated for it to display. It still takes a long time to create the mask, but at least now it is my computer doing the work and now me!

Well, at least I know it's working in 64-bit now...

What A Wake Up!

Having watched the election results with fascination for some time last night, I was awoken this morning by a banging on my door and a very nice delivery man handing over the CS5 version of Production Premium.
I sit here now, listening the the fact that we have a hung parliament, still bewildered by the fact that hundreds of voters were turned away from the polling stations, swigging coffee to stay awake and installing new software (only had 5 hours of sleep after following the election results).
The biggest election surprise for me (and many others by the sounds of it) is that the Lib Dems have actually lost more seats than they have gained and that their overall votes have increased by only 1%.
On a local note, Oxford East is looking like a spot of Red in a sea of Blue as Oxford West has now gone from Gold to Blue.
http://news.bbc.co.uk/1/shared/election2010/results/constituency/d47.stm

Lightsaber Effects

Being a Star Wars fan, one of the first thing I looked at doing when I started editing video was to create some lightsaber effects.

I trawled the internet for some examples and tutorials and found this excellent one from videocopilot.net. I tried it out on a simple animation I was working on at the time and was very impressed. Having got it out of my system, I left well alone and got on with other, more serious, bits of work.

While putting together the DVD for the latest show by my drama group, the Appleton Players, I took the opportunity to recycle some footage of a rehearsal of a sword fight and add the lightsaber effects for the amusement of all.

The clip I used was only just over 30 seconds long, but there was no way I was going to manually track both ends of two swords for the 900+ frames, so I decided to make things a little easier with motion tracking and expressions in after effects.

I followed the tutorial to set up the lightsabers and get the look that I liked, then I set about creating 4 trackers, one to track the tip and base of each sword. This was the time consuming bit. While the motion tracking tool in After Effects is OK, when you have fast moving items or light coloured items against a light coloured background, it has some difficulty. These swords were both fast moving and light against light (sigh). So, after many hours of correcting the motion tracking, I had my points plotted. By using expressions similar to the following I set the start and end position of each of the lightsabers to use these tracked points.

comp("mycomp").layer("clip.m2t").motionTracker("Tracker 1")("Track Point 1").attachPoint


With these expressions plugged in, my effects followed the action in the clip as expected. Unfortunately, I had to manually mask out items as they passed in front of the lightsabers, as shown in the tutorial, however I am hopeful that the new Roto Brush in CS5 will automate this too.


So I have my video, but a lightsaber just isn't right without that sound. So I went trawling again. I found a number of samples of lightsabers (and I already have some myself), but I was after a "clean" sound so that I can modify it as required. Eventually I found this site describing how to create you own sounds in a similar way to the way Ben Burtt did for the original film. While this is a bit too technical and long winded for what I was after, there was a download of a premixed lightsaber hum, so I grabbed that instead.


As I was using two lightsabers, I pitched one up slightly (the green one) and both slowed and pitched down the other (red), so that each had a unique sound.


The final part was to adjust the volume of the lightsaber as the sword was moved on screen so that the faster the blade is moved, the louder the buzz.


After some experimentation and research I created the following expression for the audio level of the lightsaber.


// Use the combined speed of both the tip and base of the lightsaber
totalSpeed = thisComp.layer("Green Saber").effect("Saber_Controls")("Start Position").speed + thisComp.layer("Green Saber").effect("Saber_Controls")("End Position").speed;
// Convert the speed to a volume, maximum volume of 2.5dB
vol = Math.min(totalSpeed * 0.05 - 12, 2.5);
// return the volume for left and right channels
[vol,vol]


My short clip, "Darth Mo", was finally complete.

Progress at Last!

I have finally made some progress in the editing of video from my camcorder (a JVC Everio)!

It seems that support for MPEG-2 in After Effects is a "known issue" as the official Adobe support recommended I use a different source.

After serveral attempts at conversion with serveral different tools to several different formats, I have now found one that seems to work.

the wonderful ffmpeg is a command line tool that can convert from pretty much any video format to another.
For my camcorder and after effects, I have found that conversion to WMV2 seems to work best.
Anyone trying to reproduce this should try the following:

ffmpeg -i {input file} -vcodec wmv2 -vb 8900000 /
-acodec wmav2 -ab 384000 {output file}

This should keep the video at the same quality (or close enough) as the original.

Adobe MPEG-2 Support

Grr!

After struggling last time to import MPEG-2 files in Premier Pro, this week I have been having issues with poor support for the encoding in After Effects. There is a problem where the rendering suffers from red artifacts, normally filling the entire frame.

This is not a problem with the source as I can play the file in other software without issue, but the red flicker is being introduce in After Effects. After browsing the web for this issue, the concensus seems to be to convert your video files from MPEG-2 into something else before editing them, however, this is a time consuming process and adds the possibility of introducing more errors.

To add insult to injury, whenever I try to report an issue on the Adobe support site, it seems to be down for maintainance (do they do this every Saturday?).

Adobe CS3 Production Premium

Okay, so one of my interests is film making (not that I've made any yet) and to that end I have just purchased a copy of Adobe CS3 Production Premium (actually I bought it over a week ago, but have been so busy I had done nothing but install it before today). Now I'm no expert, in fact most of this is completely new to me, so here is a beginner experience of video editing in CS3.

I had some video files lying around from my holiday last year, so I thought I'd have a go at editing one of those. It was a video of the dolphins at sea world (aww, cute).

One feature that caught my attention when I was reading about the package was the AutoComposer which you can use to create scores for your video. So, first thing I did was open up Bridge, as this is what pulls every thing together, and clicked on the olive coloured Sb box.

This did not, as I had hoped, open Soundbooth, but a page with information about SoundBooth; including, as luck would have it, a link to a tutorial on using AutoComposer. I clicked the link and while the tutorial opened, I opened up Soundbooth.

The tutorial started and I started along with it.
Choose a score it said. I did.
Click on the score and you will hear a preview it said. I didn't, hear a preview that is. I paused the tutorial and started poking around. I discovered a preview panel under the window menu, I heard the previews. I resumed the tutorial.
There is some explanation on various items on the composer panel.
Select the video file as the reference link it says. I pause it again. I have not yet got my video clip in the application, so I load it up and select it as the reference clip. I continue the tutorial.
It explains about intros and outros and how the score works with keyframes.
It explains how to lower the volume of the score while a person is talking. I lower the volume of my score only to find that no-one is talking when they should be. In fact, my video has no sound at all!!
I paused the tutorial and went on a hunt for an explanation.
After trawling through help and googling, I found a forum post that said the sound in MPEG files is not supported as it is mixed in with the video, or something like that.
Hmmm.
Most of my video files are MPEG. In fact my new camcorder, a JVC Everio, records everything in MPEG.

I am really annoyed that CS3 does not support audio in MPEG files, I have used free video editing tools that do this, surely one I have paid so much for can do it!
Come on Adobe get your act together!!

I tried the video file in Premier Pro (the video editing package), no luck there either. So I googled again and found a freeware app to rip sound from MPEG, this worked... sort of.
The audio file was shorter the the original MPEG.
In Premier Pro I could change the duration, by I was not sure it would sync correctly. I decided that would do for now, I'd try out some other features.

I decided to use After Effects to try out stabilizing my video footage (hand held camcorders get a bit wobbly) and this was great! Load up the file, drop it into the composition, right click and select stabilize. Easy.
I turned on every option, selected 2 points then let it rip. I waited for it to process a few second, then stopped it, applied the results and hit playback. When it played back it was devoid of shakes, but even more impressive, it had sound! It seems After Effects can do what the other apps couldn't and can read sound from an MPEG file.

A plan began to form.

I saved my After Effects project and then tried exporting as a Premier Pro project.
I started up Premier Pro. I opened my new project. It didn't work.
The video data could not be found and the was still no audio.
Oh well, back to the drawing board.

At least I have about 3 seconds of video footage of dolphins that doesn't shake.