Experiments into 3D Video

Now I don't have any fancy 3D TV or projectors, but I do have some old amber/blue 3d glasses lying around, so I thought I'd have  go at 3D  video on no budget.

I have experimented with 3D images before, using Poser and Photoshop, but this time I would be working with real-world video (not CG) so I needed 2 cameras for my DIY 3D rig. I have 2 cameras. I have a very nice Sony-Z5 for high quality video and I have a cheap little JVC Everio for anything else. The way that the amber/blue 3D glasses work (so I am informed) is the the amber image has most of the colour, while the blue provides the depth. So I decided to use the Sony for the amber (left) and the JVC for the right (blue).
The first task was to patch together a 3D rig to hold both cameras. I didn't have much time, or materials, so I came up with an interesting workaround. I set up the Sony on a tripod as usual, then used a Joby Gorillapod to suspend the JVC, upside down, from the microphone mount on the right of the camera. With a bit of trial and error I got both cameras at about the same level and zoom and started recording.

Once I had my footage I imported both into After Effects as separate compositions, created a parent composition containing the two and began the work of lining them both up. Obviously, the first thing to do was to rotate the JVC footage 180 degrees so that it was the right way up. Once I had done that, I created a solid layer in the parent composition and applied the 3d glasses effect. I will not be using this effect for the final result, but it is useful to set it up to interlace the videos while aligning.
While displaying the interlaced view in one window, I scaled, rotated, resized and distorted the JVC footage, until it was a close match to the Sony footage (as the Sony footage was to contain the details I did the work on the JVC footage).

Once the clips were aligned, I hid the solid layer and tinted the JVC composition so that it appeared in greyscale. Making sure the Sony layer was above the JVC layer, I used the Set Channels effect on the Sony layer, setting the blue channel to be sourced from the JVC footage's lightness.

Et Voilà!
(obviously you will need amber/blue 3D glasses to see the effect)

Lightsaber Effects

Being a Star Wars fan, one of the first thing I looked at doing when I started editing video was to create some lightsaber effects.

I trawled the internet for some examples and tutorials and found this excellent one from videocopilot.net. I tried it out on a simple animation I was working on at the time and was very impressed. Having got it out of my system, I left well alone and got on with other, more serious, bits of work.

While putting together the DVD for the latest show by my drama group, the Appleton Players, I took the opportunity to recycle some footage of a rehearsal of a sword fight and add the lightsaber effects for the amusement of all.

The clip I used was only just over 30 seconds long, but there was no way I was going to manually track both ends of two swords for the 900+ frames, so I decided to make things a little easier with motion tracking and expressions in after effects.

I followed the tutorial to set up the lightsabers and get the look that I liked, then I set about creating 4 trackers, one to track the tip and base of each sword. This was the time consuming bit. While the motion tracking tool in After Effects is OK, when you have fast moving items or light coloured items against a light coloured background, it has some difficulty. These swords were both fast moving and light against light (sigh). So, after many hours of correcting the motion tracking, I had my points plotted. By using expressions similar to the following I set the start and end position of each of the lightsabers to use these tracked points.

comp("mycomp").layer("clip.m2t").motionTracker("Tracker 1")("Track Point 1").attachPoint


With these expressions plugged in, my effects followed the action in the clip as expected. Unfortunately, I had to manually mask out items as they passed in front of the lightsabers, as shown in the tutorial, however I am hopeful that the new Roto Brush in CS5 will automate this too.


So I have my video, but a lightsaber just isn't right without that sound. So I went trawling again. I found a number of samples of lightsabers (and I already have some myself), but I was after a "clean" sound so that I can modify it as required. Eventually I found this site describing how to create you own sounds in a similar way to the way Ben Burtt did for the original film. While this is a bit too technical and long winded for what I was after, there was a download of a premixed lightsaber hum, so I grabbed that instead.


As I was using two lightsabers, I pitched one up slightly (the green one) and both slowed and pitched down the other (red), so that each had a unique sound.


The final part was to adjust the volume of the lightsaber as the sword was moved on screen so that the faster the blade is moved, the louder the buzz.


After some experimentation and research I created the following expression for the audio level of the lightsaber.


// Use the combined speed of both the tip and base of the lightsaber
totalSpeed = thisComp.layer("Green Saber").effect("Saber_Controls")("Start Position").speed + thisComp.layer("Green Saber").effect("Saber_Controls")("End Position").speed;
// Convert the speed to a volume, maximum volume of 2.5dB
vol = Math.min(totalSpeed * 0.05 - 12, 2.5);
// return the volume for left and right channels
[vol,vol]


My short clip, "Darth Mo", was finally complete.

Shooting Theatre (on the cheap... ish)

My purchase of a shiny new video camera at the start of the year happened to co-inside with the start of rehearsals  for this year's production of my amateur dramatics group's show.

As I have worked with our producer to film and produce DVDs of the last few shows, I naturally decided that this would make a good first project to familiarise myself with the equipment and to get a, hopefully, better quality of video this year.

I soon realised that shooting theatre, especially the amateur kind, is very different from shooting for video. When shooting for film you can set up your shots as you want them and shoot in an efficient order. With theatre, you have to work with what you've got. You start at the beginning and continue to film until the end.

I'm learning as I go along, but here is a workflow I am currently using.

Record the Rehearsals
As I am appearing in the show as an actor, I have employed a cameraman to help out (I say employed, there is no money involved, just a willing volunteer). By watching the rehearsals and making notes, the cameraman gets to know the show and knows what should be happening when. Having watched some of the previous footage we have shot, the constant zooming and panning to catch entrances can be nauseating and is caused by the camera operator not being prepared for an event and rushing to catch up.
Review the Recordings
Obvious really, but having recorded the rehearsals take the time to review the footage and note where various camera moves can be made. Also note where close-ups would be nice to have, you may not be able to use them, but it is better to have shots you can't use than to miss those you can.
Take Time Setting Up
It won't be practical, or possible, to change things once the show has started so make sure you take the time to set things up right before it begins. Work with the sound and lighting guys, or gals, to get the everything as it will be during the show. Ideally, do this for the first time during a technical rehearsal so that you have time to iron out any bugs in the system.
Record Close-Ups Beforehand
Take the opportunity to record close-ups and other, non-standard shots, during the dress and technical rehearsals when there is no audience to restrict placement of the camera. This should give you material you can drop into the video during editing to improve the composition.
Record Every Performance
Always plan on recording all performances, these are the only times you will get to record audience reactions. Besides, things can go wrong, and if you only shoot the last show, that is the performance you are stuck with. Remember also, that the more material you shoot, the more you have to work with when editing.

Star Wars: Fade to Black

I have been working for some time now on a new Star Wars RPG campaign named "Fade to Black".

This time around I am hoping to make it a multimedia experience and have been working away on character images, video, software and a number of other items to make the most of today's technology to enhance the role-play. Much of it is still being kept secret from the players, but once the first session is completed on Saturday, I will be looking to provide details on what I did and how well it worked (or not).

Progress at Last!

I have finally made some progress in the editing of video from my camcorder (a JVC Everio)!

It seems that support for MPEG-2 in After Effects is a "known issue" as the official Adobe support recommended I use a different source.

After serveral attempts at conversion with serveral different tools to several different formats, I have now found one that seems to work.

the wonderful ffmpeg is a command line tool that can convert from pretty much any video format to another.
For my camcorder and after effects, I have found that conversion to WMV2 seems to work best.
Anyone trying to reproduce this should try the following:

ffmpeg -i {input file} -vcodec wmv2 -vb 8900000 /
-acodec wmav2 -ab 384000 {output file}

This should keep the video at the same quality (or close enough) as the original.

Adobe MPEG-2 Support

Grr!

After struggling last time to import MPEG-2 files in Premier Pro, this week I have been having issues with poor support for the encoding in After Effects. There is a problem where the rendering suffers from red artifacts, normally filling the entire frame.

This is not a problem with the source as I can play the file in other software without issue, but the red flicker is being introduce in After Effects. After browsing the web for this issue, the concensus seems to be to convert your video files from MPEG-2 into something else before editing them, however, this is a time consuming process and adds the possibility of introducing more errors.

To add insult to injury, whenever I try to report an issue on the Adobe support site, it seems to be down for maintainance (do they do this every Saturday?).

Adobe CS3 Production Premium

Okay, so one of my interests is film making (not that I've made any yet) and to that end I have just purchased a copy of Adobe CS3 Production Premium (actually I bought it over a week ago, but have been so busy I had done nothing but install it before today). Now I'm no expert, in fact most of this is completely new to me, so here is a beginner experience of video editing in CS3.

I had some video files lying around from my holiday last year, so I thought I'd have a go at editing one of those. It was a video of the dolphins at sea world (aww, cute).

One feature that caught my attention when I was reading about the package was the AutoComposer which you can use to create scores for your video. So, first thing I did was open up Bridge, as this is what pulls every thing together, and clicked on the olive coloured Sb box.

This did not, as I had hoped, open Soundbooth, but a page with information about SoundBooth; including, as luck would have it, a link to a tutorial on using AutoComposer. I clicked the link and while the tutorial opened, I opened up Soundbooth.

The tutorial started and I started along with it.
Choose a score it said. I did.
Click on the score and you will hear a preview it said. I didn't, hear a preview that is. I paused the tutorial and started poking around. I discovered a preview panel under the window menu, I heard the previews. I resumed the tutorial.
There is some explanation on various items on the composer panel.
Select the video file as the reference link it says. I pause it again. I have not yet got my video clip in the application, so I load it up and select it as the reference clip. I continue the tutorial.
It explains about intros and outros and how the score works with keyframes.
It explains how to lower the volume of the score while a person is talking. I lower the volume of my score only to find that no-one is talking when they should be. In fact, my video has no sound at all!!
I paused the tutorial and went on a hunt for an explanation.
After trawling through help and googling, I found a forum post that said the sound in MPEG files is not supported as it is mixed in with the video, or something like that.
Hmmm.
Most of my video files are MPEG. In fact my new camcorder, a JVC Everio, records everything in MPEG.

I am really annoyed that CS3 does not support audio in MPEG files, I have used free video editing tools that do this, surely one I have paid so much for can do it!
Come on Adobe get your act together!!

I tried the video file in Premier Pro (the video editing package), no luck there either. So I googled again and found a freeware app to rip sound from MPEG, this worked... sort of.
The audio file was shorter the the original MPEG.
In Premier Pro I could change the duration, by I was not sure it would sync correctly. I decided that would do for now, I'd try out some other features.

I decided to use After Effects to try out stabilizing my video footage (hand held camcorders get a bit wobbly) and this was great! Load up the file, drop it into the composition, right click and select stabilize. Easy.
I turned on every option, selected 2 points then let it rip. I waited for it to process a few second, then stopped it, applied the results and hit playback. When it played back it was devoid of shakes, but even more impressive, it had sound! It seems After Effects can do what the other apps couldn't and can read sound from an MPEG file.

A plan began to form.

I saved my After Effects project and then tried exporting as a Premier Pro project.
I started up Premier Pro. I opened my new project. It didn't work.
The video data could not be found and the was still no audio.
Oh well, back to the drawing board.

At least I have about 3 seconds of video footage of dolphins that doesn't shake.