vid.stab - Was kann es

Goal

Provide a reasonably easy and flexible way to stabilize (deshake) even strongly jiggled video clips with transcode or other great open source video tools.

Demonstration

Example movie: Cycling downhill:

The cycling video from above:
Original vs. deshaked (low quality XViD4 3.5MB):

High quality 1290x1460(XViD4 30MB) Downhill skiing clip:
Original vs. deshaked using default parameters (low quality Theora 6.7MB).

High quality 1290x1460 (XViD4 17MB)

Another snow clip with bad weather (XViD4 4.5MB) (still with version 0.4):

Features

fast detection of subsequent transformations e.g. translation and rotations up to a given extend
low pass filtered smoothing with adjustable horizont
detection algorithms:
- smart and fast multi measurement fields algorithm with contrast selection
- brute force algorithm only for translations
command line specification of many parameters
clipping options: blank (black), keep from previous frames
optional drawing of measurement fields and detected transformations
zooming possible to get rid of jiggling borders (automatic mode)
resulting images are interpolated (different algorithms)
sharpening of the stabilised movie to compensate for interpolation effects due to rotation/zooming
single pass filter for streaming NEW
virtual tripod mode for removing all motion NEW

Requirements / Installation

You need a Linux with transcode >= 1.1.0 installed, that's it! The newest version is independent of transcode, but in transcode 1.1.7 there is a decent version (0.8).

How to install a newer version of the stabiliser plugin?

The easiest way is to download the binary bundle, if you have the right version of transcode and the plattform. (Otherwise you need to compile the sources, then use the CVS or HG version of transcode)

After you downloaded the appropriate binary tar-ball from the download page extract it. The newest versions contain an install script, which you can invoke with ./install.sh. Otherwise copy the .so files into /usr/lib/transcode (as root). Done!

For ffmpeg the installation is a bit more complicated at the moment, you have to compile ffmpeg yourself and make some changes, see libavfilter/README.

Usage

Overview

You have to do two transcode runs for each movie. One to detect the transformations and the second to apply the smoothed transformations.

Learning by Example

Instead of long bla bla I give some examples since it is easy to understand. I use the following place holders which should be replaced with your concrete values:

MOVIE: you orignal movie
STABILE: stabilised output movie
OUTPUTOPTIONS: options for the movie output e.g.-y ffmpeg,null -F mpeg2video or -y xvid

default case:

transcode -J stabilize -i MOVIE -y null,null -o dummy transcode -J transform -i MOVIE OUTPUTOPTIONS -o STABILE

The first line creates a file MOVIE.trf with the detected relative transformations between subsequent frames. The second line uses this file to convert your movie into a stabilized one. The maximal translation is limited to avout 1/12th of the height of the videos and the time smoothing is 10 frames.

single pass case:

transcode -J deshake -i MOVIE OUTPUTOPTIONS -o STABILE

The single pass version has the combined options of the two-pass filters. It produces less good results since it only uses past frames, whereas the two-pass version uses future frames as well.

If the movie is more shaky or has fast camera movement then we can increase the shakiness parameters.

transcode -J stabilize=shakiness=8:show=1,preview -i MOVIE -y null,null -o dummy

The shakiness parameters extends the search for possible transformations. Here an example: Original vs. deshaked above each other (lower quality Theora 14MB):

High quality 1290x1460(DivX 24MB)
The show=1 option allows to see how the detection works. With the preview plugin you can see it online, but here is slow-motion video of a couple of seconds.

Independently we can do more or less smoothing at the second pass, e.g.

transcode -J transform=smoothing=30 -i MOVIE OUTPUTOPTIONS -o STABILE

A long smoothing means that we tranform away more camera movements. The resulting clip has a lower change in camera speed. Here is the result for short/small smoothing (3) and long/large smoothing (30). Both versions for comparison are combined here.

Another aspect is the zooming and the border area. By default the optimal zooming is calculated, which means that 90% of transformations are hidden. This can be switched off by optzoom=0. Additionally a zoom value can be given with zoom=percentage, which is added to the optimal value. By default the border of the transformed frames contains the pixels from previous frames. If instead black pixel should be filled in, we can do the following:

transcode -J transform=crop=1:optzoom=0 -i MOVIE OUTPUTOPTIONS -o STABILE

Here we disabled automatic optimal zooming (optzoom=0): The result (XViD4 4.5MB):

All Parameters

you can also use

tcmodinfo -i stabilize tcmodinfo -i transform

to find out the following information:

Stabilize plugin

[filter_stabilize.so] v0.75 (2010-04-07) extracts relative transformations of
    subsequent frames (used for stabilization together with the
    transform filter in a second pass)
[filter_stabilize.so] Overview:
    Generates a file with relative transform information
     (translation, rotation) about subsequent frames. See also transform.
Options
    'result'      path to the file used to write the transforms
                  (def:inputfile.stab)
    'shakiness'   how shaky is the video and how quick is the camera?
                  1: little (fast) 10: very strong/quick (slow) (def: 4)
    'accuracy'    accuracy of detection process (>=shakiness)
                  1: low (fast) 15: high (slow) (def: 4)
    'stepsize'    stepsize of search process, region around minimum
                  is scanned with 1 pixel resolution (def: 6)
    'algo'        0: brute force (translation only);
                  1: small measurement fields (def)
    'mincontrast' below this contrast a field is discarded (0-1) (def: 0.3)
    'tripod'      virtual tripod mode (if >0): motion is compared to a
                  reference frame (frame # is the value) (def: 0) <--- NEW
    'show'        0: draw nothing (def); 1,2: show fields and transforms
                  in the resulting frames. Consider the 'preview' filter
    'help'        print this help message

Transform plugin

[filter_transform.so] v0.75 (2009-10-31) transforms each frame according to transformations
 given in an input file (e.g. translation, rotate) see also filter stabilize
Overview
    Reads a file with transform information for each frame
     and applies them. See also filter stabilize.
Options
    'input'     path to the file used to read the transforms
                (def: inputfile.stab)
    'smoothing' number of frames*2 + 1 used for lowpass filtering
                used for stabilizing (def: 10)
    'maxshift'  maximal number of pixels to translate image
                (def: -1 no limit)
    'maxangle'  maximal angle in rad to rotate image (def: -1 no limit)
    'crop'      0: keep border (def), 1: black background
    'invert'    1: invert transforms(def: 0)
    'relative'  consider transforms as 0: absolute, 1: relative (def)
    'zoom'      percentage to zoom >0: zoom in, <0 zoom out (def: 0)
    'optzoom'   0: nothing, 1: determine optimal zoom (def)
                i.e. no (or only little) border should be visible.
                Note that the value given at 'zoom' is added to the
                here calculated one
    'interpol'  type of interpolation: 0: no interpolation,
                1: linear (horizontal) (def), 2: bi-linear 3: quadratic
    'sharpen'   amount of sharpening: 0: no sharpening (def: 0.8)
                uses filter unsharp with 5x5 matrix
    'tripod'    virtual tripod mode (=relative=0:smoothing=0) <--- NEW
    'help'      print this help message

How to get two videos in one like above

Assume the videos are valled v1 and v2.

With transcode

transcode -i v1 -y ppm,null -o f1_ transcode -i v2 -y ppm,null -o f2_ for F in f1_*; do echo $F; montage -background "#444444" -geometry +10+5 $F f2${F#f1} f3${F#f1}; done ls -1 f3_* > list RES=`identify -format "%wx%h" \`head -n 1 list\`` transcode -i list -g $RES -V rgb24 -f 30,4 -x imlist,null -y xvid,null -o v1and2.avi rm f1_* f2_* f3_*;

With mplayer/mencoder (and vertical tiling)

mkdir f1; cd f1 mplayer ../v1 -vo jpeg cd ../ mkdir f2; cd f2 mplayer ../v2 -vo jpeg cd ../ mkdir f3 for F in f1/*; do echo $F; montage -background "#444444" -geometry +10+5 -tile 1x2 $F f2/${F#f1/} f3/${F#f1/}; done mencoder mf://f3/* -mf fps=30:type=jpg -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=6400 -oac copy -o v1and2.avi rm -r f1 f2 f3;

Further examples

Example movie (XViD4 3.6MB):