two-pass encoding in Cinelerra

So I was creating a video to showcase my cousin's ambient/electroacoustic music and needed the best output possible. I put together the project with scraps of 1080P video from my Canon 5D that I had lying around.

Normally, I would do my edits and then post an MPEG-TS to Vimeo. However, as Florian had called out recently, I found that portions of the final video were slightly underwhelming in quality. Specifically, some timelapses in the video looked grainy. This was because the MPEG2 render parameters I had been using in the above link were giving me less than best quality. So my solution was to use a two pass encode from Cinelerra via the YUV4MPEG stream capability into an MPEG4.

Easier said then done.

There are three steps:
1) export the audio
2) perform the first pass encoding into /dev/null
3) perform the second pass encoding with the audio file into the final file

Export the Audio
As I was having some sync issues, I pushed the audio a little further down the timeline using a negative nudge value.

First Pass Encode
Cinelerra seems to be very sensitive to any garbage in the command line sent to the YUV4MPEG stream. Look at this output:

Render::run 10
trying popen(ffmpeg -threads 8 -y -i - -an -v 1 -vcodec libx264 -aspect 1.7777 -b 9000k -bt 7775k -refs 1 -deblockalpha 0 -deblockbeta 0 -subq 1 -me_range 21 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 10000k -bufsize 2M -cmp 1 -f mp4 -pass 1 /dev/null� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ���)
FFmpeg version 0.6-rpmfusion, Copyright (c) 2000-2010 the FFmpeg developers
  built on Sep 23 2010 18:55:52 with gcc 4.4.4 20100630 (Red Hat 4.4.4-10)
  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --extra-cflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --extra-version=rpmfusion --enable-bzlib --enable-libdc1394 --enable-libdirac --enable-libfaac --enable-nonfree --enable-libfaad --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab --enable-avfilter --enable-avfilter-lavf --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-runtime-cpudetect
  libavutil     50.15. 1 / 50.15. 1
  libavcodec    52.72. 2 / 52.72. 2
  libavformat   52.64. 2 / 52.64. 2
  libavdevice   52. 2. 0 / 52. 2. 0
  libavfilter    1.19. 0 /  1.19. 0
  libswscale     0.11. 0 /  0.11. 0
  libpostproc   51. 2. 0 / 51. 2. 0
[yuv4mpegpipe @ 0xf0a5f0]Estimating duration from bitrate, this may be inaccurate
Input #0, yuv4mpegpipe, from 'pipe:':
  Duration: N/A, bitrate: N/A
    Stream #0.0: Video: rawvideo, yuv420p, 1920x1088, PAR 1:1 DAR 30:17, 30 tbr, 30 tbn, 30 tbc
/dev/null� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ���: File name too long
Received sigpipe
Received sigpipe
Render::run: Session finished.

That is just damn ugly. It looks like some crap got pasted into the command string. Even though I couldn't see anything at the end of the line in the YUV4MPEG stream, I hit the delete key a few times. The method cleaned out whatever invisible control characters got into the command.

My first pass encoding string was this:

ffmpeg -threads 8 -y -i - -an -v 1 -vcodec libx264 -aspect 1.7777 -b 9000k -bt 7775k -refs 1 -deblockalpha 0 -deblockbeta 0 -subq 1 -me_range 21 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 10000k -bufsize 2M -cmp 1 -f mp4 -pass 1 /dev/null

Note that
1) the audio file is not merged at this first pass step.
2) there is no output file in the first pass encode, the output goes directly to /dev/null

Second Pass Encode
The second pass encode string is where I add my audio file and my output. In the second pass encode string below, note the appearance of the audio file path and the % at the end of the YUV4MPEG stream indicating the output file:

ffmpeg -threads 8 -y -i - -i /mnt/videos/projects/rickr/final.mp3 -v 1 -vcodec libx264 -aspect 1.7777 -b 9000k -bt 7775k -refs 1 -deblockalpha 0 -deblockbeta 0 -subq 1 -me_range 21 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 10000k -bufsize 2M -acodec libfaac -ab 256k -ar 44000 -ac 2 -cmp 1 -f mp4 -pass 2 %

Note that I could have done these commands outside of Cinelerra if I had output the video to some intermediate file format, usually Quicktime for Linux with JPEG or MJPEG compression for best results.

Every project I do leads me to find some bit of minutiae that sticks in my brain. This time, the minutiae is an FFMPEG bug wherein the parameter to make adjustments to the audio sync does not work for MPEG4 files:
https://roundup.ffmpeg.org/issue359

Final Result
All-in-all, I think this pain gave some very nice results. Let me know what you think:

A Little Time Away from crazed mule on Vimeo.

Music By Rick Rascati from his album Cinemusica
http://myspace.com/​rickrmusic
Shot with a Canon 5D (90%), other JVC HD10U

Another one bites the dust. Have a good one, folks.
the mule

Reference
http://howto-pages.org/ffmpeg/

best practices to alleviate audio sync problems in Cinelerra

Audio synchronization problems are the bane of video editors everywhere. Here's what I do in Cinelerra to try to prevent them.

The first thing to understand is that playback in Cinelerra is a separate process from rendering (exporting) your project to a final format. So it is a good idea to calibrate the audio-video synchronization of your files when they are played back in the Cinelerra compositor as well as the a/v sync of rendered output. More specifically:
1) You need to calibrate the audio you see being played back in the compositor. Do this by setting Audio Offset in the Preferences -> Playback -> Audio Out so that your playback audio is in sync with the video. I generally use ALSA as my audio driver under Preferences -> Playback. (You might also want to try to use OSS or ESound to see if those audio drivers help your playback sync issues.) For reference, my offset is .1 seconds.

2) You'll need to calibrate the audio and video once it is rendered out of Cinelerra. This doesn't have to be the absolute final format of the file. Just an intermediate file format to determine that audio is not drifting and in sync. You'll do this by setting Nudge on your audio tracks so that your rendered audio and video are in sync. Here's an article on nudge:
http://crazedmuleproductions.blogspot.com/2007/05/nudge-avidemux2-and-reminder-about.html

Let me give you an example.

My projects usually run about an hour on the timeline. When I start a project, I will concatenate a bunch of clips or raw videos on the timeline. I will then render a short segment of video, about 15 seconds long, from about 45 minutes into the timeline. The segment I take should have key features like someone speaking or someone hitting a drum that I can use to determine audio synchronization. I then render the file to a format that works well in the various Linux media players:
-Quicktime for Linux using MPEG4 video compression and MP4 or twos complement (pcm) audio compression
-Quicktime for Linux container using JPEG video compression and MP4 or twos complement

The side benefit of using MPEG4 video compression is that it renders damn fast.

The good thing about these steps is that if you work with the same format of video over and over, you won't need to change #1, the Audio Offset.

Here are notes regarding specific source formats and output formats.

MPEG SOURCE FILES to MPEG RENDERED OUTPUT
After much study and pain, I've found that when working with MPEG-TS HDV files as my source (and MPEG formats in general), the best pipe to render from Cinelerra in order to alleviate sync headaches is mpeg2enc. I'm not five channel yet, so I just export my 48.1Khz stereo audio as MPEG Layer II audio MP3s @ 384kbps. I then use mplex to combine my audio and video streams into a program stream. Further on, I use VLC to convert that PS into MPEG-TS (HDV) and then use FFMPEG to convert the HDV into DVD and iPod/iPhone compatible formats.

QUICKTIME (QT) SOURCE FILES to QT RENDERED OUTPUT
If my source video format is Quicktime for Linux (like from a screen capture), then I have no sync problems rendering out to QT for Linux.

MPEG SOURCE FILES to QT RENDERED OUTPUT
If I render my MPEG-TS, HDV projects using Quicktime for Linux as an output format (MPEG4 or JPEG video w/twos complement or MP4 audio compression), then I will get sync problems in the rendered output. I then follow the above best practices to alleviate the problem.

TEST YOUR OUTPUT
In Fedora, I've found that the best media players to test final output are mplayer, vlc and ffmpeg.

Finally, there are options to mplex and ffmpeg to offset audio and video sync problems:
mplex -O ; eg, mplex -O 350
ffmpeg -itsoffset ; eg, ffmpeg -itsoffset 0.35

Hope this helps,
the mule

Reference
My Article on Cinelerra's "nudge" parameter
FFMPEG HowTo
http://man-wiki.net/index.php/1:mplex

audio sync, nudge, avidemux2 and a reminder about archiving

I spent most of the night trying to debug an audio synchronization problem in Cinelerra. The video was trailing the audio by a little bit. So I used the Nudge feature to slide the audio .5 seconds behind to the right.

A word about nudge:
If the audio is running behind the video, use POSITIVE values for nudge.
If the video is running behind the audio, use NEGATIVE values for nudge.

Another way to put it:
If your audio is lagging and is behind the video (more to the RIGHT on the timeline than the video), push it FORWARD using positive values.
If your audio is running too fast and is ahead of the video (more to the LEFT on the timeline than the video), push it BACKWARD using negative values.

2/17/2008
Since nudge is good for fixing an entire track, one way to fix synchronization problems that occur on only part of a track is to use the Delay Video video effect or the Delay Audio audio effect:

• The Delay Video track works when audio lags behind the video.
• The Delay Audio track works when the video lags behind the audio.

The two effects take positive values for the delay in seconds. So they only work "one way", so to speak.
:)

This syncronization problem was after another discovery that Cinelerra on the Fedora Core 6 box I just built is not letting me drag and drop video. So I couldn't manually slide my video around. I'm not sure why this is not working for this particular project, as the Record button is enabled and my editing Preferences seem correct. The funny thing is that it does not happen when I create a new project. My gut feeling is that this version of Cinelerra I am using is SLIGHTLY newer than the one on my main box. So the XML is probably screwed up in some way. Ugh. Tiring.

But not tired enough to write a reminder to everybody that if they want the MOST PORTABLE video files, keep them below 4GB. That way they will fit on a DVD for archiving or transferring, fit on a FAT32 file system and be welcomed on any modern file system. Any larger, and it becomes a nightmare to transfer or archive. As I edit in HDV, file sizes get huge pretty quickly. But keep 'em below 4GB kids! It'll save you much pain in the long run.

Your friend in the battle to trim files down to size is AVIDEMUX2, a wonderful little program I've mentioned before that chops up large video files in no time flat. On my system, a Dell 400SC, 3.2Ghz, 2GB RAM, multiple drives, it took about four minutes total to chop up a 5GB HDV file into two 2.5 chunks. Not bad! One thing I do is to make sure that when you are saving out your file, use Copy for both the Audio and Video tracks and then save to your final format. The Copy feature will save out a new file to the exact same specs as your source file. Very nice feature!

audio/video sync problems..first complete editing job completed

After struggling with video/audio synchronization issues, I've finally got 56 minutes of passable video. Due to my own stupidity (the fact that my Audio-Technica AT822 stereo mic was attached to, but not plugged in to the video cam) and that the audio coming from the cam is pretty lousy, I needed to use my backup recording device. So I had to sync the audio from my Creative Labs Nomad. Cinelerra is good with audio, but I had to resample to 48000hz to match the audio sampling rate of the original video.

I first tried to match the Nomad audio with the video, but that became an exercise in frustration. I still had the camera audio, so matching the Nomad audio to the cam's audio was MUCH easier than matching up to the video. As well, this method allowed me to play all four audio tracks (2 stereo from Nomad/2 stereo from cam) back at the same time. Monitoring this, if I heard bouncback echo, I knew my synchronization was off. So rule to remember, if you have a couple sources of audio, match to the audio, not what's on the screen, cause you'll be pulling out all your hair trying to match to the vid.

Glad this one is in the bag. It is truly my first complete video created in Cinelerra. Nothing special within, just fades/titles and a little time clock at the end. It was reduced from the original HDV to 320x180, 160kbps sound (247MB). Also, the video quality pretty much sucks because I didn't whitebalance or even focus for that matter. It was inevitable, as I was rushing to get setup for the three measly hours we had in the rehearsal space. Ah well.
http://content.serveftp.net/video/20060406.mpg

Update 2/25/08
Here is a helpful article for anyone needing to troubleshoot audio/video sync problems in Cinelerra:
/2007/05/nudge-avidemux2-and-reminder-about.html