converting 1080p video, part II

A couple nights back, I spent a few hours finding a usable intermediate format for 1080P video editing. Of course, my goal was to prove that I can still use my preferred NLE Cinelerra in order to edit the gorgeous, high definition output from the Canon EOS 5D Mark II. An alternative way of putting it is that this is my justification for purchasing what will amount to a $4000 camera!

Import: which format works well?
Since Cinelerra is rather picky about the types of video it accepts (see my original Quicktime compatability document), it was a challenge to find a Cinelerra-editable intermediate format for that test 1080p video I told you about in the first part of this two-part blog entry. The video is a five-second city scene of a bicycler launching from a stop. That video is 23MB.

Keep in mind that various versions of media encoders (like ffmpeg and mplayer) do not always implement codecs and container formats consistently. Also, program dependencies for the encoders across different Linux distros complicate matters. So what I state as working below may not work for you. (Your mileage may vary). Note that I am currently using Fedora 7, 64-bit.

Caveats aside, transcoding the biker video to the following formats did NOT work:
- I420 from ffmpeg
- rawvideo (RAW YV12) from ffmpeg

Here are a couple formats that did work:
1) MJPEG output from ffmpeg
2) RAW YV12 output from mplayer

MJPEG from FFmpeg
Playback Performance
This format seems to be a likely candidate, as I got about 15fps on my dual, quad core Dell SC1430 in Cinelerra.

File Size
The resulting file size is small..about 35MB for five seconds of 1080P.

Image Quality Compared to Original
The image quality is very good. However, the colors seem bit oversaturated and dark in contrast to the original. With a bit of tweaking to the conversion command, I might be able to get more realistic colors out of the conversion. Here's a comparison shot:


Editing Notes
No problems surfaced with a couple initial edits and renderings. As this is a compressed format, there will be some loss of quality over successive encodings.

Here is the command I used to convert the video:
ffmpeg -i Video1.MOV -b 3000k -vcodec mjpeg -ab 256k -ar 44100 -acodec mpeg4aac -coder 1 -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -me hex -subq 5 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 testmjpeg.mov

This is the same conversion command that I ran in part I of this blog entry.

Information about the file looks like this in mplayer:
Playing testmjpeg.mov. ISO: File Type Major Brand: Original QuickTime Quicktime/MOV file format detected. [mov] Video stream found, -vid 0 VIDEO: [jpeg] 1920x1080 24bpp 30.000 fps 0.0 kbps ( 0.0 kbyte/s) ========================================================================== Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family Selected video codec: [ffmjpeg] vfm: ffmpeg (FFmpeg MJPEG decoder) ==========================================================================


RAWYV12 from mplayer
Playback Performance
Playback is slow, 7fps, roughly 1/2 the speed of MJPEG. Not a show stopper.

File Size
Huge. About 450MB for about five seconds of video. Woah!

Image Quality Compared to Original
Image quality is very good as well. Less saturated then the mjpeg version. That's what almost 100MB/sec pays for.


Editing Notes
This is an uncompressed format, corresponding to the YUV 4:2:0 spec:
http://en.wikipedia.org/wiki/YUV#Y.27UV420p_.28and_Y.27V12.29

Also, here is an excellent article on understanding color sampling:
http://www.dvxuser.com/articles/colorspace/

Cinelerra loads the raw YV12 file, but the timeline does not show thumbnails. Not a show stopper, but a bit of a bummer. The one benefit of yuv4mpeg stream from mplayer is that you can feed that stream directly into mjpegtools for further processing.

Update 2008/11/22
It seems that 1080p in raw yuv format is actually unusable in my Fedora 9 Cinelerra install. When I load a project with a large raw yuv file, Cinelerra hangs for ten minutes as it loads the file. Once loaded, the GUI tends to freeze for another ten minutes. Thus, it seems that MJPEG may be my only choice for rendering 1080p.

I will continue to experiment.
end update

Here is the command I used to convert the video:
mplayer Video1.MOV -vo yuv4mpeg

I received this message while trying to convert the video
************************************************
**** Your system is too SLOW to play this! ****
************************************************
Possible reasons, problems, workarounds:
- Most common: broken/buggy _audio_ driver
- Try -ao sdl or use the OSS emulation of ALSA.
- Experiment with different values for -autosync, 30 is a good start.

So, I amended my command to add the autosync feature, which gradually adjusts the audio/video synchronization based on audio delay:
mplayer Video1.MOV -autosync 30 -vo yuv4mpeg

That seemed to alleviate the messages. Without redirecting to a specified filename, Mplayer outputs a file called "stream.yuv".

Information about the file looks like this in mplayer:
Playing stream.yuv. YUV4MPEG2 file format detected. YUV4MPEG2 Video stream 0 size: display: 1920x1080, codec: 1920x1080 VIDEO: [YV12] 1920x1080 12bpp 30.000 fps 0.0 kbps ( 0.0 kbyte/s) ========================================================================== Opening video decoder: [raw] RAW Uncompressed Video VDec: vo config request - 1920 x 1080 (preferred colorspace: Planar YV12) VDec: using Planar YV12 as output csp (no 0) Movie-Aspect is 1.78:1 - prescaling to correct movie aspect. VO: [xv] 1920x1080 => 1920x1080 Planar YV12 [ASPECT] Warning: No suitable new res found! [ASPECT] Warning: No suitable new res found! Selected video codec: [rawyv12] vfm: raw (RAW YV12) ==========================================================================

Quality
There seem to be quite a difference in the colors rendered by each format. The MJPEG-formatted file seems to much closer to the original than the raw YV12.

Applying Effects

For fun, I applied an oil painting effect to this five seconds of video. I then rendered the output to MJPEG format. I was disappointed to find out that five seconds of video with the oil paint effect applied takes a full three minutes and twenty seconds to render at 1080p in my dual, quad core Dell SC1430. Holy cow! I guess I won't be doing Star Wars green screens in 1080p!


In the interest of full disclosure, the oil paint effect is a notorious CPU hog and is not indicative of the type of work the average editor will perform. Another note: while rendering the output, all eight CPUs were maxed out at 95% utilization. Kowabunga!

To give people hope, applying a simple title to the image doesn't do too much damage to load on the CPU. The video with a title rendered to MJPEG format took twenty seconds.

Thoughts
My goal is to get some really fantastic looking video rendered from Cinelerra. In combination with the new Canon and some wise encoding choices, I believe I should be able to do this. This article shows that I'm getting closer, but I'm not there yet. I'm hoping to get closer with another round of experimentation.

One final thing..be ready to spend money on RAM. I noticed my paltry 2GB was immediately sucked up by Cinelerra when I was doing these various test videos. I'm going to get another 4GB and will let you know if that helps.

more to come.
The Mule

converting 1080p video, part I

I was interested in buying the new Canon EOS 5D Mark II, a really sweet bit of kit:
Amazon: Canon EOS 5D Mark II
Canon Product Page
Canon Learning Center
Rob Galbraith Review
Manual

I was excited at the possibility of stellar high quality video at a somewhat reasonable price. See Vincent Laforet's great demo video here:
http://blog.vincentlaforet.com/2008/10/10/without-further-ado-reverie/

So I downloaded test video output from a review of the preproduction cam here:
http://www.dpreview.com/previews/canoneos5dmarkII/page15.asp

Disclaimer
DPReview stated that these videos are direct from the cam, though according to Canon, the production model uses MPEG4 video compression. This does not jibe with my results, as I found by reviewing the output of mplayer and Quicktime on my MacBook that these test videos actually use H264 video compression. Strange. Of course, we'll see what the real deal is once the camera is available for purchase. Regardless, my testing procedures below maybe helpful for people using Linux and/or Cinelerra that need to work around similar problems.

Update 10/25/2008
I downloaded a couple longer clips from the review here:
http://www.dpreview.com/news/0809/08091705canon_5dmarkII.asp

My results were consistent with the original post. Interestingly, it took six minutes to convert one minute of 1080P video on my dual, quad core Dell SC1430. Oh Momma, 1080P is a resource hog!
End Update

Linux Media Player Results
Not so good. I was able to replay the video in mplayer fine, but vlc and avidemux2 hung half way, and xine played, though very choppily. Most of the above showed output errors of various kinds.

Next, I thought I'd see if its 1080P video output was usable at all in Cinelerra.

Cinelerra Results
Putting the original 1080P video into Cinelerra gave me a bunch of ugly errors, like so:
[h264 @ 0x2aaaaaf7e1b0]concealing 8160 DC, 8160 AC, 8160 MV errors [h264 @ 0x2aaaaaf7e1b0]AVC: Consumed only 161417 bytes instead of 161420 [h264 @ 0x2aaaaaf7e1b0]Unknown NAL code: 22

And the Cinelerra timeline and compositor windows weren't very promising either:


Convert!
So, I did what all good Cinelerra users do:
Convert the video to a usable format!

This command seems to produce very good output from a 1080P, H264 encoded video:
ffmpeg -i Video1.MOV -b 3000k -vcodec h264 -ab 256k -ar 44100 -acodec mpeg4aac -coder 1 -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -me hex -subq 5 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 test.mp4

Let's take the command options apart:
-i Video1.MOV: my input file
-b 3000k: bitrate of 3000K
-vcodec h264: use the h264 output video compression algorithm
-ab 256k: use an audio bitrate of 256kbps
-ar 44100: use an audio sampling rate of 44.1Khz
-acodec mpeg4aac: use the Apple Audio Compression codec

Next set of options related specifically to x264
-coder 1: enables CABAC entropy encoder used by x264 (better compression)
-flags +loop: enables loop filter
-cmp +chroma
-partitions +parti4x4+partp8x8+partb8x8: improves quality by looking at inter/intra partitions (i b p)
-me hex: set motion estimation method to hex (range-2 search of six surrounding points)
-me_range 16: max range of motion search (hex is limited to 16), higher useful on HD
-subq 5: good medium for higher speed encoding
-g 250: group of pictures size
-keyint_min 25: set minimum length between IDR frames (keyframes)
-sc_threshold 40: adjusts sensitivity of x264's scenecut detection
-i_qfactor 0.71: qscale (quality) difference between I-frames and P-frames

Finally, the output:
test.mp4: my output file

I don't know much about those switches. Here are two good pages explaining at least the words, if not their meaning, of these cryptic options:
FFmpeg x264 mapping
x264 (H264) options

Update 10/25/2008
However, I did see a number of these errors again while doing the conversion:
[h264 @ 0x3c2d71ab30]AVC: Consumed only 197265 bytes instead of 197268ts/s
[h264 @ 0x3c2d71ab30]AVC: nal size 0

Searching the web, it seems that they may have to do with an old version of ffmpeg:
http://lists.mplayerhq.hu/pipermail/ffmpeg-user/2007-October/012075.html

I am currently running the following: =============================================================================
Package Arch Version Repository Size
=============================================================================
Removing:
ffmpeg x86_64 0.4.9-0.37.20070503.lvn7 installed 527 k
ffmpeg-libs x86_64 0.4.9-0.37.20070503.lvn7 installed 3.8 M

And it's going to be a pain to compile with external ffmpeg libraries. I think I will hold off on that for now.
End Update

The original (Video1.MOV) and converted (test.mp4) files were marketedly different sizes:
-rw-r--r-- 1 root root 4214234 2008-10-23 23:15 test.mp4 -rw-r--r-- 1 root root 23690277 2008-10-23 22:55 Video1.MOV

As read by mplayer with -identify option, diffing the original and converted file's properties showed no differences. Here is what mplayer thinks both files :
[mov] Video stream found, -vid 0
VIDEO: [avc1] 1920x1080 24bpp 30.000 fps 0.0 kbps ( 0.0 kbyte/s)
ID_DEMUXER=mov
ID_VIDEO_FORMAT=avc1
ID_VIDEO_BITRATE=0
ID_VIDEO_WIDTH=1920
ID_VIDEO_HEIGHT=1080
ID_VIDEO_FPS=30.000
ID_VIDEO_ASPECT=0.0000
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
ID_VIDEO_CODEC=ffh264
Audio: no sound
Starting playback...
VDec: vo config request - 1920 x 1080 (preferred colorspace: Planar YV12)
VDec: using Planar YV12 as output csp (no 0)
Movie-Aspect is undefined - no prescaling applied.
VO: [xv] 1920x1080 => 1920x1080 Planar YV12

This was odd, as mplayer spat out errors like this while playing the original:
[h264 @ 0xcfa780]AVC: Consumed only 197265 bytes instead of 197268 [h264 @ 0xcfa780]AVC: nal size 0

Once the file was converted, mplayer showed none of these errors.

Linux Media Player Results
I'm happy to say that the converted file replayed well in all the Linux media players, xine, mplayer and vlc, with the notable exception that playback in avidemux2 was slow.

In mplayer, the converted file did not show any errors, outside of a minor ASPECT warning error:
[ASPECT] Warning: No suitable new res found!

From a still, you can't really tell much of a difference..at least I couldn't. Note that the below images have been cropped for easier comparison, so they are not the full 1080P (1920x1280) resolution.

Original


Converted


The big test was bringing the converted file into Cinelerra. Since the OS level media players didn't have a problem, I was hopeful that Cinelerra wouldn't have any issues. Loading the file, I received no errors. However, playback was horrendous. I'd say that I got about 1fps in the compositor, though Preferences (Shift-P) showed that I was getting 32fps. Hmmm. Something definetly wrong here.

Final shot before I go to sleep
While in Cinelerra, I rendered the already converted 1080P content into one of the formats Cinelerra is happiest using, Quicktime using JPEG Photo compression. Rendering that video took no time at all (there was no audio track). Reloading the video back into Cinelerra, the file play more smoothly than the original conversion, but still had problems. I got 6fps. CPU on my dual, quad core was about 20% across all CPUs during playback (see below image). But memory use went through the roof! Swap was up too:



Woah! What's up with that? Well, looks like I have more testing to do. The possibility of gorgeous 1080P HD video has me slavering. Perhaps my eyes are too big for my stomach? As in everything Linux, this will take time.

to be continued..
the mule

Reference
http://bbs.archlinux.org/viewtopic.php?id=35836
thanks Skottish!

iTunes/iPod video workflow, scripted

I am going to revisit my conversation about workflow. Here's what we briefly discussed:
- the transformation of an idea (video) into reality and distributing it (in this case, using iTunes)

With today's internet, the steps involved are broad-based:
- idea creation, storyboard, distribution, production, archiving, marketing

Update 2008/11/22
I've added a few more details to my workflow in a new post here.
end update

I shall give you an example and how I reduced the amount of time spent creating and distributing my content.

When not working, my world is playing music with a band of itinerant musicians called the StormPigs. We gather together once every couple of months to play freeform music. No prior thought involved, just play. The joy of this is being together and having a good time. Otherwise, we are all busy professionals with full-time jobs and families. In remembrance of that good time, I produce videos of the event, distributed via iTunes and YouTube.

As time seems compressed these days, I want to spend as little time as possible behind the keyboard (though I am a technologist by trade). And as readers of this blog know, I am avid proponent of Linux. The beauty of Linux is that the system is completely configurable and flexible. But with this power comes a price. You have to invest the time to learn the shell, some bits of scripting and other Linux arcana. Consequently, it is daunting to the newcomer. But the benefits return to you many times over, as time you once spent on minutiae can now be spent thinking of new ideas for shows and creating new content, rather than simply focusing on the details of getting a file up to the server or copy and pasting content from one application to the other.

I am not a programmer of fanciful GUI front ends. I am a guy who just needs to get work done. So I try to solve my problems in the simplest way possible using the Linux programs and bash shell scripts to tie multiple programs together.

Here is the latest problem I needed to solve:
- how to I get my videos onto the web and in an iTunes ready form as quickly as possible?

In that light, I have come up with some scripts to help me on my way:
- the first encodes my editing video project into various formats (HDV, DVD, podcast)
- the second creates the list of songs from the video that will go into the podcast
- the third merges that songlist information into the podcast
- the fourth creates a new iTunes RSS feed (XML file) from the songlist information
- the fifth uploads the new podcast to my webserver
- the last is a wrapper that starts the other five

The encoding script is most useful and is the basis for the others. This script takes 720P video and 48Khz MPEG, Layer II stereo audio output rendered from a Cinelerra project and converts it to various formats: MPEG2 Program Stream, HDV, DVD and iTunes/iPod compatible formats. You will need the following programs installed for this script to work:
-mplex
-vlc
-ffmpeg

I've found the quality of each output file to be very good to excellent.

You can wrap some validation and input around this script, but the guts of the script look like this, where the arguments enclosed in curly braces will be replaced by the names of your input and output files:
#!/bin/bash
echo "Input: M2A audio and MPEG2 720P video streams"
echo "Output: program stream"
mplex -f 3 -b 2000 ${AUDIO} ${VIDEO} -o ${PS}

echo "Input: program stream"
echo "Output: MPEG2-TS, HDV"
vlc ${PS} --sout '#duplicate{dst=std{access=file,mux=ts,dst="'${HDV}'"}}' vlc:quit

echo "Input: MPEG2-TS, HDV"
echo "Output: MPEG2 DVD"
ffmpeg -i ${HDV} -target dvd -threads 8 ${DVD}

echo "Input: MPEG2 DVD"
echo "Output: iTunes/iPod compatible MP4"
ffmpeg -y -i ${DVD} -an -v 1 -threads 8 -vcodec h264 -b 250k -bt 175k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 1 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 450k -bufsize 2M -cmp 1 -s 720x480 -f mp4 -pass 1 /dev/null

ffmpeg -y -i ${DVD} -v 1 -threads 8 -vcodec h264 -b 250k -bt 175k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 6 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 450k -bufsize 2M -cmp 1 -s 720x480 -acodec aac -ab 160k -ar 48000 -ac 2 -f mp4 -pass 2 -threads 8 ${MP4}

Stay tuned for discussions of the remaining scripts to make your life easier,
CM

H264 encoding for video iPod

The below commands will use FFMPEG to create a video suitable for playback within both iTunes and on a video iPod.

You will need a source file as input and a filename for the resulting output. My source file came from the rendered output of a Cinelerra project that I had just completed. The output was a DVD resolution MPEG program stream, created from directions in my Beginner's Guide to Exporting Video from Cinelerra.

A Two-Step Process
Below are two sets of FFMPEG commands. These two commands encapsulate a "two pass" encoding method. On the first pass, FFMPEG examines the file to be encoded and creates an optimization index. On the second pass, FFMPEG renders the output file based on the optimizations present in the index file.

In the below example, I am encoding a typical DVD resolution (720x480, 1.5 aspect ratio) to the following container format, using the following codecs:
- MPEG4 container
- H264 video codec
- AAC audio codec

For HDV fans, if you wish to reduce your videos to an iPod compatible size, you may use a resolution of 720x400 in order to keep the aspect ratio similar to that of 720P content (1.78 aspect ratio). Though the 720x400 resolution exceeds Apples' recommendations, I've found that iTunes and the iPod will still accept videos encoded to this size.

1) First pass (create index of optimizations)
ffmpeg -y -i sourceFileName.ext -an -v 1 -threads 8 -vcodec h264 -b 500k -bt 175k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 1 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 768k -bufsize 2M -cmp 1 -s 720x480 -f mp4 -pass 1 /dev/null

This first pass will create a file called x264_2pass.log that will be used as an implicit input to the second pass.

2) Second pass (render the output file)
ffmpeg -y -i sourceFileName.ext -v 1 -threads 8 -vcodec h264 -b 500k -bt 175k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 6 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 768k -bufsize 2M -cmp 1 -s 720x480 -acodec aac -ab 160k -ar 48000 -ac 2 -f mp4 -pass 2 -threads 8 outputFileName.ext

I found that the file resulting from the above commands was a bit large, so I lowered the bitrate (-b) from 500 to 250. Also, I lowered the maximum instantaneous bitrate spike (-maxrate) from 768 to 450. As this was a music video and I am a bit of an audiofile, I kept the audio bitrate (-ab) at 160k. Doing this, my file size dropped by 33%, from 450MB to 300MB. This is for a video with a duration of 1hr and 39min. Impressive! Replaying both files in mplayer and xine, I noticed that there was very little appreciable loss in video quality with the lowered bitrates. Go H264!!

Here are the final two commands I used to get that 300MB file:
ffmpeg -y -i sourceFileName.ext -an -v 1 -threads 8 -vcodec h264 -b 250k -bt 175k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 1 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 450k -bufsize 2M -cmp 1 -s 720x480 -f mp4 -pass 1 /dev/null

ffmpeg -y -i sourceFileName.ext -v 1 -threads 8 -vcodec h264 -b 250k -bt 175k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 6 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 450k -bufsize 2M -cmp 1 -s 720x480 -acodec aac -ab 160k -ar 48000 -ac 2 -f mp4 -pass 2 -threads 8 outputFileName.ext

Results
As this encoding work was completed on my Fedora editing box, I SCP'd the file over to my MacBook Pro. I dragged the file over onto the Movies section within iTunes and iTunes accepted it without error. In order to verify that the file played correctly, I double-clicked on it and Quicktime played the file back. Hooray!

The moment of truth came when I plugged in my iPod and sync'd it to my iTunes Library. Normally, iTunes will immediately display an error if it doesn't like the new file. Thankfully, iTunes didn't show any errors and uploaded the file to my iTouch. As the file was about 300MB, this took about two minutes. I then started playback and was very happy to see my video on the beautiful, but small iTouch screen.

If you'd like to see the end result, load up one of these videos and don't mind a bit of loud rock musicians doing their thing, you could try loading my band's video podcast:
http://www.stormpigs.com/vodcast.xml

One anomaly I noticed was that when you fast forward to a different section within the movie on the iPod, the video will speed up for a second until the timeline catches up to the requested moment. Not a big deal, just something I didn't expect.

Hope this helps intrepid open source folks looking for a way into the walled garden of Apple's iTunes.

The Mule

Reference
https://help.ubuntu.com/community/iPodVideoEncoding#H.264%20Encoding

MPEG4 - H264 comparison

Background
I've been wanting to produce some high quality, H.264 content for my bands' music for some time, but x264 had been broken on my Fedora system. Since I've finally fixed the problem, I can now render a proper H.264 file. Without H.264, I would choose MPEG4 video compression and an AAC audio codec.

MPEG4 file
I use ffmpeg to reduce a DVD sized mpeg down to a file that I could upload to iTunes, like so:
ffmpeg -i ${NAME}.m2t -f mov -vcodec mpeg4 -qscale 7 -s 320x180 -r 29.97 -aspect 16:9 -acodec aac -ac 2 -ab 128 ${NAME}.mov

The variable in this render string is "qscale" (good explanation of which at the bottom of this post) which varies quite a bit depending on the content of your video. The final resolution of the MPEG4 video was 320x180.

H.264 file
I used AVIdemux2 to output the H.264 video. I selected the defaults with a Single Pass - Quality Quantizer (Average) encoding and a similar AAC audio encoding of the MPEG4 of 128K.

The final resolution of the H.264 video was 720x400.

I'll translate the following output into a command line x264 statement at some point, but right now, here were the detailed options to avidemux2:
vui.i_sar_width = 1, vui.i_sar_height = 1, rc.f_qcompress = 0.60, analyse.i_direct_mv_pred = 1, rc.i_qp_min = 10, rc.i_qp_max = 51, rc.i_qp_step = 4, i_frame_reference = 1, i_scenecut_threshold = 40, i_keyint_min = 25, i_keyint_max = 250, i_bframe = 2, i_bframe_bias = 0, b_bframe_pyramid = 0, analyse. b_bidir_me = 0, b_bframe_adaptive = 1, analyse.b_weighted_bipred = 1, b_cabac = 1, analyse.i_trellis = 1, analyse.i_subpel_refine = 5, analyse.b_chroma_me = 1, b_deblocking_filter = 1, i_deblocking_filter_alphac0 = 0, i_deblocking_filter_beta = 0, analyse.i_me_method = 1, analyse.i_me_range = 16, analyse.b_transform_8x8 = 1, analyse.b_mixed_references = 0, analyse.i_noise_reduction = 0, _8x8P is on, _8x8B is on, _8x8I is on, _4x4I is on

Comparing the Output
The MPEG4 file from FFMPEG was 312MB.
The H264 file from AVIdemux2 was 438MB.

Here is how a couple of snips from each file look side by side:







Conclusions
I probably haven't experimented enough with FFMPEG to get the most out of the MPEG4 compression. But the quality and compression rate of H.264 is most impressive. Even though the resolution of the H.264 encoding video was twice that of than the MPEG4, the file size was only 33% greater. That speaks to the efficiency of the H.264 encoding scheme.

I'd say that the H264 output was well worth the extra 120MB, eh?

In case you wish to see the whole hour long shebang in either format, here are the two feeds:
The StormPigs 1/11/2007 jam in MPEG4 format, courtesy of FFMPEG (312MB)
The StormPigs 1/11/2007 jam in H264 format, courtesy of AVIdemux2 (438MB)

And if you are truly insane and wish to subscribe to the StormPigs iTunes feed, here it is:
http://www.stormpigs.com/vodcast.xml

:)
The Mule

creating a bootable iso in Linux

OK. So call me stupid, but I rarely create bootable DVDs from iso images in Fedora. Here is how I created a bootable Ubuntu 7.10 disk using growisofs.

Burn the ISO image to a DVD
1) Put your DVD in your DVD drive

2) Make sure your drive is reading the disk properly by using dvd+rw-mediainfo. This program is part of the dvd+rw tools. The command should also give you a write speed estimate, which is handy.

[mule@ogre ~]# dvd+rw-mediainfo /dev/dvd
INQUIRY:                [HP      ][DVD Writer 940d ][3H23]
GET [CURRENT] CONFIGURATION:
 Mounted Media:         11h, DVD-R Sequential
 Media ID:              ProdiscS03  
 Current Write Speed:   4.0x1385=5540KB/s
 Write Speed #0:        4.0x1385=5540KB/s
 Write Speed #1:        2.0x1385=2770KB/s
GET [CURRENT] PERFORMANCE:
 Write Performance:     4.0x1385=5540KB/s@[0 -> 0]
 Speed Descriptor#0:    00/0 R@6.4x1385=8864KB/s W@4.0x1385=5540KB/s
 Speed Descriptor#1:    00/0 R@6.4x1385=8864KB/s W@2.0x1385=2770KB/s
READ DVD STRUCTURE[#10h]:
 Media Book Type:       00h, DVD-ROM book [revision 0]
 Legacy lead-out at:    2298496*2KB=4707319808
READ DVD STRUCTURE[#0h]:
 Media Book Type:       25h, DVD-R book [revision 5]
 Last border-out at:    2045*2KB=4188160
READ DISC INFORMATION:
 Disc status:           blank
 Number of Sessions:    1
 State of Last Session: empty
 "Next" Track:          1
 Number of Tracks:      1
READ TRACK INFORMATION[#1]:
 Track State:           invisible incremental
 Track Start Address:   0*2KB
 Next Writable Address: 0*2KB
 Free Blocks:           2297888*2KB
 Track Size:            2297888*2KB
READ CAPACITY:          0*2048=0

3) Using growisofs, burn the ISO image to a DVD. Note that I used this command on a DVD RW:
[mule@ogre ~]# growisofs -dvd-compat -Z /dev/dvd=ubuntu-7.10-desktop-amd64.iso
Executing 'builtin_dd if=ubuntu-7.10-desktop-amd64.iso of=/dev/dvd obs=32k seek=0'
/dev/dvd: "Current Write Speed" is 2.0x1352KBps.
3407872/2005147648 ( 0.2%) @0.5x, remaining 39:09 RBU 100.0% UBU 6.5%
12288000/2005147648 ( 0.6%) @1.9x, remaining 18:55 RBU 100.0% UBU 100.0%
..
1992097792/2005147648 (99.3%) @2.0x, remaining 0:04 RBU 77.9% UBU 100.0%
2001076224/2005147648 (99.8%) @1.9x, remaining 0:01 RBU 24.4% UBU 100.0%
builtin_dd: 979088*2KB out @ average 1.9x1352KBps
/dev/dvd: flushing cache
/dev/dvd: writing lead-out

The disc..she boots!
The Mule

Xine install on Fedora 7 x86-64

Another reason to dump FreshRPMs. I wanted to Xine on my Fedora 7, x86-64 system and got the following error:
Error: Missing Dependency: xine-lib = 1.1.7 is needed by package xine-lib-moles

Yuck. Upon investigation, it looks like FreshRPMs were screwing me up:
http://forums.fedoraforum.org/showthread.php?t=179979
http://forums.fedoraforum.org/showthread.php?t=180425

I was using FreshRPMs because they had previously worked on my Fedora Core 6 install:
/2007/06/xine-no-demuxer-plugin-available-to.html

Now that I was on Fedora 7, things had changed and those FreshRPM packages were no longer good. Specifically, I had some xine-lib packages installed from FreshRPMs which conflicted with either the Livna or Fedora base packages. I removed them:
[root@ogre ~]# yum remove xine-lib
=============================================================================
Package Arch Version Repository Size =============================================================================
Removing:
xine-lib x86_64 1.1.6-2.fc7 installed 3.9 M
Removing for dependencies:
xine-lib-devel x86_64 1.1.6-2.fc7 installed 649 k
xine-lib-extras-nonfree x86_64 1.1.7-1.lvn7 installed 1.1 M

I then installed xine from only the Fedora base and Livna repositories:
[root@ogre ~]# yum install xine --disablerepo freshrpms
Loading "installonlyn" plugin
Setting up
Install Process
Parsing package install arguments
Resolving Dependencies -->
Running transaction check --->
Package xine.x86_64 0:0.99.5-1.lvn7 set to be updated -->
Processing Dependency: libxine.so.1()(64bit) for package: xine -->
Restarting Dependency Resolution with new changes. -->
Running transaction check --->
Package xine-lib.x86_64 0:1.1.10.1-1.fc7 set to be updated

Dependencies Resolved
=============================================================================
Package Arch Version Repository Size =============================================================================
Installing: xine x86_64 0.99.5-1.lvn7 livna 1.6 M
Installing for dependencies: xine-lib x86_64 1.1.10.1-1.fc7 updates 2.3 M

Transaction Summary
=============================================================================
Install 2 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 3.8 M Is this ok [y/N]: y
Downloading Packages:
(1/2): xine-0.99.5-1.lvn7 100% |=========================| 1.6 MB 00:02
(2/2): xine-lib-1.1.10.1- 100% |=========================| 2.3 MB 00:02

Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded

Running Transaction
Installing:xine-lib ######################### [1/2]
Installing: xine ######################### [2/2]
Installed: xine.x86_64 0:0.99.5-1.lvn7
Dependency Installed: xine-lib.x86_64 0:1.1.10.1-1.fc7
Complete!

Testing the xine install with "xine-check", I got this error:
[root@ogre ~]# xine-check
[ hint ] No xine-config found. Assuming xine from RPMs The xine-config script can be used to determine some file locations used by xine-lib, but you don't have such a script on your system. However, it looks like you installed xine from the RedHat packages. So I'll just guess that you are using the standard locations. If you want me to be sure about those file locations, you can install the 'xine-lib-devel' package (or 'xine-devel', depend on what packages you're using, which contains xine-config. However, this package is not really needed to run xine... press to continue...

Doing what the man says, I installed xine-lib-devel from Fedora Updates:
[root@ogre ~]# yum install xine-lib-devel --disablerepo freshrpms
=============================================================================
Package Arch Version Repository Size ============================================================================= Installing:
xine-lib-devel i386 1.1.10.1-1.fc7 updates 282 k
xine-lib-devel x86_64 1.1.10.1-1.fc7 updates 282 k
Installing for dependencies:
xine-lib i386 1.1.10.1-1.fc7 updates 2.6 M

When I ran xine-check again, I no longer received that error. Good. But when I started up xine to play a test mpeg video, I got the dreaded error:
There is no demuxer plugin available to handle "file:/usr/share/xine/skins/xine-ui_logo.mpv".
Usually this means the file format was not recognized.

Hmm. I listed the packages I installed and found that I didn't have the "xine-lib-extras" packages installed. Specifically, the "xine-lib-extras-nonfree" one from Livna:
[root@ogre ~]# rpm -qa | grep xine
xine-lib-devel-1.1.10.1-1.fc7
xine-0.99.5-1.lvn7
xine-lib-1.1.10.1-1.fc7
xine-lib-devel-1.1.10.1-1.fc7
xine-lib-arts-1.1.10.1-1.fc7
xine-lib-1.1.10.1-1.fc7

I then installed "xine-lib-extras-nonfree" from Livna:
=============================================================================
Package Arch Version Repository Size ============================================================================= Installing:
xine-lib-extras-nonfree x86_64 1.1.10.1-1.lvn7 livna 558 k

Transaction Summary =============================================================================
Install 1 Package(s) Update 0 Package(s) Remove 0 Package(s)

Happily, my mpeg video played! Also, an x264 video played. Great! So it looks like the Livna "nonfree" package is responsible for this joy. Thanks, Livna!

The lesson here folks is that you have to watch your libraries. And stay away from FreshRPMs, if at all possible.

CM

x264 troubles..ends well

Anyone who reads my blog knows that although I am a loyal Penguin, I'm all about getting the work done expeditiously and distributing my videos to the widest array of formats possible. One of those formats is Apple's Quicktime. And for me, a rendered Quicktime usually ends up as an MPEG4 video in iTunes. With Apple announcing that the new Apple TV 2.0 displays web content as well as iTunes podcasts, higher definition content on iTunes is poised for much growth.

Therefore, it is in that light that I have tasked myself to encode and share some of my band videos to iTunes in glorious (or in the case of my band, un-glorious) DVD resolution. And while I'm at it, I should use tasty H.264 compression for maximum quality at a minimum file size. But of course, this seemingly simple task was much easier said than done.

For me, the rub is that encoding to x264 in Cinelerra is broken. I was getting this error when encoding to H.264:
x264 [error]: no ratecontrol method specified

I had seen this error a few weeks back and emailed the Cinelerra CVS board, but no one had a clue. But tonight, I was going to figure out what the hell was going on. So my first stop was to try encoding to h264 with my old mainstay, ffmpeg. Alas, my friend through thick and thin segfaulted:
[root@ogre 20080206]# ffmpeg -i StormPigs20080206.mpg -an -pass 1 -vcodec h264 -b 1500 -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -me epzs -subq 1 -trellis 0 -refs 1 -bf 3 -b_strategy 1 -coder 1 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 1500 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 output.mp4
FFmpeg version SVN-r8876,
Copyright (c) 2000-2007 Fabrice Bellard, et al.
configuration: --prefix=/usr --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --extra-cflags=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic --enable-libmp3lame --enable-libogg --enable-libtheora --enable-libvorbis --enable-libfaad --enable-libfaac --enable-libgsm --enable-xvid --enable-x264 --enable-liba52 --enable-liba52bin --enable-libdts --enable-pp --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-opts --disable-strip
libavutil version: 49.4.0
libavcodec version: 51.40.4
libavformat version: 51.12.1
built on May 3 2007 12:41:19, gcc: 4.1.2 20070424 (Red Hat 4.1.2-11)
Input #0, mpeg, from 'StormPigs20080206.mpg':
Duration: 01:50:37.0, start: 0.500000, bitrate: 4677 kb/s
Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x480, 9000 kb/s, 29.97 fps(r)
Stream #0.1[0x80]: Audio: ac3, 48000 Hz, stereo, 448 kb/s
Output #0, mp4, to 'output.mp4':
Stream #0.0: Video: h264, yuv420p, 720x480, q=10-51, pass 1, 1 kb/s, 29.97 fps(c)
Stream mapping:
Stream #0.0 -> #0.0
Segmentation fault


FFmpeg had crapped out, so I gave avidemux2 a shot. When I tried to encode using either command line x264 or x264 encoding through avidemux2, I would get this error:
x264 [error]: width or height not divisible by 2 (480x1)

Update 2009/02/12
Here's a more recent entry regarding proper divisors for x264 compression:
http://crazedmuleproductions.blogspot.com/2009/01/high-quality-h264-output.html
*** end update ***

Well..the video is 720x480. Certainly those two numbers are divisible by two, so what gives with the logic of that error?

After Googling and trying various tweaks for about an hour, it occurred to me that there seemed to be no rhyme or reason to the error I was seeing. With that thought in mind, I decided to remove the x264 package entirely. I had installed the package from freshrpms:

[root@ogre 20080206]# yum remove x264*
Loading "installonlyn" plugin
Setting up
Remove Process
Resolving Dependencies -->
Running transaction check
---> Package x264.x86_64 0:0.0.0-0.3.20070529.fc7 set to be erased
---> Package x264-devel.x86_64 0:0.0.0-0.3.20070529.fc7 set to be erased

Dependencies Resolved =============================================================================
Package Arch Version Repository Size ============================================================================= Removing:
x264 x86_64 0.0.0-0.3.20070529.fc7 installed 1.0 M
x264-devel x86_64 0.0.0-0.3.20070529.fc7 installed 712 k

Hmmm. Looking at the version number, it seemed like the x264 from FreshRPMs was a VERY early version number. Let me see if another repository has a different version. Luckily, Livna had some love waiting for me:
[root@ogre 20080206]# yum install x264* --disablerepo=freshrpms
Loading "installonlyn" plugin
Setting up Install Process
Parsing package install arguments
Resolving Dependencies -->
Running transaction check
---> Package x264.x86_64 0:0-0.8.20061028.lvn7 set to be updated
---> Package x264-devel.x86_64 0:0-0.8.20061028.lvn7 set to be updated

Dependencies Resolved =============================================================================
Package Arch Version Repository Size ============================================================================= Installing:
x264 x86_64 0-0.8.20061028.lvn7 livna 227 k
x264-devel x86_64 0-0.8.20061028.lvn7 livna 15 k

Now that version looks a little more recent, yes? With the new Livna repository x264 installed, I tried my render using the shortest path possible: avidemux2. Lo and behold, x264 no longer complained about my height and width not being divisible by 2! Better yet, I was able to render a complete file using avidemux2's two pass defaults without any errors! Some folks might appreciate seeing the verbose output:
**saving:**
Output format:7
AVI family
Enc X264, using /mnt/videos/20070912/StormPigs20070912.mp4.stat as logfile

X264 Encoder ready , w: 720 h:480 mode:3X264 pass 1
x264 codec using /mnt/videos/20070912/StormPigs20070912.mp4.stat as statfile
Opening X264 for 720 x 480
vui.i_sar_width = 1
vui.i_sar_height = 1
rc.f_qcompress = 0.60
analyse.i_direct_mv_pred = 1
rc.i_qp_min = 10
rc.i_qp_max = 51
rc.i_qp_step = 4
i_frame_reference = 1
i_scenecut_threshold = 40
i_keyint_min = 25
i_keyint_max = 250
i_bframe = 2
i_bframe_bias = 0
b_bframe_pyramid = 0
analyse. b_bidir_me = 0
b_bframe_adaptive = 1
analyse.b_weighted_bipred = 0
b_cabac = 1
analyse.i_trellis = 0
analyse.i_subpel_refine = 2
analyse.b_chroma_me = 1
b_deblocking_filter = 1
i_deblocking_filter_alphac0 = 0
i_deblocking_filter_beta = 0
analyse.i_me_method = 0
analyse.i_me_range = 16
analyse.b_transform_8x8 = 0
analyse.b_mixed_references = 0
analyse.i_noise_reduction = 0
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
X264 init ok (atom mode : 1)
X264 using 4 threads
Nb nal :3
?? type :6 in nal 0
X264 :Unknown image type:0
X264 :Unknown image type:0

Starting pass 2 (720x480)
[x264] Size 700, average bitrate 1007301 kb/s
x264 [info]: slice I:703 Avg QP: 0.00 size:153296 PSNR Mean Y:70.14 U:71.46 V:71.57 Avg:70.54 Global:67.35
x264 [info]: slice P:124246 Avg QP: 2.00 size: 98655 PSNR Mean Y:59.56 U:60.40 V:60.59 Avg:59.82 Global:57.88
x264 [info]: slice B:49758 Avg QP: 4.00 size: 59273 PSNR Mean Y:66.82 U:67.47 V:67.73 Avg:67.04 Global:57.02
x264 [info]: mb I I16..4: 19.0% 0.0% 81.0%
x264 [info]: mb P I16..4: 9.0% 0.0% 0.0% P16..4: 85.4% 0.0% 0.0% 0.0% 0.0% skip: 5.6%
x264 [info]: mb B I16..4: 4.3% 0.0% 0.0% B16..8: 39.5% 0.0% 0.0% direct:30.0% skip:26.3%
x264 [info]: SSIM Mean Y:0.9992809
x264 [info]: PSNR Mean Y:61.673 U:62.456 V:62.668 Avg:61.920 Global:57.635 kb/s:21017.02

X264 dual size: 700 MB
X264 pass 2, using bitrate of 1007
x264 codec using /mnt/videos/20070912/StormPigs20070912.mp4.stat as statfile
Opening X264 for 720 x 480
vui.i_sar_width = 1
vui.i_sar_height = 1
rc.f_qcompress = 0.60
analyse.i_direct_mv_pred = 1
rc.i_qp_min = 10
rc.i_qp_max = 51
rc.i_qp_step = 4
i_frame_reference = 1
i_scenecut_threshold = 40
i_keyint_min = 25
i_keyint_max = 250
i_bframe = 2
i_bframe_bias = 0
b_bframe_pyramid = 0
analyse. b_bidir_me = 0
b_bframe_adaptive = 1
analyse.b_weighted_bipred = 1
b_cabac = 1
analyse.i_trellis = 2
analyse.i_subpel_refine = 5
analyse.b_chroma_me = 1
b_deblocking_filter = 1
i_deblocking_filter_alphac0 = 0
i_deblocking_filter_beta = 0
analyse.i_me_method = 1
analyse.i_me_range = 16
analyse.b_transform_8x8 = 1
analyse.b_mixed_references = 0
analyse.i_noise_reduction = 0
_8x8P is on
_8x8B is on
_8x8I is on
_4x4I is on
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
X264 init ok (atom mode : 1)
X264 using 4 threads
Nb nal :3
?? type :6 in nal 0
x264 has 39 extra bytes
We have extradata for video in copy mode (39)
X264 :Unknown image type:0
X264 :Unknown image type:0
[Bridge] Going to time 0

Syncing on 8192
Sync found at offset 0
A52 sync found at 0 + 0
[Bridge] Starting with time 0, shift 0
[Bridge] Ending with time 0, sample 0
[Bridge] Going to time 0

Syncing on 8192
Sync found at offset 0
A52 sync found at 0 + 0
[Bridge] Going to time 0

Syncing on 8192
Sync found at offset 0
A52 sync found at 0 + 0
[FAAC] : Sample input:2048, max byte output1536
[Faac] Initialized :
[Faac]Version : 1.25
[Faac]Bitrate : 192000
[Faac]Mpeg2 (1)/4(0) : 0
[Faac]Use lfe ) : 0
[Faac]Sample output : 1024
[Faac]Bitrate : 384000
[LavFormat] Bitrate 384
Output #0, mp4, to '/mnt/videos/20070912/StormPigs20070912.mp4':
Stream #0.0, 29.97 fps(c): Video: 0x0000, 720x480, q=2-31, 2000 kb/s
Stream #0.1: Audio: 0x0000, 48000 Hz, stereo, 384 kb/s
lavformat mpeg muxer initialized
PacketQueue MP4 audioQ created
[AudioQueueThread] Starting
DMX_audio Going out of bound (position : 326445000 asked 5000 end326446848)
DMX_audio Going out of bound (position : 326446848 asked 5000 end326446848)
**PKTZ:READ ERROR
**END OF AUDIO STREAM
EditorPacket:Read failed; retrying (were at seg 0sample 279810048
/ 279814208)
EditorPacket : End of *last* stream
read failed, end of stream ?
EditorPacket:Read failed; retrying (were at seg 0sample 279810048
/ 279814208)
EditorPacket : End of *last* stream
read failed, end of stream ?
[Bridge] End of stream
EditorPacket:Read failed; retrying (were at seg 0sample 279810048
/ 279814208)
EditorPacket : End of *last* stream
read failed, end of stream ?
[Bridge] End of stream
EditorPacket:Read failed; retrying (were at seg 0sample 279810048
/ 279814208)
EditorPacket : End of *last* stream
read failed, end of stream ?
[Bridge] End of stream
[bridge] No data in 0 max 24448 out 0
[AudioQueueThread] Exiting
[AudioThread] Target 4294901760, got 279810048, 0.065149 %
LavMuxer:Audio DTS is too low 5829354666 / 5829362696!
x264 [error]: 2nd pass has more frames than 1st pass (174709)
x264 [error]: continuing anyway, at constant QP=22
x264 [error]: disabling adaptive B-frames
LavMuxer:Audio DTS is too low 5829354666 / 5829396062!
x264 [info]: slice I:703 Avg QP:18.35 size: 28210 PSNR Mean Y:52.78 U:53.43 V:54.33 Avg:53.09 Global:47.62
x264 [info]: slice P:124248 Avg QP:21.42 size: 5232 PSNR Mean Y:47.14 U:47.26 V:48.82 Avg:47.39 Global:44.70
x264 [info]: slice B:49758 Avg QP:21.06 size: 1286 PSNR Mean Y:58.14 U:58.09 V:59.69 Avg:58.30 Global:45.20
x264 [info]: mb I I16..4: 17.5% 42.4% 40.1%
x264 [info]: mb P I16..4: 2.4% 9.0% 0.2% P16..4: 39.2% 12.0% 2.3% 0.0% 0.0% skip:34.8%
x264 [info]: mb B I16..4: 0.1% 1.2% 0.0% B16..8: 7.7% 0.4% 0.8% direct:14.5% skip:75.3%
x264 [info]: 8x8 transform intra:76.4% inter:65.0%
x264 [info]: SSIM Mean Y:0.9828109
x264 [info]: PSNR Mean Y:50.297 U:50.371 V:51.938 Avg:50.519 Global:44.843 kb/s:1007.06
[Bridge] Destroying bridge
[FAAC] Deleting faac
yv12close called
ba010000 -> ba010000


And best yet, the rendered x264 file loads into iTunes! Hooray!

Of course, I haven't tried actually bringing it to an iPod, but that will be the next thing to do. I'll let you know.
cheers,
da mule

Reference
Here are Apple's specifications for making a podcast:
http://www.apple.com/itunes/store/podcaststechspecs.html

Why Cinelerra?

I received a question from a person unfamiliar with Cinelerra and I thought that my response should be shared. The person has a friend who runs Cinelerra in Ubuntu and he wanted a more experienced users' perspective on how Cinelerra compares to the more established, commercial packages like Premiere or Vegas. Here is how I responded to him.

Cinelerra is very powerful software if you can overcome some of its idiosyncracies. Idiosyncracies like:
1) it is difficult to install for most people
2) it sometimes crashes. There are more crashes on 32-bit systems. However, the consequences of a crash aren't bad, as you can simply restart Cinelerra, load the automatically generated backup file and pick up where you left off.
3) rendering to final formats is challenging

Cinelerra is buggy in a consistent way. In other words, if you have the time and energy to figure out what works and what doesn't work, then you can base a workflow around that. But that effort is a huge time sink.

It doesn't get any better when you upgrade your system, because what once
worked in a previous distro will break in your new distro. So, you spend
oodles of time figuring out how to fix it.

It takes a certain kind of person who relishes the constant challenges of Cinelerra and Linux to power through these difficulties. Cinelerra is not everyone's cup of tea for sure, but you can get usable content out of Cinelerra if you know what works and what doesn't. Otherwise, save your valuable time and effort, buy a dual quad core Mac or PC with Final Cut Pro or Avid or Premiere or Vegas and be happy that everything works out of the box. Which isn't always the case even with those softwares.

Because Cinelerra is free software, it will never be as well supported as those commercial products. That being said, I have Cinelerra installed on Fedora 10 x86, 64-bit and the only time it has crashed is when I've done something stupid like try to write to a filesystem that is full or trigger a known bug, like using the broken DV import-record function.

I have spent the last two years learning the software and documenting specific processes using Cinelerra. You might glean some useful information by subscribing to my blog's RSS feed, because it documents the challenges I face in getting the software to work properly:
http://crazedmuleproductions.blogspot.com/

I would suggest that a 64-bit machine is most stable for Cinelerra, moreso than a 32-bit machine. That is due to the fact that the 64-bit build will take full advantage of the memory resources on your computer and video editing is very taxing on a computers memory.

From the conversations on http://cvs.cinelerra.org/, the largest installed base of users seems to be on Ubuntu, Fedora and Debian distros. I would suggest starting with one of those distros, as many people on the CVS will be able to help you if you encounter problems.

There are two versions of Cinelerra. The original version you can get from http://www.heroinewarrior.com/, but the author does not support the software. The most actively supported version of the software is the Community Version (CV) and can be found on http://cvs.cinelerra.org/. That community of which I am part, has done significant work to fix bugs, accumulate and pass on knowledge to people interested in this great, free software and video editing in general.

The RPMFusion repository is the one main Fedora repository has Cinelerra RPMs for most recent Fedora versions:
rpmfusion.org
Information on installing Cinelerra for all other distributions can be found here:
http://cvs.cinelerra.org/getting_cinelerra.php

Here is a list of related articles regarding first-time installations on Fedora:
Getting Started with Cinelerra
Building Cinelerra on 64-bit Fedora 9 from Source
Beginner's Guide to Exporting Video from Cinelerra
Exports and Linux Player Compatibility Chart from the article Render Compatibility on Fedora 10, x86-64
Cinelerra for Grandma

Finally, if you don't want to fuss with your own install, you may download one of two VMware virtual machines that I have created specifically for video editing with Cinelerra and all the tools you need to get started:
Fedora Core 6, 32-bit VMware virtual machine ~1GB
Fedora 10, x86-64 VMware virtual machine ~3GB

The Mule

capturing WinAmp visuals

Like a lot of us who are into film and video editing, I am also a part-time musician. For Christmas, I received a very exciting present: a digital model of an old analog synthesizer, the Creamware Prodyssey:
Sound on Sound review of Prodyssey

Oh, how lovely it is to create sounds with this box of sliders and switches! The music part will come later, but right now I am enjoying creating rythymic bleeps and bloops the old fashioned way using oscillators, filters and envelopes. Attack, decay, sustain, release, LFO and all that lot. As I was creating this cacaphony, I thought that it would be cool to have some visualizations to go along with the audio.

Years ago using Windows XP, I streamed music visualizations from WinAmp out through S-Video to a second computer that would capture the pretty pictures. The quality wasn't that great. Of course, if WinAmp had an export feature, that would be the best. But working with what I had today, I figured now that I've purchased a mighty dual quad core, I should be able to capture the screens within the machine itself. Of course, my primary OS is now Linux. And WinAmp is not made for Linux. And I didn't know of a Linux based visualizer that looked quite as good as WinAmp. What to do? I remember using Wine, the Windows emulator for Linux, back in 2001, but it wasn't very stable. I wonder if it's more stable these days?

With thoughts of sugar-plum visualizations in my head, I installed Wine the other day. Given my past experiences, I wasn't very hopeful. I was pleasantly surprised to find that the latest version of Wine was easy to setup on my Fedora 7 x86-64 system. Once Wine was setup, I installed WinAmp through a short series of steps. Excellent! I opened an MP3 file and was again, glad to hear the MP3 playing. Wow! Two in a row! Can we go for a trifecta? With the audio coming out of WinAmp through Wine, I decided to go for broke and started up the Advanced Visualization Studio visualizer. I could hardly believe my eyes when AVS played in gorgeous colors in a window on one of my monitors!


So now I have WinAmp working under Wine. But how to capture the beautiful screens? Here, Cinelerra came to the rescue. Using my BFG Geforce 8500 GT PCI Express 256MB card and a dual-head configuration, I opened Cinelerra in my right monitor, started WinAmp under Wine and kicked off the visualization in the left monitor. I expanded the window to roughly DVD resolution (720x480) and tweaked my Cinelerra screencapture recording settings to match the visualization window. Lo and behold, Cinelerra was capturing the resolutions at DVD quality! I looked carefully for frame drops. This is listed in the Cinelerra Recording window as "Frames Behind". I did not see one frame drop! Awesome!


I synchronized the results of the screen capture with the music on the Cinelerra timeline and output the audio and video to DVD-compatible formats. I then burnt a menuless DVD and was off to the races! I had created some "music", added a visualization track and burned a DVD! Sweet!

Next up:
How cool would it be to capture the visualizations in HD and play them on my HDTV at 720P resolution? Ooooh. Aaaah.

new 500GB SATA RAID1 mirror

For fault tolerance, I finally got around to upgrading my drives to a RAID1 mirror this weekend. I bought a pair of Seagate ST3500641AS-RK Barracuda from Outpost.com for $99 each. Sale is over now, of course.

Here's what I had to do:

1. partition the drives using fdisk
2. create the mirror
3. wait for the mirror to sync
   
4. format the mirror
5. test a mount
6. add to /etc/fstab
7. reboot to make sure everything comes up
   

Partition the drives with good ol' fdisk
I had to partition each drive and give each drive a partition type of "Linux raid autodetect". This is type "fd". Here is a snip of my fdisk session for one of the drives:

[mule@ogre ~]# fdisk /dev/sdf
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.


The number of cylinders for this disk is set to 60801.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): p

Disk /dev/sdf: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4):
Value out of range.
Partition number (1-4): 1
First cylinder (1-60801, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-60801, default 60801):
Using default value 60801

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd

Command (m for help): p

Disk /dev/sdf: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdf1 1 60801 488384001 fd Linux raid autodetect

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Create the mirror
I created the mirror using mdadm. Note the "--level=1" for the RAID1 mirror:
[mule@ogre ~]# mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/sde1 /dev/sdf1

Let's check out the details:
[mule@ogre ~]# mdadm --detail /dev/md2
/dev/md2:
Version : 00.90.03
Creation Time : Thu Jan 17 19:58:30 2008
Raid Level : raid1
Array Size : 488383936 (465.76 GiB 500.11 GB)
Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 2
Persistence : Superblock is persistent

Update Time : Sun Jan 20 16:19:10 2008
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

UUID : 1705b387:1c71d83e:364b60b4:fb0cce92
Events : 0.6

Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sde1
1 8 49 1 active sync /dev/sdf1

Wait for the mirror to build
You can also look in /proc/mdstat for status information:
[mule@ogre ~]# cat /proc/mdstat
Personalities : [raid0] [raid6] [raid5] [raid4] [raid1]
md2 : active raid1 sdb1[0] sdd1[1]
488383936 blocks [2/2] [UU]

md0 : active raid0 sda3[0] sdc2[1]
483411456 blocks 256k chunks

unused devices:

Format the partition
The -j parameter to mke2fs creates the filesystem with an ext3 journal, for another level of safety. File access on an ext3 filesystem is a bit slower than on an ext2 filesystem. However, the benefit is that you have a journaling filesystem that recovers from errors quickly and safely. Here's some documentation on the ext3 filesystem:
http://www.ibm.com/developerworks/linux/library/l-fs7.html

Here is the command I ran to create the ext3 filesystem:
[mule@ogre ~]# mke2fs -j /dev/md2

Test the mount
You should always test mounting the new filesystem. Since this filesystem is going to store all my videos, I am going to create my mount point and mount the new RAID mirror to /mnt/videos:
[mule@ogre ~]# mkdir /mnt/videos
[mule@ogre ~]# mount -t ext3 /dev/md2 /mnt/videos
[mule@ogre ~]#

You see no errors after the mount command. This means that the mount was successful. Yes!

Add to /etc/fstab
Of course, we want the new filesystem to be mounted when we reboot, so I add the following line to my /etc/fstab:
/dev/md2 /mnt/videos ext3 defaults 1 1

Reboot!
The final test is to reboot. The following lines in dmesg output make me happy:
md: md2 stopped.
md: bind
md: bind
md: raid1 personality registered for level 1
raid1: raid set md2 active with 2 out of 2 mirrors

Rock and roll! We're good with the new RAID mirror and safe from drive failures!
The Mule

dvd file for iTunes

Last night, I was putting together a presentation on my MacBook and wanted to incorporate a DVD resolution video from Cinelerra. Since the Mac world is so tightly controlled, getting the format correct for iTunes was a royal pain. I tried a number of different renders, but only one worked:

file format video audio result
QTforLinux MPEG4 MPEG4 no (crashed iTunes 7.1)
QTforLinux MPEG4 MP3 no
QTforLinux MPEG4 WAV yes
QTforLinux H.264 MPEG4 cinelerra hung

At this point, I'm not sure if its my Fedora 7, x86-64 libraries or just iTunes being very selective about what it likes and dislikes. Just for kicks, tonight I will render out the same file using Fedora 6, i386 and see if the results are the same. Once I finish my research, I'll update the Beginner's Guide.

the mule