VMware virtual appliance for video editing

Over the weekend, I created a VMware Partner Account and got my Fedora 10, x86-64 virtual machine approved to be listed on VMware's Virtual Appliance listings:
http://www.vmware.com/appliances/directory/148183

If you want to try out Cinelerra and you use 64-bit VMware Player, Workstation or Server, this is an easy way to get started. I'd appreciate someone giving it a shot and letting me know how it works.

the mule

Fedora 10 x86-64 Cinelerra build

Update 2009/02/24
You can avoid having to build Cinelerra from source by using Nicolas Chauvet's (Kwizart) precompiled Cinelerra installs:
1) install the Kwizart yum repositories
http://rpms.kwizart.net/kwizart-release-10.rpm

2) install cinelerra-cv
[mule@ogre doc]$ sudo yum install cinelerra-cv* --enablerepo=kwizart
[sudo] password for sfrase:
Loaded plugins: refresh-packagekit
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
--> Running transaction check
---> Package cinelerra-cv.x86_64 0:2.1-21.git20081103.fc10 set to be updated
--> Processing Dependency: bitstream-vera-fonts for package: cinelerra-cv
--> Processing Dependency: libmpeg3-utils for package: cinelerra-cv
---> Package cinelerra-cv-debuginfo.x86_64 0:2.1-21.git20081103.fc10 set to be updated
--> Running transaction check
---> Package libmpeg3-utils.x86_64 0:1.8-1.fc10 set to be updated
---> Package bitstream-vera-fonts.noarch 0:1.10-8 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
cinelerra-cv x86_64 2.1-21.git20081103.fc10 kwizart 6.3 M
cinelerra-cv-debuginfo x86_64 2.1-21.git20081103.fc10 kwizart 9.6 M
Installing for dependencies:
bitstream-vera-fonts noarch 1.10-8 fedora 345 k
libmpeg3-utils x86_64 1.8-1.fc10 rpmfusion-free 19 k

Transaction Summary
================================================================================
Install 4 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 16 M
Is this ok [y/N]: y
Downloading Packages:
(1/4): libmpeg3-utils-1.8-1.fc10.x86_64.rpm | 19 kB 00:00
(2/4): bitstream-vera-fonts-1.10-8.noarch.rpm | 345 kB 00:00
(3/4): cinelerra-cv-2.1-21.git20081103.fc10.x86_64.rpm | 6.3 MB 00:05
(4/4): cinelerra-cv-debuginfo-2.1-21.git20081103.fc10.x8 | 9.6 MB 00:11
--------------------------------------------------------------------------------
Total 688 kB/s | 16 MB 00:24
warning: rpmts_HdrFromFdno: Header V3 DSA signature: NOKEY, key ID 5b01f801
kwizart/gpgkey | 1.7 kB 00:00
Importing GPG key 0x5B01F801 "Nicolas Chauvet (kwizart) " from /etc/pki/rpm-gpg/RPM-GPG-KEY-kwizart
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : libmpeg3-utils 1/4
Installing : bitstream-vera-fonts 2/4
Installing : cinelerra-cv-debuginfo 3/4
Installing : cinelerra-cv 4/4

Installed:
cinelerra-cv.x86_64 0:2.1-21.git20081103.fc10
cinelerra-cv-debuginfo.x86_64 0:2.1-21.git20081103.fc10

You're done!
*** end update ***

Building from Source
Though editing video on Linux is never easy, I'm happy to say that Fedora 10 is finally stable, after I've resolved or worked around the various bugs I've encountered.

I built Cinelerra from the CVS repository (not Heroine Warrior's) on Fedora 10 x86-64 about a month and a half ago, but haven't had time to post the steps. I can say I've put the Fedora 10 build through its paces by editing all different formats in the context of 1080p video. I will add the caveat that Cinelerra is very choosy about the formats it likes, as shown in my testing results below:

* Note that I haven't tested all combinations of containers and compression schemes, but this is a good first step

The steps are the same as the steps I ran to build Cinelerra on Fedora 9. Though this post will be rather short, consult my Fedora 9 post for all the details. FYI - the Fedora 9 system and Cinelerra build was so fraught with problems that I opted to move on to Fedora 10. I suggest you do the same.

Detail
The below steps should all be run as "root" or sudo
1) install Fedora 10
I usually select the Developer's package, as it will include many of the developer libraries necessary to build Cinelerra from source. Be aware that this install is rather large, weighing in at around 7GB.

Update 2008/02/17
After reviewing the storage consuming "Developer" install, I decided to build out a "Custom" install of Fedora. The base + Cinelerra dependencies yielded a slimmer install, at about 3.5GB.

However, for ease of use, it is probably easier to go ahead and install the "Developer" install. I did not do this, and even with all the Cinelerra dependencies checking out as "Found", I encountered three problems:
1) g++ was missing (go ahead and do "yum install gcc-c++" to resolve this)
2) libXv-devel was missing (the Cinelerra make process failed on a libxv header file)
3) libXxf86vm-devel was missing (the Cinelerra process failed on "/usr/bin/ld cannot find -lXxf86vm")

Oh, the fun we have!
*** end update ***

2) add the RPM Fusion repository for yum
http://rpmfusion.org/Configuration

3) install the dependencies for Cinelerra
For this step, I've provided a script below that installs all dependent programs for a Cinelerra installation from two repos: Fedora base and RPM Fusion.

Paste the below text into a file, save it and run it as a script. Don't forget to "chmod a+x yourFile" in order to make your script executable. The script will install all the dependencies in order to build Cinelerra
yum install gsm-devel \
libvorbis* \
libogg* \
libtool* \
libtheora* \
libpng* \
libjpeg* \
libtiff* \
esound* \
audiofile* \
libraw1394* \
libavc1394* \
freetype* \
fontconfig* \
nasm \
e2fsprogs* \
OpenEXR* \
fftw \
fftw-devel \
libsndfile* \
libiec61883* \
libdv* \
libquicktime \
ffmpeg \
xvidcore* \
lame \
lame-devel \
a52* \
faad2* \
x264* \
mjpegtools* \
faac* \
vlc*

4) get the Cinelerra source
git clone git://git.cinelerra.org/j6t/cinelerra.git cinelerra_source

5) in the Cinelerra source directory, run ./autogen.sh

6) in the Cinelerra source directory, run ./configure

7) As long as configure shows no errors, go ahead and run "make"

8) As long as make showed no errors, run "make install"

That should be it. Again, consult my Fedora 9 Cinelerra install post for more detail on these steps.

Lastly, you could avoid the whole build process and just use my Fedora 10, x86-64 VMware virtual machine, about 3GB, here:
Fedora10 VM

Please drop me a line and let me know how it goes..love to hear from you.

Good luck,
The Mule

the dark of winter has me in its grasp

The Mule has been working long hours for himself and you, valued video compatriots!

That sounds positive, as it should be. Though in truth, I am feeling less positive than that message implies. Personal and professional life has got me down, but is par for the course these days. Oh well. A pithy quote to pick myself up would be rather nice here. Instead, let me regale you of the past weeks activities, as some of the tribulations may help individuals in similar need.

Sh*t Storm
This week, as I look back at my notes, I see a hailstorm of problems that I've dealt with:
-Fedora 10, x86-64 spontaneous system lockups/reboots (workaround: noapic on kernel cmd line)
-pulseaudio screwing up my audio
-usb keyboard stops working (workaround: disable keyboard acceleration)
-Gnome session saving broken (the workaround seems more of a pain than its worth)
-1080p editing eats RAM! (bought more RAM)
-Belkin firewire card causing reboots
-I didn't order my RAM in matched pairs, so I'm stuck waiting until Monday for RAM! (finally got it!)
-Evolution has trouble fetching mail from Comcast's POP servers, so I've reverted to use Pine (now "Alpine")

Needless to say, my productivity dropped and frustration was running high.

The Good News
Knock on wood, I think I was able to workaround the spontaneous reboots using "noapic" boot option to the kernel. Whereas the box was rebooting every six hours, now it has been up a full two days without a reboot! Of course, this isn't a true fix and I will have to submit a bug to the Fedora team. And the other problems still exist.

Most importantly, I've discovered a new scheme for solid, fast 1080P editing in Cinelerra:
1) convert Canon 5D video to MPEG2-TS
2) import into Cinelerra
3) render to any format you need

A Couple of Options
In my initial post on editing Canon 5D video, I found that the easiest way for me to get content from the Canon 5D into Cinelerra was using a conversion to MJPEG. However, the drawback with using mjpeg is that the image quality is lacking. Specifically, the output is darker than the original content. So over the past week, I found two solutions to convert the beautiful output of the Canon:

1) convert to H264 using this two pass string:
#CONVERT CANON USING H264, pass 1
ffmpeg -y -i INPUT.MOV -an -v 1 -threads 8 -vcodec libx264 -aspect 1.7777 -b 9000k -bt 7775k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 1 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 10000k -bufsize 2M -cmp 1 -f mp4 -pass 1 /dev/null

#CONVERT CANON USING H264, pass 2
ffmpeg -y -i INPUT.MOV -v 1 -threads 8 -vcodec libx264 -aspect 1.7777 -b 9000k -bt 7775k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 1 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 10000k -bufsize 2M -acodec libfaac -ab 160k -ar 48000 -ac 2 -cmp 1 -f mp4 -pass 2 OUTPUT.mp4

Now, this H264 content is beautiful, will import into Cinelerra and is editable. However, I found that when I went to render the final output, four minutes of the 1080p, H264 content took SIX HOURS to render!! That is unacceptable. I believe the lengthy render time has something to do with the color space or internal conversion that Cinelerra is doing. This bears further research.

If you're not familiar with H264 (x264 libraries on Linux), here's some useful H264 reference material.

2) convert to MPEG2-TS

Converting Canon to 1080p, MPEG2-TS
Now, there are a few steps here.

a. Take a file from the Canon and use ffmpeg to pass a lossless yuv4mpegpipe stream into mpeg2enc, with the result a video stream with no audio:
ffmpeg -i INPUT.MOV -threads 8 -s 1920x1088 -f yuv4mpegpipe - | mpeg2enc --multi-thread 8 --verbose 0 --aspect 3 --format 13 --frame-rate 5 --video-bitrate 24000 --nonvideo-bitrate 384 --interlace-mode 0 --force-b-b-p --video-buffer 448 --video-norm n --keep-hf --no-constraints --sequence-header-every-gop --min-gop-size 6 --max-gop-size 6 -o OUTPUT.m2v

Next, render out the audio:
ffmpeg -y -i INPUT.MOV -acodec mp2 -ar 44100 -ab 256k -ac 2 OUTPUT.m2a

Using mplex, mux the video and audio streams together:
mplex -f 3 -b 2000 OUTPUT.m2a OUTPUT.m2v -o OUTPUT.ps

Using VLC, convert the MPEG2-PS into an MPEG2-TS:
cvlc OUTPUT.ps --sout '#duplicate{dst=std{access=file,mux=ts,dst="OUTPUT.m2t"}}' vlc://quit

Update 2009/02/13
I've found that VLC is not writing proper keyframes at the beginning of the converted MPEG-PS video output from mplex. This is only for 1080p video. The VLC command for 720p video still works. For the 1080p, I've found a workaround using our savior, ffmpeg:
ffmpeg -y -i OUTPUT.ps -acodec copy -f mpegts -qscale 1 OUTPUT.m2t
*** end update ***

I used this method to output a new version of my Water video from Cinelerra to Vimeo here:
/2009/01/water-new-canon-5d-video.html

The quality and the colors are definetly improved upon over the old version. However, the larger file size is a drawback (479MB for 4m16s of video). So I'd like to get the H264 output without compression artifacts during the scenes with a lot of motion. So now its time to figure that out. Erg.

In general though, I think this is some good news!

Until the next time,
the mule

Cinelerra render compatibility on Fedora 10, x86-64

I spent the last four days building a new Fedora 10, x86-64 server for video editing based on programs available through the RPM Fusion repository. I'll post something soon about that whole process, which required more work and frustration than expected. However, my primary goal was to create a flexible and solid Cinelerra video editing rig.

Update 2009/02/15
Here's a post on how to build Cinelerra from source on Fedora 10, x86-64
*** end update ***

Testing the New System
In order to verify my new build, I needed to test high quality, rendered output from Cinelerra. More specifically, I wanted to test formats that required no external multiplexing (muxing). This will save me time in the future.

Assumptions
Also, note the test will be conducted against a larger resolution, 720P project. This will influence the playback ability of some of the programs.

Testing Procedure
During the testing phase, I was able to put Cinelerra's batch processing feature to use. This is very nice if you need to render large masses of files. Especially nice if you want to repeat or standardize that process across different Linux distributions.

Thinking more about this, it might be a boon to the cvs.cinelerra.org community if I made the XML of the batch list available to the community. That way, we could standardize the testing of output on different Linux distributions. At the end of the day, we could say things like "mplayer on SUSE 11 doesn't playback a 720P Quicktime rendered from Cinelerra that uses MPEG4 video compression with AAC audio encoding. However, that same file does work in mplayer on Fedora 10." So, it would be helpful to the QA process. I will bring this up as an idea to the community. Here is my latest effort at making the batch render feature part of the Cinelerra testing process:
http://crazedmuleproductions.blogspot.com/2010/01/batch-render-redux.html

Source File
The original video is 21 seconds long and is in 720P format, 1280x720 at 29.97fps.

In order to test the new operating system, I exported various combinations of audio and video formats, choosing fixed bitrates that seemed appropriate given the resolution of the source project:


I tested the output in standard Linux media players (vlc/mplayer/ffplay/totem), as well as reimported them into Cinelerra.

Results
In order to make it easy for people to comprehend the results, I've put together this graphic:

Using a stop light analogy, green means that the file plays with no problem. So, if you want to know the "best" or "good", compatible formats to use in Linux media players and Cinelerra, go for the green. Red means the file doesn't play at all. And yellow means that the file plays, but plays with some problem. For instance, the audio may be out of sync or stutters, or the video plays but maybe at the wrong resolution. Be advised that my system is fairly powerful, so your mileage may vary:
* Dell SC1430, dual quad core, 1.6Ghz, 2GB RAM
* RAID0 sys/working drive, RAID1 storage drive
* NVidia 8800GT video card

Observations
JPEG and MPEG-4 Quicktimes seem to work well and have the side benefit of being re-usable within Cinelerra. MPEG-4 rendering is unbelievably quick. H264 direct out of Cinelerra seems not to work at all in most cases. There was a tie for the file type with the widest compatibility among the chosen media players: Ogg/Vorbis and surprisingly, Microsoft MPEG4 using twos complement audio.

It is interesting to note the varying file sizes of the output from Cinelerra: Besides the uncompressed formats, Motion JPEG A creates the largest file sizes, while either of the MPEG4 formats are the most space efficient. Given the chart above, this is an important data point that video editors will be able to consider when selecting a particular format.

Conclusion
Though it is powerful, Cinelerra has been somewhat crippled by new users' ability to get usable content out of Cinelerra. I think this chart will improve that situation, even if it has only been proven out on Fedora 10, 64-bit. In that light, it would be interesting to see the same chart from an alternate Linux distribution. With my source Cinelerra and batch process EDL files, it should be easy to replicate the procedure on another system. I'd be happy to share them if an interested party wants to drop me a line.

Finally, I realize there are some missing file types like avi container, MPEG2 program and transport stream formats. The MPEG2 formates generally require external muxing and thus, were off the list for this test. However, I will try to test the at some point.

enjoy,
The Mule