Wednesday, January 27, 2010

batch render redux

As a follow up to this original post, I thought I'd give a few hints on working with the batch render function in Cinelerra. I've been using the batch render to prepare me for my new Fedora 12, x86-64 system. Specifically, I am using batch render to profile the capacity and speed of my current Fedora 10 system. After I install the new Fedora 12 system, I can then use the Fedora 10 performance baseline to tell me how much faster (or slower) the new system is in relation to the old system.

Selecting a Cinelerra Project
The first thing I did was use a short (~1 min) clip of a project that I had been working on:


I then used this project to output a short clip as the basis for the batch render:


Batch render is accessed by typing Shift-B within Cinelerra:


You can save a batch render list to XML format. This format is similar to the XML of the edit decision list (EDL) that Cinelerra stores when a project is saved. You can then load that XML to use later.

Batch Render Gotchas
I used a previous batch list to render out that short clip to about twenty different file formats. The batch render blew up a few times, so I had to get over a few obstacles:
1) the batch list I had saved months ago was out of date and the directory pointers in the XML were incorrect. I fixed the incorrect pointers by going into the vi text editor and doing a wholesale conversion with a sed construct:
:1,$ s/videos\/oldpath/videos\/newpath/g

1,$ says look for and make replacements on all lines. s means "search for.." The matching expression looks for the string "videos/oldpath" in the file and replaces it with "videos/newpath". g means "do the replacement globally"

2) my project had an improperly formatted Quicktime video track on the timeline. I kept getting Quicktime errors when the batch ran, which would crash Cinelerra. Once I removed that errant track, the batch render worked correctly.

3) once I got the batch running, my disk would fill up quickly as I was rendering to a few uncompressed formats. I deleted some very large, extraneous files and the batch was able to complete.

So you can see that you need to prepare both the base project and your system if you expect your batch render to run properly.

Command Line Batching
Once you get your batch working, another nice feature of batch rendering you can take advantage of is the ability of Cinelerra to do this at a command line. I just kicked off a batch job at the command line and see that it does work on a box with X installed. Nicely, it also give you an ETA:

[sodo@tbear ~]$ cinelerra -r /mnt/videos/cinelerra/batch/batchList.xml
/mnt/videos/cinelerra/batch/renderCompatibility.xml
Cinelerra 2.1CV (C) 2006 Heroine Virtual Ltd.
Compiled on Sat Jan 23 01:32:17 EST 2010

Cinelerra is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. There is absolutely no warranty for Cinelerra.
Render::run: /mnt/videos/cinelerra/batch/renderCompatibility.xml
Render::render: starting render farm
[mpeg4 @ 0x7ff313d3ba00]warning: first frame is no keyframe
[mpeg4 @ 0x7ff313d3ba00]concealing 1 DC, 1 AC, 1 MV errors
12% ETA: 0:07:45



Batch Results
So my goal was to profile my current system's capacity by capturing the system's cpu and disk utilization while the files rendered. To really see what is going on though, I thought it might be nice to have a graphical representation of the render as it occurs over time. So I spent some time writing a gnuplot script to plot the system utilization as the files rendered:

A bit of explanation may be required. I captured the output of vmstat to a file. VMstat has CPU load and wait i/o (disk utilization) statistics. While capturing that output, I kicked off the batch render.

In the graphic, you can see that different types of renders have different utilization profiles. For example, the mpeg4 renders were generally lower in CPU utilization (red line), while h264 renders used a lot of CPU. Similarly, the uncompressed formats like rgb/rgba/yuv420planar stress out the disk quite a bit (green line). Please excuse the fact that the filenames aren't perfectly lined up with each file's render profile..this was my first effort at graphing render times.

It will be interesting to see how the new Fedora 12 install affects the CPU. Also, I am planning on installing a new hardware RAID set, so I expect those green lines to go to zero (hopefully)!

ciao,
the mule

No comments: