Saturday, September 29, 2007

first impressions: Cinelerra on dual, quad core Dell SC1430

I'm on my dual, new quad-core, 64-bit FC6 box (woah!) and I'm happy to say that I've got Cinelerra running, with some minor speed bumps along the way (read /2007/09/building-cinelerra-on-fc6-64-bit.html and
http://www.techanswerguy.com/2007/09/setting-up-dell-sc1430.html

Update 2008/02/03
Here are some related articles:
/2007/10/multithreading-in-ffmpeg-and-mpstat.html
/2007/10/rendering-on-dual-quad-core.html
*** end update ***

Eight Core Power!
I see from the output of /proc/cpuinfo, that I've got eight processors (first and last listed below):

[root@localhost ~]# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU E5310 @ 1.60GHz
stepping : 7
cpu MHz : 1595.929
cache size : 4096 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
bogomips : 3195.17
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
...
processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU E5310 @ 1.60GHz
stepping : 7
cpu MHz : 1595.929
cache size : 4096 KB
physical id : 1
siblings : 4
core id : 3
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
bogomips : 3191.96
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:


HDV Render Test: Project with no effect
Now that I've got her up, I'm testing a render of 40 minutes of MPEG-TS HDV content. While rendering, it seems that my CPUs are sloughin about..only about 30% busy. What is going on here? Shouldn't all the CPUs be working at 100%?


Here is the output of iostat, showing 70% idle CPUs:
Time: 12:14:23 AM
avg-cpu: %user %nice %system %iowait %steal %idle
27.75 0.00 0.60 1.00 0.00 70.65

Time: 12:14:28 AM
avg-cpu: %user %nice %system %iowait %steal %idle
27.35 0.00 0.37 0.95 0.00 71.33


Top seems a little funny too, whereby the third line down shows 27.4%, near what iostat reports, but then the listed CPU utilization for Cinelerra is about 220 in the process list below:

top - 00:15:42 up 46 min, 4 users, load average: 2.83, 2.35, 1.82
Tasks: 186 total, 2 running, 184 sleeping, 0 stopped, 0 zombie
Cpu(s): 27.4%us, 0.4%sy, 0.0%ni, 70.9%id, 1.2%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 2055048k total, 2038468k used, 16580k free, 736k buffers
Swap: 4562440k total, 632k used, 4561808k free, 514344k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5640 root 17 0 3170m 1.0g 45m S 220 53.2 41:39.23 cinelerra
3540 root 15 0 183m 89m 34m S 1 4.4 6:43.95 Xorg


I will investigate that inconsistency later.

The render finished in about 50 minutes, so that's a little bit longer than Heroine Warrior's assertion that encoding HD content with a 64-bit system should be in realtime:
http://www.heroinewarrior.com/cinelerra7.php3

HDV Render Test 2: Project with Oil Painting effect
So then I ran a Cinelerra project with a more computationally intense render (one with an Oil Painting effect applied to an HDV track), CPU utilization is now up to 80-90% per cpu (attached graphic):


Here's the test video:


Analysis
There are many cores and they are relatively low speed (1.6Ghz). As Cinelerra takes advantage of all the processors, each CPU is working on a task (the render) and those tasks process at the speed of each corresponding CPU. In other words, as fast as each individual processor can handle. So simple tasks, like rendering raw HDV video with no effects, don't take up much individual CPU. However, more computationally intense tasks, like adding the oil painting effect to a video, utilize more of each CPU. Thus, the video rendered with the oil paint effect takes more CPU horsepower than the simple, no effect render.

This server is kind of like having a big dump truck. The thing has a powerful engine and can haul a ton of stuff, but can't go very fast. Unlike a Ferrari that can't handle much luggage, but can go super fast. I think we might see that a box with faster CPUs would get simple renders done quicker than what my Dell can do, but it might choke on renders with lots of complex effects.

Now if I could only harness these CPUs for speed! I'd be interested in getting people's opinions on 64-bit and multicore operation, as this is my first multicore/64 system.

Other Observations on 64-bit Fedora Core 6
1) I see that command-line ffmpeg is not multithreaded for multi processing machines. I will have see if I can use the external ffmpeg compile option in Cinelerra.
2) I can't get an NVidia card to work in this box. Much pain listed here:
http://forums.anandtech.com/messageview.aspx?catid=31&threadid=2080542
3) The Adobe Flash plugin only works in 32-bit mode Firefox

more to come,
the crazed mule

Thursday, September 27, 2007

building Cinelerra on FC6 64-bit

Update 10/26/2007:
These instructions work well for F7 (Fedora 7), 64-bit, too!


Update 10/29/2008:
These instructions still kick ass..a year later when I blew up my system!


Update 11/20/2008:
I've rebuilt my Fedora 7 system as Fedora 9. The Cinelerra build instructions are different than the Fedora 7 instructions in that they use ATrpms repository. But all-in-all, the install went well!

I just purchased a Dell SC1430 dual Xeon, quad-core server to replace my aging Dell SC400 tower. I will give a full review of that server in a later post. More immediately however, I thought I'd give the 64-bit Fedora Core 6 installation a whirl, as these Xeons support 64-bit. I figured that Cinelerra should fly with 64-bit install! Also, to ease my pain, I figured I'd stay with FC6, as I'm well familiar with this distribution. Eventually, I'll build out a 64-bit Debian Lenny distro to see how that performs (thanks for the suggestion, Graham)!

Here's How Things Went
The Fedora install is the same installer you get with the 32-bit version and went very smoothly.

I then proceeded to install the dependencies for building Cinelerra from source. In a nutshell, the yum installation was very similar to my 32-bit FC6 install which I described here, with some minor differences:
/2007/03/fedora-core-6-cinelerra-dependencies.html

The quick steps are:
1) have the proper yum repositories online (all Fedora, Livna and Dries)
2) make sure you have your GPG-KEYs loaded from Livna and Dries
3) remove the dries repositories (with the --disablerepo=dries switch shown) and install using the script below with only Fedora and Livna online:
4) remove the Livna repositories, enable FreshRPMs and install mjpegtools using the command:
yum --disablerepo=livna install mjpegtools*
5) get Cinelerra source code (refer to previous post on those details)
6) build from source (refer to previous post on those details)
7) install your favorite multimedia applications

The only differences between the 32-bit install and the 64-bit install that I noticed were that I had to specify the install of fftw specifically on 64-bit:
1) fftw (instead of fftw3)
2) fftw-devel

I've amended the script below to reflect those differences. Another interesting note is that with FC6 and above, no SMP-specific kernel is necessary. This is good! Finally, I installed my favorite multimedia apps:
yum install avidemux mencoder mplayer transcode vlc* xmms

However, to avoid this error, you should install xine separately without Livna repos online:
yum --disablerepo=livna install xine

I haven't gotten to really test out Cinelerra's capabilities on 64-bit, but I did render out a 40 minute Cinelerra project with full 720P HDV in a little over an hour. I rendered using QT container, MPEG-4 video compression with a constant bit rate of 10,000,000 and twos complement audio. This *almost* achieves what Heroine Warrior states about being able to achieve real-time rendering with HDV. Obviously, I believe some performance tweaks are in order and will need to work through these issues.

Here is that yum script which should get you going on 64-bit Fedora Core 6:

The Cinelerra FC6 64-bit Source Dependencies Install Script
yum install --disablerepo=dries \
libquicktime \
ffmpeg \
gsm-devel \
xvidcore* \
lame \
lame-devel \
libvorbis* \
libogg* \
libtool* \
a52* \
libtheora* \
libpng* \
libjpeg* \
libtiff* \
esound* \
audiofile* \
libraw1394* \
libavc1394* \
freetype* \
fontconfig* \
nasm \
e2fsprogs* \
faad2-dev* \
OpenEXR* \
fftw \
fftw-devel \
libsndfile* \
libiec61883* \
x264 \
x264-d* \
faac* \
libdv*


Don't forget to chmod a+x the file!

cheers,
the mule

Friday, September 14, 2007

color space and color sampling reference

I was having a hard time finding information about the differences between the color spaces used in Cinelerra. Then I actually looked up some entries in Wikipedia:

Color Space (general)
http://en.wikipedia.org/wiki/Color_space

RGB color space
http://en.wikipedia.org/wiki/RGB_color_space

YCbCr color space
http://en.wikipedia.org/wiki/YCbCr

YUV color space
http://en.wikipedia.org/wiki/YUV

Also, here is an excellent article on understanding color sampling:
http://www.dvxuser.com/articles/colorspace/

YUV-RGB Conversion Formula

Wow. I finally understand color sampling!
cheers.

Monday, September 10, 2007

migrated my FC6 system to faster disk

When I was on Fedora Core 4, I had been using a fast, 250GB Western Digital SATA drive as my main root partition. Since I live in both XP and Fedora worlds, I divided up the drive into the following partitions:
/dev/sda1: XP boot (40GB)
/dev/sda2: Fedora Core 4 /boot (128MB)
/dev/sda3: Fedora Core 4 / (root) (120GB)
/dev/sda4: unused
/dev/sda5: swap (6GB)
/dev/sda6: FAT32 (to transfer files in between Linux and XP) (60GB)


This configuration allows me to dual boot XP and FC easily, as well as transfer files in between the two OSs. Speaking of NTFS-ext2/3 transfers, I'm slowing phasing out the FAT32 partition for transfers because I have had success using ntfs-3g (http://www.ntfs-3g.org) to write to and read from my NTFS partitions. NTFS-3g has been very solid for reading and writing from Linux to NTFS.

Earlier this year, I installed Fedora Core 6 as a development/test system on a separate 40GB drive. Doing the install this way instead of an upgrade of FC4 meant that I didn't have to cripple the FC4 system in case anything went wrong with FC6. As it happens, Cinelerra on FC6 happens to be the most stable Cinelerra I've used these past two years, apart from an audio sync issue on rendered videos. I correct the sync problem using the Nudge feature on each audio track, detailed here:
/2007/05/nudge-avidemux2-and-reminder-about.html

Lo these past six months, I have been editing on FC6, albeit on a slower IDE drive, a 7200RPM Barracuda. The 250GB SATA with 16MB memory cache is quite a bit faster. So the goal was to move the boot/root of the FC6 system to the boot/root of the faster SATA drive.

LVM versus ext3
The first issue I encountered was that the default installation of FC6 automatically creates a Volume Group and Logical Volumes instead of plain old ext2/ext3 filesystems. My FC4 system used ext3 for boot/root, so my original plan was to do a simple partimage backup of the two FC6 filesystems. That idea would not work, as FC6 was on LVM and partimage doesn't recognize LVM partitions.

As an experiment (and because I wanted to get up and running quickly), I simplified the plan to install the new drive in the machine, mount the FC4 filesystems on the fast drive, delete the content on them and replace the content with the FC6 system files. I'd have to then tweak fstab and grub.conf appropriately on the updated drive to make the whole thing work.

The Plan
1) install both drives in the same machine with FC6 as the primary system,
2) mount the FC4 partitions of the fast drive in FC6 (/mnt/fc4boot & /mnt/fc4root),
3) remove all the files off of the FC4 boot/root filesystems,
4) copy the files from the FC6 boot/root parts to the FC4 partitions,
5) carefully tweak grub.conf and fstab to reference the new partition scheme
6) hope for the best on reboot

Here is where we had "a few issues" as an old boss of mine would say.

Problems!
I guess it is my cursory knowledge of the boot process that got me into trouble. The first issue after performing these four steps was the following error:
mount: could not find filesystem '/dev/root'
setuproot: moving /dev failed: No such file or directory
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!


Ouch! That doesn't look good. Following the advice of this thread:
http://www.linuxquestions.org/questions/showthread.php?t=497332

First, using SysRescCD, I verified my partition labels on the fast drive using e2label:
e2label /dev/hdN
where N is the Fedora partition

I double-checked my grub.conf and fstab:

/boot/grub/grub.conf
Here's the old entry:
title Fedora Core (2.6.19-1.2911.fc6)
root (hd0,0)
kernel /vmlinuz-2.6.19-1.2911.fc6 ro root=/dev/VolGroup00/LogVol00
initrd /initrd-2.6.19-1.2911.fc6.img


Here's the new entry:
title Fedora Core (2.6.19-1.2911.fc6)
root (hd0,1)
kernel /vmlinuz-2.6.19-1.2911.fc6 ro root=LABEL=/
initrd /initrd-2.6.19-1.2911.fc6.img


Note the changes in bold that I needed to enter in order to move the root from the first partition on the older, slow drive to the second partition on the faster drive. Also, LVM goes away to be replaced by the simpler naming scheme of the ext3 formatted partition.

/etc/fstab
Secondly, the fstab changes as well, from:
/dev/VolGroup00/LogVol00 / ext3 defaults 1 1
/dev/VolGroup00/LogVol01 swap swap defaults 0 0


To:
LABEL=/ / ext3 defaults 1 1
/dev/sda5 swap swap defaults 0 0


The /boot entry stays the same:
LABEL=/boot /boot ext3 defaults 1 2

The output of e2label showed that my fast drive partition labels matched those in grub.conf and fstab. So no smoking gun there.

The next thing suggested in the thread was to create mkinitrd, the initial ram disk the Linux kernel uses for system files. Oh boy. I've forgotten about this one. I knew I had the correct partition labels, but I needed a little more background on mkinitrd and these articles helped:
http://www.faqs.org/docs/Linux-mini/Loopback-Root-FS.html
http://www.opennet.ru/docs/HOWTO/Kernel-HOWTO-11.html+creating+initrd
http://www.ibm.com/developerworks/linux/library/l-initrd.html

Redhat/Fedora has a nice utility called mkinitrd that simplifies many of the steps mentioned in the IBM and HOWTO articles above. I regenerated the initial ramdisk image on the new boot partition using the following command:
mkinitrd ./initrd-2.6.19-1.2911.fc6.img 2.6.19-1.2911.fc6

OK. There were no errors on creation and the file is there and has data:
-rw------- 1 root root 2398566 Sep 9 11:52 /boot/initrd-2.6.19-1.2911.fc6.img

Cool. Now for the moment of truth. The reboot! I powered down and waited for the system to come back up. Thankfully, the kernel got beyond the panic and booted properly! Hooray! However, I did see some errors still:
No volume groups found
Volume group "VolGroup00" not found
Unable to access resume device (/dev/VolGroup00/LogVol01)


But, the system loads fine and Cinelerra and all my apps seem to work. So I will have to research this error further.

Also, I saw when I started Evolution, I got the following error:
Error while storing folder 'Inbox'

Researching this, I saw that there were inconsistencies in the index files that Evolution creates. The workaround is simple: delete any mail folders in ~./evolution/mail/local with an extension of ev.~summary. These are indexing files only and you can safely remove them. Reference:
https://bugs.launchpad.net/ubuntu/+source/evolution/+bug/27014

For the past couple of days, I have been editing and Cinelerra is performing well. I'm glad I went through this exercise. Though, I really need to learn more about how to backup and restore LVM partitions. I might have saved myself some pain.

cheers!
The Mule