Saturday, September 29, 2007

first impressions: Cinelerra on dual, quad core Dell SC1430

I'm on my dual, new quad-core, 64-bit FC6 box (woah!) and I'm happy to say that I've got Cinelerra running, with some minor speed bumps along the way (read /2007/09/building-cinelerra-on-fc6-64-bit.html and
http://www.techanswerguy.com/2007/09/setting-up-dell-sc1430.html

Update 2008/02/03
Here are some related articles:
/2007/10/multithreading-in-ffmpeg-and-mpstat.html
/2007/10/rendering-on-dual-quad-core.html
*** end update ***

Eight Core Power!
I see from the output of /proc/cpuinfo, that I've got eight processors (first and last listed below):

[root@localhost ~]# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU E5310 @ 1.60GHz
stepping : 7
cpu MHz : 1595.929
cache size : 4096 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
bogomips : 3195.17
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
...
processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU E5310 @ 1.60GHz
stepping : 7
cpu MHz : 1595.929
cache size : 4096 KB
physical id : 1
siblings : 4
core id : 3
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx tm2 cx16 xtpr lahf_lm
bogomips : 3191.96
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:


HDV Render Test: Project with no effect
Now that I've got her up, I'm testing a render of 40 minutes of MPEG-TS HDV content. While rendering, it seems that my CPUs are sloughin about..only about 30% busy. What is going on here? Shouldn't all the CPUs be working at 100%?


Here is the output of iostat, showing 70% idle CPUs:
Time: 12:14:23 AM
avg-cpu: %user %nice %system %iowait %steal %idle
27.75 0.00 0.60 1.00 0.00 70.65

Time: 12:14:28 AM
avg-cpu: %user %nice %system %iowait %steal %idle
27.35 0.00 0.37 0.95 0.00 71.33


Top seems a little funny too, whereby the third line down shows 27.4%, near what iostat reports, but then the listed CPU utilization for Cinelerra is about 220 in the process list below:

top - 00:15:42 up 46 min, 4 users, load average: 2.83, 2.35, 1.82
Tasks: 186 total, 2 running, 184 sleeping, 0 stopped, 0 zombie
Cpu(s): 27.4%us, 0.4%sy, 0.0%ni, 70.9%id, 1.2%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 2055048k total, 2038468k used, 16580k free, 736k buffers
Swap: 4562440k total, 632k used, 4561808k free, 514344k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5640 root 17 0 3170m 1.0g 45m S 220 53.2 41:39.23 cinelerra
3540 root 15 0 183m 89m 34m S 1 4.4 6:43.95 Xorg


I will investigate that inconsistency later.

The render finished in about 50 minutes, so that's a little bit longer than Heroine Warrior's assertion that encoding HD content with a 64-bit system should be in realtime:
http://www.heroinewarrior.com/cinelerra7.php3

HDV Render Test 2: Project with Oil Painting effect
So then I ran a Cinelerra project with a more computationally intense render (one with an Oil Painting effect applied to an HDV track), CPU utilization is now up to 80-90% per cpu (attached graphic):


Here's the test video:


Analysis
There are many cores and they are relatively low speed (1.6Ghz). As Cinelerra takes advantage of all the processors, each CPU is working on a task (the render) and those tasks process at the speed of each corresponding CPU. In other words, as fast as each individual processor can handle. So simple tasks, like rendering raw HDV video with no effects, don't take up much individual CPU. However, more computationally intense tasks, like adding the oil painting effect to a video, utilize more of each CPU. Thus, the video rendered with the oil paint effect takes more CPU horsepower than the simple, no effect render.

This server is kind of like having a big dump truck. The thing has a powerful engine and can haul a ton of stuff, but can't go very fast. Unlike a Ferrari that can't handle much luggage, but can go super fast. I think we might see that a box with faster CPUs would get simple renders done quicker than what my Dell can do, but it might choke on renders with lots of complex effects.

Now if I could only harness these CPUs for speed! I'd be interested in getting people's opinions on 64-bit and multicore operation, as this is my first multicore/64 system.

Other Observations on 64-bit Fedora Core 6
1) I see that command-line ffmpeg is not multithreaded for multi processing machines. I will have see if I can use the external ffmpeg compile option in Cinelerra.
2) I can't get an NVidia card to work in this box. Much pain listed here:
http://forums.anandtech.com/messageview.aspx?catid=31&threadid=2080542
3) The Adobe Flash plugin only works in 32-bit mode Firefox

more to come,
the crazed mule

No comments: