Showing posts with label compile. Show all posts
Showing posts with label compile. Show all posts

Friday, July 16, 2010

Green Beret parachutes in to help Cinelerra

Unbelievably, just in the past week, the Cinelerra CV community (http://cvs.cinelerra.org/) has had a heavyweight developer parachute in and start fixing stuff all over the place. The guy is Chris "Monty" Montgomery who developed the Ogg multimedia container and Vorbis audio formats (http://en.wikipedia.org/wiki/Chris_Montgomery). He is a developer currently working for Red Hat.

Monty, You're The Man!
His primary effort has been to yank out the kludgy and incompatible file loader in Cinelerra and replace it using good old FFmpeg. FFmpeg has problems of its own, but a helluva lot less problems than the Cinelerra loader. This is a task that the original developer of Cinelerra, Heroine Warrior (http://www.heroinewarrior.com/), has wisely done with the 4.1 version of Cinelerra. Much props to Father HV, for without him, this blog and Open Source video editing would be very different, if exist at all.

Monty is patching the version of FFmpeg that comes with the Cinelerra CV source code (http://cinelerra.org/getting_cinelerra.php). In Cinelerra CV's source code tree, FFmpeg is found under quicktime/ffmpeg. This means that once you compile Cinelerra from source code and add Monty's code fixes, files like the Canon 5D/7D H264 AVC files (the 1080P files spat out of the Canons') will load natively in Cinelerra.

AVC, oh AVC, Why Are You So Stubborn?
Problem here is that you can't really edit with AVC files because the decoding of them is so bloody CPU intensive. AVC playback in Cinelerra on my dual, quad core Dell SC1430 with 10GB RAM is 3-4fps. This is unacceptable for previewing edits. There is a workaround to this such that if you really wanted to edit AVC, you'd need to have background rendering enabled, make your edits and then wait for the background render to finish. But that bg render takes a long time too. You could speed this up to by creating a ramdisk (http://www.techanswerguy.com/2009/02/creating-ramdisk-in-linux.html) and setting the background render to use that ramdisk. But that's a whole other task.

Transcoding Canon 5D/7D Files, as usual
Thus, we're still left with the task of transcoding Canon 5D/7D files into a more compatible format. There are two solutions:
1) get my varied MPEG-TS conversion magic to work for you (http://crazedmuleproductions.blogspot.com/2009/02/dark-of-winter-has-me-in-its-grasp.html)
2) convert the 5D/7D files using FFmpeg into another format that works since FFmpeg is now Cinelerra's loader (with Monty's additions)

Both require research/work, but I did a quick conversion that I tested in Cinelerra with Monty's code additions:
ffmpeg -i [inputFileFromCanon] -acodec mp2 -ar 48000 -ab 384k -ac 2 -vcodec mpeg2video -qscale 1 [outputFile]

The audio format is mpeg, layer II and the video format is mpeg2video. This loads in Cinelerra and is fast to edit with. Once edited, render out from Cinelerra as usual.

How to Compile Cinelerra for Ubuntu and Fedora users
So Monty's additions require compiling from source. I'll provide some quick links here. For Ubuntu folks, Raffaella Traniello has put a nice post together on how to compile Cinelerra in Ubuntu:
http://www.g-raffa.eu/Cinelerra/HOWTO/compilation.html

For Fedora users, I have a post here:
http://crazedmuleproductions.blogspot.com/2010/02/fedora-12-x86-64-cinelerra-install.html

Get Monty's Additions Working
Once you get the source code compiled, install it (make install) and just test that Cinelerra works at a basic level. Once you've tested that your compile has worked, move onto apply Monty's code additions:
- remove your installed Cinelerra (make uninstall)
- download Monty's additions at http://people.xiph.org/~xiphmont/cinelerra/patches/
- apply each of them to the Cinelerra source code tree one at a time (git apply [patch name])
or join in Monty's git branch:
1) git remote add xiphmont git://git.xiph.org/users/xiphmont/cinelerraCV.git
2) git remote update
- compile Cinelerra, this time with all Monty's additions (make clean;make;make install)

After starting CinCV with Monty's additions, you should see the startup info change:
[mule@ogre my_cinelerra]$ cinelerra
Cinelerra 2.1CVxiphmont
GIT::8c01c13bbb21d30aff2733454a49fbfe641e51ca
(C) 2006 Heroine Virtual Ltd.
(C) 2006-2010 The CinelerraCV Community
Internal ffmpeg 0.6+fixes
Compiled on Fri Jul 16 20:40:38 EDT 2010


As well, you should see that FFmpeg becomes the default loader for videos in you goto the Resources window, right-click on a video and click "Info". You should see FFMPEG there under File Format:


A Few Git Notes..
While I'm at it..

View current git branches
[mule@ogre my_cinelerra]$ git branch
* master

Add Monty's remote branch
[mule@ogre my_cinelerra]$ git remote add xiphmont git://git.xiph.org/users/xiphmont/cinelerraCV.git

Update the local copy of Monty's remote branch
[mule@ogre my_cinelerra]$ git remote update
Fetching origin
Fetching xiphmont
remote: Counting objects: 2754, done.
remote: Compressing objects: 100% (1904/1904), done.
remote: Total 1981 (delta 997), reused 0 (delta 0)
Receiving objects: 100% (1981/1981), 2.77 MiB 1.07 MiB/s, done.
Resolving deltas: 100% (997/997), completed with 711 local objects.
From git://git.xiph.org/users/xiphmont/cinelerraCV
* [new branch] master -> xiphmont/master


Check that Monty's branch has been added
[mule@ogre my_cinelerra]$ git branch
* master
xiphmont

I then was able to switch branches
[mule@ogre my_cinelerra]$ git checkout -b xiphmont xiphmont/master
Branch xiphmont set up to track remote branch master from xiphmont.
Switched to a new branch 'xiphmont'


"git branch" shows that I have switched to Monty's branch
[mule@ogre my_cinelerra]$ git branch
master
* xiphmont

Now I can switch easily between branches (Monty's and CinelerraCV)
[mule@ogre my_cinelerra]$ git checkout master
Switched to branch 'master'


Switching to Monty's branch
[mule@ogre my_cinelerra]$ git checkout xiphmont
Switched to branch 'xiphmont'


Show the current status of the git code repositories
[sfrase@ogre my_cinelerra]$ git status | head -6
# On branch xiphmont
# Untracked files:
# (use "git add ..." to include in what will be committed)
#
# cinelerra-cvs-current.spec
# cinelerra/.deps/
>



View the latest change log
[mule@ogre my_cinelerra]$ git log xiphmont/master | head -6
commit 8c01c13bbb21d30aff2733454a49fbfe641e51ca
Author: Monty
Date: Fri Jul 16 07:30:10 2010 -0400

iAlter/extend versioning to avoid any confusion between this temporarily
forked version and the main CinelerraCV repo

Pull in the latest code updates:
[mule@ogre my_cinelerra]$ git pull


Troubleshooting
I tried to switch branches and got this error
[mule@ogre my_cinelerra]$ git checkout -b xiphmont xiphmont/master
error: You have local changes to 'po/de.po'; cannot switch branches.

I did a hard reset to clear the above error
[mule@ogre my_cinelerra]$ git reset --hard HEAD
HEAD is now at d95887f Pass --libdir given to configure on to ffmpeg configure.

Last resort for repos not working, a destructive clean:
[mule@ogre my_cinelerra]$ git clean -dfx



Update 2010/07/16
Looks like the YUV4MPEG render stream is broken..this simple render pipe outputs a file unreadable in FF
ffmpeg -f yuv4mpegpipe -i - -y -target dvd -f mpeg2video %

I talked to Monty and he will fix..something about colorspace converter.
*** end update ***

Note that you have to compile Cinelerra and do a full reinstall, as the Cinelerra "make" process hard links the installation directory into the cinelerra executable.

So I hope Monty stays with us as long as possible. Thanks man! Hope springs eternal!
the mule

References
http://www.itworld.com/open-source/115063/breaking-open-video-frontier-despite-mpeg-la

Sunday, January 24, 2010

compile time performance improved!

I was compiling Cinelerra today and noticed that CPU usage was very low during the compile..around 10-15% utilized. I have a dual CPU, quad core box. This makes for a total of eight processors. So with all those CPUs, I figured there must be a way to make compiling faster.

Actually, this low CPU use during compiles was something I had noticed the first time I installed Cinelerra. Ashamedly, I've forgotten to investigate this issue in the two years that I've had the box. So today I googled for "make compiler see multiple CPUs" and found the -j switch to "make" the program that does the compiling:
http://blogs.koolwal.net/2009/04/20/tip-compile-your-programs-fasters-with-multiple-processor-machines/

This article also mentions the CONCURRENCY_LEVEL environment variable, but that variable did not work for my box, a Dell SC1430. So I used the -j switch to make instead:
[mule@ogre my_cinelerra]$ make -j7
(CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /home/sfrase/my_cinelerra/missing --run autoheader)
rm -f stamp-h1


Here are my results.

the time it took to compile Cinelerra normally, without -j:
3min 45s
the time it took to compile Cinelerra with the -j8 (for eight cores):
1min 16s

Holy crap! That's a 300% speed improvement!

Glad I finally researched this.
scott

ps - one other note: 7z is a multithreaded version of tar. On Fedora, use 7za
Installing : p7zip-4.65-2.fc12.x86_64
7za - A file archiver with highest compression ratio

SYNOPSIS
7za [adeltux] [-] [SWITCH]


Related Posts
http://crazedmuleproductions.blogspot.com/2007/10/multithreading-in-ffmpeg-and-mpstat.html

Saturday, May 30, 2009

Cinelerra 4 just not working

I was interested in trying out Cinelerra 4, Heroine Warrior's latest version of Cinelerra. To reiterate, there are two versions of Cinelerra available:
Heroine Warrior's, the original coded by Adam Williams (bow down to the man!) and
Cinelerra Community Version, Cinelerra CV

On my Fedora 10, x86-64 setup, I gave the latest HV Cinelerra 4 version a try. I encountered a bunch of hurdles, mainly the GCC 4.3 changes that caused missing header information in mjpegtools-1.9.0_rc3 listed here:
http://bugs.gentoo.org/show_bug.cgi?id=200767

I got the software to compile. At the final "make install" step, the installation starts, but does not complete with only the first 40 or so lines of the install finishing:
[mule@ogre bin]# make install
make -f build/Makefile.cinelerra install
make[1]: Entering directory `/usr/src/cinelerra-4'
make -C plugins install
make[2]: Entering directory `/usr/src/cinelerra-4/plugins'
mkdir -p ../bin/fonts
cp fonts/* ../bin/fonts
mkdir -p ../bin/shapes
cp shapes/* ../bin/shapes
cp ../thirdparty/mjpegtools*/mpeg2enc/mpeg2enc ../bin/mpeg2enc.plugin
make[2]: Leaving directory `/usr/src/cinelerra-4/plugins'
DST=../bin make -C libmpeg3 install
make[2]: Entering directory `/usr/src/cinelerra-4/libmpeg3'
cp x86_64/mpeg3dump x86_64/mpeg3peek x86_64/mpeg3toc
x86_64/mpeg3cat ../bin
make[2]: Leaving directory `/usr/src/cinelerra-4/libmpeg3'
make -C po install
make[2]: Entering directory `/usr/src/cinelerra-4/po'
mkdir -p ../bin/locale/de/LC_MESSAGES

cp sl.mo ../bin/locale/sl/LC_MESSAGES/cinelerra.mo
make[2]: Leaving directory `/usr/src/cinelerra-4/po'
make -C doc install
make[2]: Entering directory `/usr/src/cinelerra-4/doc'
mkdir -p ../bin/doc
cp arrow.png autokeyframe.png camera.png channel.png crop.png cut.png
expandpatch_checked.png eyedrop.png fitautos.png ibeam.png
left_justify.png magnify.png mask.png mutepatch_up.png paste.png
projector.png protect.png record.png recordpatch_up.png rewind.png
singleframe.png show_meters.png titlesafe.png toolwindow.png
top_justify.png wrench.png magnify.png ../bin/doc
cp: warning: source file `magnify.png' specified more than once
cp cinelerra.html ../bin/doc
make[2]: Leaving directory `/usr/src/cinelerra-4/doc'
cp COPYING README bin
make[1]: Leaving directory `/usr/src/cinelerra-4'
[mule@ogre bin]#


Therefore, the installation does not copy the cinelerra binary into /usr/local/bin. If I try to run the binary from the source code directory, I get this:
PluginServer::open_plugin: /usr/src/cinelerra-4/bin/brightness.plugin:
undefined symbol: glUseProgram
PluginServer::open_plugin: /usr/src/cinelerra-4/bin/deinterlace.plugin:
undefined symbol: glUseProgram

undefined symbol: glNormal3f
PluginServer::open_plugin: /usr/src/cinelerra-4/bin/swapchannels.plugin:
undefined symbol: glUseProgram
PluginServer::open_plugin: /usr/src/cinelerra-4/bin/threshold.plugin:
undefined symbol: glUseProgram
PluginServer::open_plugin: /usr/src/cinelerra-4/bin/zoomblur.plugin:
undefined symbol: glEnd
signal_entry: got SIGSEGV my pid=3965 execution table size=16:
awindowgui.C: create_objects: 433
awindowgui.C: create_objects: 440
awindowgui.C: create_objects: 444
awindowgui.C: create_objects: 447
awindowgui.C: create_objects: 453
suv.C: get_cwindow_sizes: 744
suv.C: get_cwindow_sizes: 774
suv.C: get_cwindow_sizes: 800
suv.C: get_cwindow_sizes: 821
editpanel.C: create_buttons: 177
editpanel.C: create_buttons: 303
editpanel.C: create_buttons: 177
editpanel.C: create_buttons: 303
mwindowgui.C: create_objects: 192
mwindowgui.C: create_objects: 195
mwindowgui.C: create_objects: 199
signal_entry: lock table size=6
0x357dc80 RemoveThread::input_lock RemoveThread::run
0x64d0370 CWindowTool::input_lock CWindowTool::run
0x64f3640 TransportQue::output_lock PlaybackEngine::run
0x3441940 TransportQue::output_lock PlaybackEngine::run
0x3442420 MainIndexes::input_lock MainIndexes::run 1
0x3442f80 Cinelerra: Program MWindow::init_gui *
BC_Signals::dump_buffers: buffer table size=0
BC_Signals::delete_temps: deleting 0 temp files
SigHandler::signal_handler total files=0



Even though I have an NVidia graphics card, the error lines above were related to OpenGL. Thus, I thought I might have better luck compiling without OpenGL enabled. After I removed those lines from from hvirtual_config.h, I did a make clean;make. This time around, I was able to get Cinelerra 4 to start properly. Though, it soon locks up when viewing my 720P MPEG-TS files:
[mule@ogre cinelerra-4]$ ./bin/cinelerra
Cinelerra 4 (C)2008 Adam Williams

Cinelerra is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. There is absolutely no warranty for Cinelerra.
[mpeg2video @ 0xeafd00]slice mismatch
[mpeg2video @ 0xeafd00]mb incr damaged
[mpeg2video @ 0xeafd00]mb incr damaged
[mpeg2video @ 0xeafd00]invalid cbp at 14 37
[mpeg2video @ 0xeafd00]slice mismatch
[mpeg2video @ 0xeafd00]ac-tex damaged at 25 40
[mpeg2video @ 0xeafd00]ac-tex damaged at 6 41
[mpeg2video @ 0xeafd00]ac-tex damaged at 7 42


So much for that experiment! I'm going back to the CV version for now.
the mule