Fedora 9 Cinelerra deps, ATrpms note

Some people have complained of incompatibilities with Fedora and ATrpms. Instead, they recommend using the better supported RPMFusion, the repository that has merged all the other repositories for Fedora. This is a good idea; however, I have not yet had a chance to test the Fusion RPMs.

Update 2009/02/15
Here's a post on how to build Cinelerra from source on Fedora 10, x86-64
*** end update ***

I'm using ATrpms for a small number of Cinelerra dependencies, sixteen programs in all (listed below). I've given my new Fedora 9 system a fairly good break in test and have not found any issues arising from my use of ATrpms.

12/17/2008 Note regarding my use of ATrpms
So often in this topsy-turvy world of new distributions and funky repositories, the moment you write that an install is without problems, that old devil fate creeps up on you and proves you wrong.

Such is the case with my recent Cinelerra build from source using ATrpms dependencies. The whole day last Sunday was shot with a nasty Cinelerra hang that was only resolved when I removed the ATrpms repo programs, installed the RPM Fusion repository and recompiled Cinelerra. I'll follow up with a more detailed explanation but for now, The Mule would like to humbly apologize to anyone he led astray by recommending ATrpms.

On the positive side, unexpected problems usually lead one to find new solutions and better ways of doing things. So that's the upside of this mess.
end note

Here are the packages that I installed from ATrpms (and subsequently removed) on my Fedora 9, x86-64 system:

Name       : ffmpeg
Version    : 0.4.9
Release    : 28_r15845.fc9
--
Name       : fftw
Version    : 3.1.2
Release    : 11.fc9
--
Name       : fftw-devel
Version    : 3.1.2
Release    : 11.fc9
--
Name       : freetype-static
Version    : 2.3.5
Release    : 4.fc9.cubbi2
--
Name       : lame
Version    : 3.98.2
Release    : 19.1.fc9
--
Name       : libdvdcss
Version    : 1.2.10
Release    : 5.fc9
--
Name       : libdvdcss-devel
Version    : 1.2.10
Release    : 5.fc9
--
Name       : libdvdcss2
Version    : 1.2.10
Release    : 5.fc9
--
Name       : libdvdnav4
Version    : 0.1.10
Release    : 2.fc9
--
Name       : libraw1394
Version    : 1.3.0
Release    : 8_11.fc9
--
Name       : libraw1394
Version    : 1.3.0
Release    : 8_11.fc9
--
Name       : libraw1394-devel
Version    : 1.3.0
Release    : 8_11.fc9
--
Name       : libraw1394_8
Version    : 1.3.0
Release    : 8_11.fc9
--
Name       : libraw1394_8
Version    : 1.3.0
Release    : 8_11.fc9
--
Name       : x264
Version    : 0.65
Release    : 8_20081108.2245.fc9
--
Name       : x264-devel
Version    : 0.65
Release    : 8_20081108.2245.fc9

I've put my system through the following tests:
-edit 720P project in Cinelerra
-via Cinelerra's YUV4MPEG streamer and mpeg2enc, export 720P resolution MPEG2 video
-from Cinelerra, export MPEG, Layer II audio
-using mplex, combine MPEG2 video and MPEG Layer II audio stream into MPEG-PS
-using VLC, convert program stream into MPEG-TS
-using FFMPEG, convert transport stream into DVD
-using FFMPEG, convert DVD file into iTunes compatible format

This is my normal workflow. Nothing seems to have broken in this workflow, so I think ATrpms has not effected my particular install.

Again, my apologies if anyone had tried using ATrpms based on my advice.
the mule

Fedora 9 x86-64, Cinelerra dependencies

I'm happy to report that the installation of Fedora 9 x86-64 went very smoothly tonight. I yanked down a DVD install from here via torrent. I then burnt an ISO dvd of Fedora 9 using my own instructions here. From the perspective of a Cinelerra advocate, the biggest change from Fedora 7 is that Fedora 9 is more reliant on the ATrpms repository for Cinelerra source code dependencies.

Update 2009/02/15
Since writing this article, I've had very strange crashes and hangs from Cinelerra on Fedora 9. So I've moved on to using Fedora 10 x86-64. The general steps listed below to install Cinelerra still apply, but here's a post that summarizes the steps on how to build Cinelerra from source on Fedora 10, x86-64
*** end update ***

Advice on the Use of Repositories
As a general note, use as few repositories as possible for the various Cinelerra dependencies. Mixing many repos will result in a compile that doesn't work or worse, poor performance or odd behavior in Cinelerra resulting from slightly incompatible libraries. I've lived both and the latter is far worse because you'll spend hours or days pulling your hair out when, in the end, you've got some natty problem that only occurs between this library or that library. Ugh! I get the shivers just thinking about it!

Building Cinelerra from Source?
If you're using Cinelerra on Fedora 9 and want to build Cinelerra from source, here's what you need to do. There will be only three repos needed:
1) Fedora
2) ATrpms
3) Freshrpms (only for mjpegtools)

12/17/2008 NOTE on my strikethrough above
It was brought to my attention that RPM Fusion is the new repository of the repositories for Fedora (a note about Fedora and ATrpms). As I experimented more with my Fedora 9 build, I found an issue with Cinelerra hanging that I will describe in a later post. I fixed the problem by removing the ATrpms dependencies and adding RPMFusion as the only additional Fedora repository.

To reiterate, going forward, I am only using RPM Fusion in addition to Fedora's own repo.

The text below has been updated to reflect this new reality.
end NOTE

There will be only two repos needed:
1) Fedora
2) RPM Fusion

Here are the steps
1) install base Fedora 9 by selecting Office/Productivity and Development tools

2) The set of Cinelerra dependencies will come mainly from the RPM Fusion repository, so add the Fusion repo from here:
http://rpmfusion.org/Configuration

3) In addition to the Fedora repo, you'll now have RPMFusion added as a repository. Use the following script to install core programs for the Cinelerra build. Don't forget to chmod a+x your script!
yum install gsm-devel \
libvorbis* \
libogg* \
libtool* \
libtheora* \
libpng* \
libjpeg* \
libtiff* \
esound* \
audiofile* \
libraw1394* \
libavc1394* \
freetype* \
fontconfig* \
nasm \
e2fsprogs* \
OpenEXR* \
fftw \
fftw-devel \
libsndfile* \
libiec61883* \
libdv* \
libquicktime \
ffmpeg \
xvidcore* \
lame \
lame-devel \
a52* \
faad2* \
x264* \
mjpegtools \
faac*

4) As the Cinelerra CVS source code is no longer available via subversion, install git:
yum install git

Here's a good intro to git
Another good git reference
http://www-cs-students.stanford.edu/~blynn/gitmagic/ch02.html

Here are a couple useful git commands:
To find an earlier version of source code:
git whatchanged --since="20 weeks ago"

To checkout an earlier version of source code:
git checkout [revision number]

example:
git checkout d91e062cadb8a5a7b8c35e9092234948d6918767

5) Grab the Cinelerra CVS source:
git clone git://git.cinelerra.org/j6t/cinelerra.git cinelerra_source

6) Run autogen.sh
[mule@ogre cinelerra_source]# ./autogen.sh
User defined paths to the preferred autoconf and automake versions.
Read the script if you would like to modify them.
AUTOMAKE=automake
ACLOCAL=aclocal
AUTOCONF=autoconf
AUTOHEADER=autoheader
..

7) Run configure
[mule@ogre cinelerra_source]# ./configure
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
..

8) Hopefully, configure ran properly and shows that you have all the prerequisites for a build from source:
Summary of mandatory components:
libogg found
libvorbis found
libvorbisenc found
libvorbisfile found
libtheora found
OpenEXR found
libdv found
libpng found
libjpeg libraries found
libjpeg headers found
libtiff libraries found
libtiff headers found
FreeType 2 found
libx264 libraries found
libx264 headers found
libuuid libraries found
libuuid headers found
mjpegtools found
libfftw3 libraries found
libfftw3 headers found
liba52 libraries found
liba52 headers found
libmp3lame libraries found
libmp3lame headers found
libsndfile libraries found
libsndfile headers found
libfaac libraries found
libfaac headers found
libfaad libraries found
libfaad headers found

Summary of optional components:
ESD subsystem found
ESD (Enlightenment Sound Daemon) is enabled
ALSA subsystem found
ALSA is enabled
libraw1394 found
libiec61883 found
libavc1394 libraries found
libavc1394 headers found
librom1394 libraries found
librom1394 headers found
Firewire is enabled
OpenGL 2.0 libraries found
Hardware acceleration using OpenGL 2.0 is enabled

Now type
make

to start compilation.

9) Run "make"

10) Hopefully, there were no errors on make, so run "make install"

11) As the topper, don't forget to install some media players and extra encoding tools!:
yum install avidemux mencoder mplayer transcode audacious vnc

Good luck!
The Mule

Update 2008/11/23
I verified that importing live HDV video from my cam via test-mpeg2 still works. This is done with the following command:
test-mpeg2 -r [node] [file]
end update

using partimage with RAID

Background
As I am planning on a purchase of a 1080p cam, I will need my system to be up on the latest and greatest kernal and software to get the highest performance from Cinelerra. In that light, I'd like to backup my current Fedora 7 boot and root filesystems, just in case something goes wrong with the Fedora 9 install.

Partimage and My System
I will use partimage to backup these filesystems. Partimage will need to see source and destination filesystems. My first task is to figure out what I have. I built this system over a year ago and don't remember all the specifics of which physical drive has x or y filesystems. I could go back into my notes to find out how I partitioned my system, but that would be cheating. So let's see what the filesystem tells me.

The first thing I do is look at the output of df:
[mule@ogre ~]# df -m
Filesystem 1M-blocks Used Available Use% Mounted on
/dev/md0 457295 6720 426972 2% /
/dev/md2 469453 417004 28602 94% /mnt/videos
/dev/sda1 99 19 76 20% /boot
tmpfs 1007 0 1007 0% /dev/shm

I have two RAID devices, one mounted as my root partition (/dev/md0) and one mounted as my video storage (/dev/md2). Next, I see that /dev/sda1 is my boot partition. Finally, there is a filesystem defined for shared memory, though I am not concerned about saving the contents of that as it is RAM.

How It Works
Partimage backs up filesystems that are not mounted. But partimage is started from a bootable rescue disk, like Knoppix or SysRescCd. The twist here is that I am using RAID partitions. Thus, when I boot with one of these CDs, I will need to assemble my RAID drives in order to have a source to backup (my root filesystem) and a destination to write to (my /mnt/videos filesystem). Partimage will not use a mounted filesystem as a source, but I will need to mount the destination.

Assembling My RAID Drives
I have forgotten the configuration of my RAID drives, so I look at /etc/mdadm.conf to figure out what partitions and UUIDs make up my two RAID sets:
[mule@ogre ~]# cat /etc/mdadm.conf
# mdadm.conf written out by anaconda
DEVICE partitions
MAILADDR root
ARRAY /dev/md0 level=raid0 num-devices=2 uuid=c0d4b597:c33b3014:ab694cee:76920165
ARRAY /dev/md2 level=raid1 num-devices=2 UUID=1705b387:1c71d83e:364b60b4:fb0cc192

This tells me that my root partition (/dev/md0) is a stripe set (RAID0) and that the storage for my important stuff, all my videos in /mnt/video is mirrored (RAID1) in case of a failure. I'm glad I built the system this way, as I like the performance benefits of a stripe set for my root partition, but I would consider it tragic if I lost all my work. Therefore, I've mirrored the video drive on two drives in case of a failure.

Video Display Problem with Linux and NVidia Card
For some reason I have not figured out, I cannot see virtual consoles once I exit Gnome. This is due to some incompatibility between the NVidia 8800GT card and my Dell SC1430. This also effects the display when I boot with either Knoppix or SysRescCD. Using these tools, the screen goes black and I can't see any terminal sessions or virtual consoles. Therefore, in order to use the boot cd, I removed the NVidia card and booted using Dell's ATI ES1000 onboard video.

Booting with Knoppix
Once Knoppix is fully booted, I need to assemble my two RAID partitions. You can use either use the UUID or the super-minor number of each RAID set to do this. I chose the super-minor, as it was simpler.

Assemble the Source RAID set
Here I am assembling the source drive, my root filesystem:
root@Knoppix:/ramdisk/home/knoppix# modprobe md
root@Knoppix:/ramdisk/home/knoppix# mdadm --assemble -m 0 /dev/md0

Mount the Destination Partition
Since I want to store the backup image on the same mirrored drive set that holds my videos, I'll mount that partition as the destination for the partimage. Of course, I first have to create the mount point:
mkdir /mnt/videos
mount -t ext3 /dev/md2 /mnt/videos

Run Partimage to Backup Root Partition
I'll need three things to run partimage:
-an assembled RAID set of the source, my root/boot partitions, unmounted
-an assembled RAID set of the destination, mounted
-a compression method

Here's the partimage process:
1) Select the partition to save and give the backup a destination and name. Note that the "Save partition into an image file" is selected as the default behavior:


2) Select a compression method:


3) Give the backup image a description (optional):


4) Partimage takes a few minutes to gather information about large (500GB+ drives), but then displays basic information about the partition to be backed up:


5) Partimage starts the imaging process. I had about 6GB to backup:


6) Partimage took about 20 minutes to create the backup image:


Backup complete. The restore process is similar, but instead of backing up an image file as in Step 1 above, you'll choose the "Restore Partition from an image file" option.

Run Partimage for Boot Partition
Since my boot partition is small 128MB, creating a backup image shouldn't take very long. My boot partition is /dev/sda1


Now I should be ready to an upgrade to Fedora 9. One hurdle I already see: the Fedora 9 installation doesn't recognize pre-existing RAID sets. Yarg. Looks like I might have to blow away the existing stripe set that is home to my root partition. Let you know how that goes.

Update 11/17/2008
The Fedora 9 x86-64 install went well. Here are some of the natty details:
http://crazedmuleproductions.blogspot.com/2008/11/fedora-9-x86-64-install.html
end update

Good day,
The Mule

References
mdadm man page