new 500GB SATA RAID1 mirror

For fault tolerance, I finally got around to upgrading my drives to a RAID1 mirror this weekend. I bought a pair of Seagate ST3500641AS-RK Barracuda from Outpost.com for $99 each. Sale is over now, of course.

Here's what I had to do:

1. partition the drives using fdisk
2. create the mirror
3. wait for the mirror to sync
   
4. format the mirror
5. test a mount
6. add to /etc/fstab
7. reboot to make sure everything comes up
   

Partition the drives with good ol' fdisk
I had to partition each drive and give each drive a partition type of "Linux raid autodetect". This is type "fd". Here is a snip of my fdisk session for one of the drives:

[mule@ogre ~]# fdisk /dev/sdf
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.


The number of cylinders for this disk is set to 60801.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): p

Disk /dev/sdf: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4):
Value out of range.
Partition number (1-4): 1
First cylinder (1-60801, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-60801, default 60801):
Using default value 60801

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd

Command (m for help): p

Disk /dev/sdf: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdf1 1 60801 488384001 fd Linux raid autodetect

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Create the mirror
I created the mirror using mdadm. Note the "--level=1" for the RAID1 mirror:
[mule@ogre ~]# mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/sde1 /dev/sdf1

Let's check out the details:
[mule@ogre ~]# mdadm --detail /dev/md2
/dev/md2:
Version : 00.90.03
Creation Time : Thu Jan 17 19:58:30 2008
Raid Level : raid1
Array Size : 488383936 (465.76 GiB 500.11 GB)
Used Dev Size : 488383936 (465.76 GiB 500.11 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 2
Persistence : Superblock is persistent

Update Time : Sun Jan 20 16:19:10 2008
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

UUID : 1705b387:1c71d83e:364b60b4:fb0cce92
Events : 0.6

Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sde1
1 8 49 1 active sync /dev/sdf1

Wait for the mirror to build
You can also look in /proc/mdstat for status information:
[mule@ogre ~]# cat /proc/mdstat
Personalities : [raid0] [raid6] [raid5] [raid4] [raid1]
md2 : active raid1 sdb1[0] sdd1[1]
488383936 blocks [2/2] [UU]

md0 : active raid0 sda3[0] sdc2[1]
483411456 blocks 256k chunks

unused devices:

Format the partition
The -j parameter to mke2fs creates the filesystem with an ext3 journal, for another level of safety. File access on an ext3 filesystem is a bit slower than on an ext2 filesystem. However, the benefit is that you have a journaling filesystem that recovers from errors quickly and safely. Here's some documentation on the ext3 filesystem:
http://www.ibm.com/developerworks/linux/library/l-fs7.html

Here is the command I ran to create the ext3 filesystem:
[mule@ogre ~]# mke2fs -j /dev/md2

Test the mount
You should always test mounting the new filesystem. Since this filesystem is going to store all my videos, I am going to create my mount point and mount the new RAID mirror to /mnt/videos:
[mule@ogre ~]# mkdir /mnt/videos
[mule@ogre ~]# mount -t ext3 /dev/md2 /mnt/videos
[mule@ogre ~]#

You see no errors after the mount command. This means that the mount was successful. Yes!

Add to /etc/fstab
Of course, we want the new filesystem to be mounted when we reboot, so I add the following line to my /etc/fstab:
/dev/md2 /mnt/videos ext3 defaults 1 1

Reboot!
The final test is to reboot. The following lines in dmesg output make me happy:
md: md2 stopped.
md: bind
md: bind
md: raid1 personality registered for level 1
raid1: raid set md2 active with 2 out of 2 mirrors

Rock and roll! We're good with the new RAID mirror and safe from drive failures!
The Mule

migrated my FC6 system to faster disk

When I was on Fedora Core 4, I had been using a fast, 250GB Western Digital SATA drive as my main root partition. Since I live in both XP and Fedora worlds, I divided up the drive into the following partitions:
/dev/sda1: XP boot (40GB)
/dev/sda2: Fedora Core 4 /boot (128MB)
/dev/sda3: Fedora Core 4 / (root) (120GB)
/dev/sda4: unused
/dev/sda5: swap (6GB)
/dev/sda6: FAT32 (to transfer files in between Linux and XP) (60GB)

This configuration allows me to dual boot XP and FC easily, as well as transfer files in between the two OSs. Speaking of NTFS-ext2/3 transfers, I'm slowing phasing out the FAT32 partition for transfers because I have had success using ntfs-3g (http://www.ntfs-3g.org) to write to and read from my NTFS partitions. NTFS-3g has been very solid for reading and writing from Linux to NTFS.

Earlier this year, I installed Fedora Core 6 as a development/test system on a separate 40GB drive. Doing the install this way instead of an upgrade of FC4 meant that I didn't have to cripple the FC4 system in case anything went wrong with FC6. As it happens, Cinelerra on FC6 happens to be the most stable Cinelerra I've used these past two years, apart from an audio sync issue on rendered videos. I correct the sync problem using the Nudge feature on each audio track, detailed here:
/2007/05/nudge-avidemux2-and-reminder-about.html

Lo these past six months, I have been editing on FC6, albeit on a slower IDE drive, a 7200RPM Barracuda. The 250GB SATA with 16MB memory cache is quite a bit faster. So the goal was to move the boot/root of the FC6 system to the boot/root of the faster SATA drive.

LVM versus ext3
The first issue I encountered was that the default installation of FC6 automatically creates a Volume Group and Logical Volumes instead of plain old ext2/ext3 filesystems. My FC4 system used ext3 for boot/root, so my original plan was to do a simple partimage backup of the two FC6 filesystems. That idea would not work, as FC6 was on LVM and partimage doesn't recognize LVM partitions.

As an experiment (and because I wanted to get up and running quickly), I simplified the plan to install the new drive in the machine, mount the FC4 filesystems on the fast drive, delete the content on them and replace the content with the FC6 system files. I'd have to then tweak fstab and grub.conf appropriately on the updated drive to make the whole thing work.

The Plan
1) install both drives in the same machine with FC6 as the primary system,
2) mount the FC4 partitions of the fast drive in FC6 (/mnt/fc4boot & /mnt/fc4root),
3) remove all the files off of the FC4 boot/root filesystems,
4) copy the files from the FC6 boot/root parts to the FC4 partitions,
5) carefully tweak grub.conf and fstab to reference the new partition scheme
6) hope for the best on reboot

Here is where we had "a few issues" as an old boss of mine would say.

Problems!
I guess it is my cursory knowledge of the boot process that got me into trouble. The first issue after performing these four steps was the following error:
mount: could not find filesystem '/dev/root'
setuproot: moving /dev failed: No such file or directory
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!

Ouch! That doesn't look good. Following the advice of this thread:
http://www.linuxquestions.org/questions/showthread.php?t=497332

First, using SysRescCD, I verified my partition labels on the fast drive using e2label:
e2label /dev/hdN
where N is the Fedora partition

I double-checked my grub.conf and fstab:

/boot/grub/grub.conf
Here's the old entry:
title Fedora Core (2.6.19-1.2911.fc6)
root (hd0,0)
kernel /vmlinuz-2.6.19-1.2911.fc6 ro root=/dev/VolGroup00/LogVol00
initrd /initrd-2.6.19-1.2911.fc6.img

Here's the new entry:
title Fedora Core (2.6.19-1.2911.fc6)
root (hd0,1)
kernel /vmlinuz-2.6.19-1.2911.fc6 ro root=LABEL=/
initrd /initrd-2.6.19-1.2911.fc6.img

Note the changes in bold that I needed to enter in order to move the root from the first partition on the older, slow drive to the second partition on the faster drive. Also, LVM goes away to be replaced by the simpler naming scheme of the ext3 formatted partition.

/etc/fstab
Secondly, the fstab changes as well, from:
/dev/VolGroup00/LogVol00 / ext3 defaults 1 1
/dev/VolGroup00/LogVol01 swap swap defaults 0 0

To:
LABEL=/ / ext3 defaults 1 1
/dev/sda5 swap swap defaults 0 0

The /boot entry stays the same:
LABEL=/boot /boot ext3 defaults 1 2

The output of e2label showed that my fast drive partition labels matched those in grub.conf and fstab. So no smoking gun there.

The next thing suggested in the thread was to create mkinitrd, the initial ram disk the Linux kernel uses for system files. Oh boy. I've forgotten about this one. I knew I had the correct partition labels, but I needed a little more background on mkinitrd and these articles helped:
http://www.faqs.org/docs/Linux-mini/Loopback-Root-FS.html
http://www.opennet.ru/docs/HOWTO/Kernel-HOWTO-11.html+creating+initrd
http://www.ibm.com/developerworks/linux/library/l-initrd.html

Redhat/Fedora has a nice utility called mkinitrd that simplifies many of the steps mentioned in the IBM and HOWTO articles above. I regenerated the initial ramdisk image on the new boot partition using the following command:
mkinitrd ./initrd-2.6.19-1.2911.fc6.img 2.6.19-1.2911.fc6

OK. There were no errors on creation and the file is there and has data:
-rw------- 1 root root 2398566 Sep 9 11:52 /boot/initrd-2.6.19-1.2911.fc6.img

Cool. Now for the moment of truth. The reboot! I powered down and waited for the system to come back up. Thankfully, the kernel got beyond the panic and booted properly! Hooray! However, I did see some errors still:
No volume groups found
Volume group "VolGroup00" not found
Unable to access resume device (/dev/VolGroup00/LogVol01)

But, the system loads fine and Cinelerra and all my apps seem to work. So I will have to research this error further.

Also, I saw when I started Evolution, I got the following error:
Error while storing folder 'Inbox'

Researching this, I saw that there were inconsistencies in the index files that Evolution creates. The workaround is simple: delete any mail folders in ~./evolution/mail/local with an extension of ev.~summary. These are indexing files only and you can safely remove them. Reference:
https://bugs.launchpad.net/ubuntu/+source/evolution/+bug/27014

For the past couple of days, I have been editing and Cinelerra is performing well. I'm glad I went through this exercise. Though, I really need to learn more about how to backup and restore LVM partitions. I might have saved myself some pain.

cheers!
The Mule

upgraded video workstation has arrived!

My video rig is finally ready to go! After one month of hardware and software upgrades, I'm ready to start editing. Here are the upgrades I did:
- from 1GB non-ECC, Dual DDR to 2GB ECC Dual DDR mem
- from a system and a content drive to a system and RAID0 content drive
- from a 128MB ATI video card to a 512MB NVidia video card
- from non-OpenGL to OpenGL editing software

Update (9/19): converted my old 80GB dual-boot XP/Core 4 system drive to shiny new 250GB SATA w/16MB cache! Took about seven hours to backup the filesystems, partition the new drive and restore, with a few lessons learned along the way. Here's how this went:
http://cacasodo.blogspot.com/2006/09/replacing-old-dual-boot-system-drive.html

So, we're ready to go with da editing! Hooray!!

Install notes:
- FC4 install
- fstab/ntfs kernel module/NVidia drivers/xorg.conf/mdadm.conf/rpmkeys/remove libdv/install libdv4
- libdvDeps.sh
- install cinelerra
- cinSourceDeps.sh
- create /mnt/win, /mnt/nt
- window prefs/google/cin window placemnet/save on exit