the big picture

With the dualdvd archive piece and scripted workflow, I was able to optimize a couple of pieces of the video production lifecycle:

idea creation -> storyboard -> production -> distribution -> marketing -> archive

Update 2008/11/22
I've added a few more details to my workflow in a new post here.
end update

I fear I'm spending too much time on the production piece and all the associated technical details. However, from a business perspective, the devil is in the details and if you have a good workflow, that means that you can do more in less time and be able to dedicated more time to the more creative/marketing aspects. So I don't think it is for nothing that I am doing this. Too bad I can't get those darn Ximeta NDAS drivers to work on Linux!!

MediaGate - Ximeta NDAS troubles with Linux

The MediaGate MG-350HD is a little wireless device that allows me to view my Cinelerra creations on my HDTV.


So, I'm trying to get it to work smoothly with Linux, as that's where my HD source files are. However, HD files will only play on the MediaGate if the files are either:
1) accessed via network share over wired connection or
2) copied via NDAS drivers to the hard drive installed in the thing

Ximeta (http://www.ximeta.com/) are the makers of the network direct attached storage (NDAS) that the MediaGate uses. I don't have a hard run down to my entertainment center, so I've been pulling my hair out in order to get the NDAS drivers working. The problem with this is that the drivers seem to work on XP only. The Linux drivers on Ximeta's site don't work worth a damn. They fail on my fedora and redhat enterprise boxes and on the two Ubuntu installs (6.0.6/6.1.0) that I did this morning. Yarg. In addition, I've tried upgrading the firmware to no avail. The frustrating thing is that the firmware is supposed to fix a problem where the MG can't do wired and wireless connections at the same time. In point of fact, the firmware upgrade has BROKEN wireless. Double YARG!!

2/11 Update: I've since fixed the wireless by first doing an AP (access point) scan of the enviroment first, and then changing the wireless mode to Ad Hoc to configure the security settings of my specific router. Gads!

So I think I'm just going to leave it wired upstairs for now, copy over the HD content when its ready and bring it back down to the HDTV. What a waste of time..I'm going to send the bozos an email.

2/19/2007 Update: A wirelessly mounted MediaGate is now working!!

scripted rendering!

I am very happy to say that I've been able to script a couple pieces of the rendering process. Once I've edited my HDV masterpiece, I export out an .m2v file and an mp3, layer 2 file, according to my previous post:
http://crazedmuleproductions.blogspot.com/2006/11/cam-compatible-hdv-edit-chain-part-ii.html

I then use the following script to:
1) combine the .m2v and .m2a as a program stream (using mplex)
2) convert the program stream to a transport stream, MPEG-TS file (using VLC)
3) convert the MPEG-TS to a DVD compatible mpg (using ffmpeg)
4) lastly, convert the DVD to an iTunes compatible format (again, using ffmpeg)

Obviously, their is inherent quality loss in each step. I tried to eliminate one conversion step by rendering to the iTunes format direct from the HDV source, but for some reason, ffmpeg gave me errors. So, I saved time and just converted from the DVD source. I will revisit this problem in a later post.

I can say that scripting VLC kicks ass! No more GUI!! Yay!

Here is the script. Don't forget to "chmod a+x " in order to run it.

Note that when run, the script will ask for a filename. That filename must be common to the .m2v and .m2a files. For example, start with two files called test.m2v and test.m2a. The script will prompt you for the filename, at which point you will enter "test" and hit enter. After that, the script will chug away and render each of the files to the filename test., where format will vary by the destination type (.ps/.m2t/.mpg/.mov).

Good luck!

#!/bin/bash -v
#
# This script converts HDV content
echo "Type in the name of the m2a/m2v files for mplex'ing"
read NAME
echo "Mplex'ing $NAME"
mplex -f 3 -b 2000 ${NAME}.m2a ${NAME}.m2v -o ${NAME}.ps

echo "VLC'ing $NAME"
vlc $NAME.ps --sout '#duplicate{dst=std{access=file,mux=ts,dst="'$NAME'.m2t"}}' vlc:quit

echo "FFMPEG convert to DVD"
ffmpeg -i ${NAME}.m2t -target dvd ${NAME}.mpg

echo "FFMPEG convert to MOV"
ffmpeg -i ${NAME}.m2t -f mov -vcodec mpeg4 -qscale 7 -s 320x180 -r 29.97 -aspect 16:9 -acodec aac -ac 2 -ab 128 ${NAME}.mov
echo "Done"

backing up your HDV content with dual-layer DVD

I finally made the jump to dual DVDs. I successfully burned a dual layer dvd using growisofs! I am very excited! This means I can get that content OFF my hard drives..FINALLY! However, there is a 4.3GB file size limitation, so keep those MPEGTS's small. Or, if you have files you need to backup that are larger than 4.3GB, go ahead and split them using AVIDEMUX.

First, get and install the dvd+rw-tools from here:
http://fy.chalmers.se/~appro/linux/DVD+RW/tools/

The install requires you to:
1) untar/gunzip the source files
2) run "make"
3) run "make install"

After that, the tools should be installed. Once the tools are installed, load up a dual-layer disc and check your media:
dvd+rw-mediainfo /dev/hda

You'll get output similar to this:
[gagazote@computer hvirtual]# dvd+rw-mediainfo /dev/hda
INQUIRY: [_NEC ][DVD_RW ND-3500AG][2.16]
GET [CURRENT] CONFIGURATION:
Mounted Media: 2Bh, DVD+R Double Layer
Media ID: MKM/001
Current Write Speed: 6.1x1385=8467KB/s
Write Speed #0: 6.1x1385=8467KB/s
Write Speed #1: 5.1x1385=7056KB/s
Write Speed #2: 4.1x1385=5645KB/s
Write Speed #3: 3.1x1385=4234KB/s
Write Speed #4: 2.0x1385=2822KB/s
Write Speed #5: 1.0x1385=1411KB/s
GET [CURRENT] PERFORMANCE:
Write Performance: 4.0x1385=5540KB/s@[0 -> 4173824]
Speed Descriptor#0: 00/4173824 R@5.0x1385=6925KB/s W@4.0x1385=5540KB/s
Speed Descriptor#1: 00/4173824 R@5.0x1385=6925KB/s W@2.4x1385=3324KB/s
DVD+R DOUBLE LAYER BOUNDARY INFORMATION:
L0 Data Zone Capacity: 2086912*2KB, can still be set
READ DISC INFORMATION:
Disc status: blank
Number of Sessions: 1
State of Last Session: empty
Number of Tracks: 1
READ TRACK INFORMATION[#1]:
Track State: blank
Track Start Address: 0*2KB
Next Writable Address: 0*2KB
Free Blocks: 4173824*2KB
Track Size: 4173824*2KB
ROM Compatibility LBA: 262144
READ CAPACITY: 1*2048=2048

Then, burn your files:
growisofs -Z /dev/hda -R -J /mnt/videos/*

This command creates an ISO 9660 image of the directory "/mnt/videos" and writes it to a dual-layer DVD.

-Z erases any previous contents

-R include rockridge extensions

-J include joliet extentions

Cinelerra hanging after I move audio on the timeline

This is a bit off topic, but is related to troubleshooting Cinelerra. I have been having a problem with Cinelerra hanging for three or four minutes after I zoom in and slide audio tracks around on the timeline. To replicate the problem, I grabbed and moved the audio track (a wav file) around quite a bit, perhaps 10 times. In order to find the hanging thread, I started gdb ("gdb cinelerra pid").
[root@localhost ~]# ps -ef | grep cin
root 3671 3571 90 20:05 pts/2 00:17:55 cinelerra
root 4263 4178 0 20:25 pts/4 00:00:00 grep cin
[root@localhost ~]# gdb cinelerra 3671

Scrolling through the thread list, I identified the offending thread:

> Thread 65 (Thread -1520473168 (LWP 5674)):
> #0 0x0813fceb in CacheBase::get_oldest (this=0x8c04068) at cachebase.C:106
> #1 0x081f00bb in MWindow::age_caches (this=0xbfc5a288) at
> mwindow.C:1569
> #2 0x081f5e7b in MWindow::move_edits (this=0xbfc5a288, edits=0x8539b70,
> track=0x920f238, position=49.724958333333333, behaviour=1) at
> mwindowedit.C:830
> #3 0x08289ed6 in TrackCanvas::drag_stop (this=0xa9e1e7e0) at
> trackcanvas.C:575
> #4 0x0828a13d in TrackCanvas::drag_stop_event (this=0xa9e1e7e0) at
> trackcanvas.C:325
> #5 0xb7b73029 in BC_WindowBase::dispatch_drag_stop (this=0xa9e1e7e0) at
> bcwindowbase.C:1316
> #6 0xb7b7300e in BC_WindowBase::dispatch_drag_stop (this=0x8c44da0) at
> bcwindowbase.C:1311
> #7 0xb7b73122 in BC_WindowBase::dispatch_button_release
> (this=0x8c44da0) at bcwindowbase.C:1162
> #8 0xb7b778b1 in BC_WindowBase::dispatch_event (this=0x8c44da0) at
> bcwindowbase.C:787
> #9 0xb7b78739 in BC_WindowBase::run_window (this=0x8c44da0) at
> bcwindowbase.C:614
> #10 0xb7b88bb8 in Thread::entrypoint (parameters=0xbfc5a288) at
> thread.C:48
> #11 0x00da0b80 in start_thread () from /lib/libpthread.so.0
> #12 0x00c11dee in clone () from /lib/libc.so.6

I then exited gdb and found that the process hang was still occurring.
In order to step through the program code, I use kdbg, a graphical debugger. I started up kdbg ("kdbg -p /usr/local/bin/cinelerra") and
selected "View -> Threads". I then used an alternating combination of
F8 and F10 to step in and out of the code. I found that kdbg hung at
this line of code: MWindow.C:1561. It took a while, maybe five minutes
for kdbg to free up. This indicates where the hang is occurring. After the release, I exited the debugger.

Here is the snippet of hvirtual/cinelerra/mwindow.C code starting at
line 1561:

> if(memory_usage > preferences->cache_size)
> {
> int target = 1;
> int oldest1 = audio_cache->get_oldest();
> int oldest2 = video_cache->get_oldest();
> if(oldest2 < target =" 2;"> int oldest3 = frame_cache->get_oldest();
> if(oldest3 <> target = 3;
> int oldest4 = wave_cache->get_oldest();
> if(oldest4 <> oldest4 < target =" 4;"> switch(target)
> {
> case 1: audio_cache->delete_oldest();
> break;
> case 2: video_cache->delete_oldest();
> break;
> case 3: frame_cache->delete_oldest();
> break;
> case 4: wave_cache->delete_oldest();
> break;
> }
> }

I'm not a very experienced C programmer, but inferring from the if statement, it looks like I only go into this control structure if cin's memory usage is
greater than the value stated in Preferences -> Cache Size. I have it
set to the default value of 10MB.

I have not yet found the cause of this particular hang, but I will update my blog once I have an answer.

AVIDEMUX is an excellent little program..

Rough cuts video of any format and saves to any format rapidly. I used it to chop up a 7GB MPEGTS HDV file and it took one minute to render out a 2.7GB! This is much better than using Cinelerra to reduce it via a re-render via Quicktime. Sweet! I don't see any loss in the image, but that will need more investigation. Still, initial impressions are very good!