Canon 5D conversion for xiphmont's git

As Monty has been working hard to make Cinelerra more input friendly with his FFmpeg loader fixes, I've been trying to help out as a QA resource. One of my main concerns has been getting my Canon 5D video into Cinelerra. As documented in the IRC log, my custom Canon conversion (ffmpeg stream -> mpeg2enc (output: .m2v/.m2a) -> mplex (output MPEG-PS) -> ffmpeg (output MPEG-TS) was full of invalid timestamps.

In addition, with some recent security updates to my Fedora system just today, it looks like my previous Canon conversion script had broken. I get the dread error:

**ERROR: [mpeg2enc] display_horizontal_size must be in range 0...16383

So I was doubly screwed. Back to the drawing board..er, mailing list. Following up on mjpegtools' Bernhard Prachinger's advice here, I first try to play a test file using a pipe from ffmpeg, like so:
ffmpeg -i mvi_2655.mov -threads 7 -s 1920x1088 -f yuv4mpegpipe - | yuvplay

The video stream plays back with no visible errors, outside of a warning about the video stream frame rate differing from the container frame rate. Here are the top few lines of the output:
Seems stream 0 codec frame rate differs from container frame rate: 48000.00 (48000/1) -> 23.98 (24000/1001)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'mvi_2655.mov':
Duration: 00:00:06.92, start: 0.000000, bitrate: 45836 kb/s
Stream #0.0(eng): Video: h264, yuv420p, 1920x1088, 44296 kb/s, 23.98 tbr, 24k tbn, 48k tbc
Stream #0.1(eng): Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
Metadata
major_brand : qt
minor_version : 537331968
compatible_brands: qt CAEP
Output #0, yuv4mpegpipe, to 'pipe:':
Stream #0.0(eng): Video: rawvideo, yuv420p, 920x108, q=2-31, 200 kb/s, 90k tbn, 23.98 tbc
Stream mapping:
Stream #0.0 -> #0.0
Press [q] to stop encoding
INFO: [yuvplay] Playing frame 0000 - 0:00:00.00
INFO: [yuvplay] Playing frame 0001 - 0:00:00.01
INFO: [yuvplay] Playing frame 0002 - 0:00:00.02
INFO: [yuvplay] Playing frame 0003 - 0:00:00.03
INFO: [yuvplay] Playing frame 0004 - 0:00:00.04
INFO: [yuvplay] Playing frame 0005 - 0:00:00.05
INFO: [yuvplay] Playing frame 0006 - 0:00:00.06
INFO: [yuvplay] Playing frame 0007 - 0:00:00.077 bitrate=752028.9kbits/s

So, streaming from FFmpeg to a y4m stream works. I then take that y4m stream and convert it to an mpeg2video for use in Cinelerra, again using Bernhard's simplified mpeg2enc pipe adjusted for aspect ratio and higher bitrate:
ffmpeg -i mvi_2655.mov -threads 7 -s 1920x1088 -f yuv4mpegpipe - | mpeg2enc --verbose 2 --format 3 --aspect 3 --video-bitrate 24000 --no-constraints --video-buffer 448 -o mvi_2655.m2v

Looking at the output (--verbose 2), there don't seem to be any major errors here:
Seems stream 0 codec frame rate differs from container frame rate: 48000.00 (48000/1) -> 23.98 (24000/1001)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'mvi_2655.mov':
Duration: 00:00:06.92, start: 0.000000, bitrate: 45836 kb/s
Stream #0.0(eng): Video: h264, yuv420p, 1920x1088, 44296 kb/s, 23.98 tbr, 24k tbn, 48k tbc
Stream #0.1(eng): Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
Metadata
major_brand : qt
minor_version : 537331968
compatible_brands: qt CAEP
Output #0, yuv4mpegpipe, to 'pipe:':
Stream #0.0(eng): Video: rawvideo, yuv420p, 920x108, q=2-31, 200 kb/s, 90k tbn, 23.98 tbc
Stream mapping:
Stream #0.0 -> #0.0
Press [q] to stop encoding
INFO: [mpeg2enc] Selecting Generic MPEG2 output profile
INFO: [mpeg2enc] Assuming norm NTSC
INFO: [mpeg2enc] Progressive input - selecting progressive encoding.
INFO: [mpeg2enc] Encoding MPEG-2 video to mvi_2655.m2v
INFO: [mpeg2enc] Horizontal size: 920 pel
INFO: [mpeg2enc] Vertical size: 108 pel
INFO: [mpeg2enc] Aspect ratio code: 3 = 16:9 display
INFO: [mpeg2enc] Frame rate code: 1 = 24000.0/1001.0 (NTSC 3:2 pulldown converted FILM)
INFO: [mpeg2enc] Bitrate: 24000 KBit/s
INFO: [mpeg2enc] Quality factor: 8 (Quantisation = 9) (1=best, 31=worst)
INFO: [mpeg2enc] Field order for input: none/progressive
INFO: [mpeg2enc] Sequence unlimited length
INFO: [mpeg2enc] Search radius: 16
INFO: [mpeg2enc] DualPrime: no
INFO: [mpeg2enc] Using one-pass rate controller
INFO: [mpeg2enc] GOP SIZE RANGE 7 TO 15
INFO: [mpeg2enc] Setting colour/gamma parameters to "NTSC"
INFO: [mpeg2enc] Progressive format frames = 1
INFO: [mpeg2enc] Using default unmodified quantization matrices
INFO: [mpeg2enc] SETTING MMX and MMX for QUANTIZER!
--DEBUG: [mpeg2enc] PAR = 7


I then take the output of the audio conversion:
ffmpeg -i mvi_2655.mov -acodec mp2 -ab 384k -ar 48000 -ac 2 test.m2a

Output:
Seems stream 0 codec frame rate differs from container frame rate: 48000.00 (48000/1) -> 23.98 (24000/1001)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'mvi_2655.mov':
Duration: 00:00:06.92, start: 0.000000, bitrate: 45836 kb/s
Stream #0.0(eng): Video: h264, yuv420p, 1920x1088, 44296 kb/s, 23.98 tbr, 24k tbn, 48k tbc
Stream #0.1(eng): Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
Metadata
major_brand : qt
minor_version : 537331968
compatible_brands: qt CAEP
Output #0, mp2, to 'mvi_2655.m2a':
Stream #0.0(eng): Audio: mp2, 48000 Hz, 2 channels, s16, 384 kb/s
Stream mapping:
Stream #0.1 -> #0.0
Press [q] to stop encoding
size= 325kB time=6.94 bitrate= 384.0kbits/s
video:0kB audio:325kB global headers:0kB muxing overhead 0.000000%

and mux both the video and audio streams to an MPEG-PS in mplex:
mplex -f 3 -b 2000 mvi_2655.m2a mvi_2655.m2v -o mvi_2655.ps

Output:
INFO: [mplex] mplex version 1.9.0 (2.2.7 $Date: 2006/02/01 22:23:01 $)
INFO: [mplex] File mvi_2655.m2a looks like an MPEG Audio stream.
INFO: [mplex] File mvi_2655.m2v looks like an MPEG Video stream.
INFO: [mplex] Found 1 audio streams and 1 video streams
INFO: [mplex] Selecting generic MPEG2 output profile
INFO: [mplex] Multiplexing video program stream!
INFO: [mplex] Scanning for header info: Audio stream c0 (mvi_2655.m2a)
INFO: [mplex] MPEG AUDIO STREAM: c0
INFO: [mplex] Audio version : 1.0
INFO: [mplex] Layer : 2
INFO: [mplex] CRC checksums : no
INFO: [mplex] Bit rate : 49152 bytes/sec (384 kbit/sec)
INFO: [mplex] Frequency : 48000 Hz
INFO: [mplex] Mode : 0 stereo
INFO: [mplex] Mode extension : 0
INFO: [mplex] Copyright bit : 0 no copyright
INFO: [mplex] Original/Copy : 1 original
INFO: [mplex] Emphasis : 0 none
INFO: [mplex] Scanning for header info: Video stream e0 (mvi_2655.m2v)
INFO: [mplex] VIDEO STREAM: e0
INFO: [mplex] Frame width : 1920
INFO: [mplex] Frame height : 1088
INFO: [mplex] Aspect ratio : 16:9 display
INFO: [mplex] Picture rate : 23.976 frames/sec
INFO: [mplex] Bit rate : 24000000 bits/sec
INFO: [mplex] Vbv buffer size : 229376 bytes
INFO: [mplex] CSPF : 0
INFO: [mplex] SYSTEMS/PROGRAM stream:
INFO: [mplex] rough-guess multiplexed stream data rate : 24894496
INFO: [mplex] Setting best-guess data rate.
INFO: [mplex] Run-in delay = 30030 Video delay = 30030 Audio delay = 33783
INFO: [mplex] New sequence commences...
INFO: [mplex] Audio c0: buf= 0 frame=000000 sector=00000000
INFO: [mplex] Video e0: buf= 0 frame=000000 sector=00000000
INFO: [mplex] Scanned to end AU 165
INFO: [mplex] STREAM e0 completed
INFO: [mplex] STREAM c0 completed
INFO: [mplex] Multiplex completion at SCR=653806.
INFO: [mplex] Audio c0: completed
INFO: [mplex] Video e0: completed
INFO: [mplex] AUDIO_STATISTICS: c0
INFO: [mplex] Audio stream length 332928 bytes.
INFO: [mplex] Syncwords : 289
INFO: [mplex] Frames : 289 padded
INFO: [mplex] Frames : 0 unpadded
INFO: [mplex] BUFFERING min 189 Buf max 1162
INFO: [mplex] VIDEO_STATISTICS: e0
INFO: [mplex] Video Stream length: 13871991 bytes
INFO: [mplex] Sequence headers: 1
INFO: [mplex] Sequence ends : 1
INFO: [mplex] No. Pictures : 166
INFO: [mplex] No. Groups : 12
INFO: [mplex] No. I Frames : 12 avg. size162167 bytes
INFO: [mplex] No. P Frames : 154 avg. size 77441 bytes
INFO: [mplex] No. B Frames : 0 avg. size 0 bytes
INFO: [mplex] Average bit-rate : 16028800 bits/sec
INFO: [mplex] Peak bit-rate : 21470400 bits/sec
INFO: [mplex] BUFFERING min 85 Buf max 1944774
INFO: [mplex] MUX STATUS: no under-runs detected

This gives me a good program stream file that is suitable for editing in Cinelerra using Monty's changes. Update 2010/09/13: Florian Cramer advised me that the video render string does not give the highest quality output. I'm still working on improvements to the string that will render at a higher quality.

Moral of the story: never update your system.
;)
the mule

Reference
http://man-wiki.net/index.php/1:mplex
Minor headache with nvidia drivers after kernel upgrade resolved
Plus another pulseaudio fuck job.."pulseaudio module-alsa-card file not found"
- had to reinstall pulseaudio and then execute "alsamixer -c0" to fix

best practices to alleviate audio sync problems in Cinelerra

Audio synchronization problems are the bane of video editors everywhere. Here's what I do in Cinelerra to try to prevent them.

The first thing to understand is that playback in Cinelerra is a separate process from rendering (exporting) your project to a final format. So it is a good idea to calibrate the audio-video synchronization of your files when they are played back in the Cinelerra compositor as well as the a/v sync of rendered output. More specifically:
1) You need to calibrate the audio you see being played back in the compositor. Do this by setting Audio Offset in the Preferences -> Playback -> Audio Out so that your playback audio is in sync with the video. I generally use ALSA as my audio driver under Preferences -> Playback. (You might also want to try to use OSS or ESound to see if those audio drivers help your playback sync issues.) For reference, my offset is .1 seconds.

2) You'll need to calibrate the audio and video once it is rendered out of Cinelerra. This doesn't have to be the absolute final format of the file. Just an intermediate file format to determine that audio is not drifting and in sync. You'll do this by setting Nudge on your audio tracks so that your rendered audio and video are in sync. Here's an article on nudge:
http://crazedmuleproductions.blogspot.com/2007/05/nudge-avidemux2-and-reminder-about.html

Let me give you an example.

My projects usually run about an hour on the timeline. When I start a project, I will concatenate a bunch of clips or raw videos on the timeline. I will then render a short segment of video, about 15 seconds long, from about 45 minutes into the timeline. The segment I take should have key features like someone speaking or someone hitting a drum that I can use to determine audio synchronization. I then render the file to a format that works well in the various Linux media players:
-Quicktime for Linux using MPEG4 video compression and MP4 or twos complement (pcm) audio compression
-Quicktime for Linux container using JPEG video compression and MP4 or twos complement

The side benefit of using MPEG4 video compression is that it renders damn fast.

The good thing about these steps is that if you work with the same format of video over and over, you won't need to change #1, the Audio Offset.

Here are notes regarding specific source formats and output formats.

MPEG SOURCE FILES to MPEG RENDERED OUTPUT
After much study and pain, I've found that when working with MPEG-TS HDV files as my source (and MPEG formats in general), the best pipe to render from Cinelerra in order to alleviate sync headaches is mpeg2enc. I'm not five channel yet, so I just export my 48.1Khz stereo audio as MPEG Layer II audio MP3s @ 384kbps. I then use mplex to combine my audio and video streams into a program stream. Further on, I use VLC to convert that PS into MPEG-TS (HDV) and then use FFMPEG to convert the HDV into DVD and iPod/iPhone compatible formats.

QUICKTIME (QT) SOURCE FILES to QT RENDERED OUTPUT
If my source video format is Quicktime for Linux (like from a screen capture), then I have no sync problems rendering out to QT for Linux.

MPEG SOURCE FILES to QT RENDERED OUTPUT
If I render my MPEG-TS, HDV projects using Quicktime for Linux as an output format (MPEG4 or JPEG video w/twos complement or MP4 audio compression), then I will get sync problems in the rendered output. I then follow the above best practices to alleviate the problem.

TEST YOUR OUTPUT
In Fedora, I've found that the best media players to test final output are mplayer, vlc and ffmpeg.

Finally, there are options to mplex and ffmpeg to offset audio and video sync problems:
mplex -O ; eg, mplex -O 350
ffmpeg -itsoffset ; eg, ffmpeg -itsoffset 0.35

Hope this helps,
the mule

Reference
My Article on Cinelerra's "nudge" parameter
FFMPEG HowTo
http://man-wiki.net/index.php/1:mplex

the dark of winter has me in its grasp

The Mule has been working long hours for himself and you, valued video compatriots!

That sounds positive, as it should be. Though in truth, I am feeling less positive than that message implies. Personal and professional life has got me down, but is par for the course these days. Oh well. A pithy quote to pick myself up would be rather nice here. Instead, let me regale you of the past weeks activities, as some of the tribulations may help individuals in similar need.

Sh*t Storm
This week, as I look back at my notes, I see a hailstorm of problems that I've dealt with:
-Fedora 10, x86-64 spontaneous system lockups/reboots (workaround: noapic on kernel cmd line)
-pulseaudio screwing up my audio
-usb keyboard stops working (workaround: disable keyboard acceleration)
-Gnome session saving broken (the workaround seems more of a pain than its worth)
-1080p editing eats RAM! (bought more RAM)
-Belkin firewire card causing reboots
-I didn't order my RAM in matched pairs, so I'm stuck waiting until Monday for RAM! (finally got it!)
-Evolution has trouble fetching mail from Comcast's POP servers, so I've reverted to use Pine (now "Alpine")

Needless to say, my productivity dropped and frustration was running high.

The Good News
Knock on wood, I think I was able to workaround the spontaneous reboots using "noapic" boot option to the kernel. Whereas the box was rebooting every six hours, now it has been up a full two days without a reboot! Of course, this isn't a true fix and I will have to submit a bug to the Fedora team. And the other problems still exist.

Most importantly, I've discovered a new scheme for solid, fast 1080P editing in Cinelerra:
1) convert Canon 5D video to MPEG2-TS
2) import into Cinelerra
3) render to any format you need

A Couple of Options
In my initial post on editing Canon 5D video, I found that the easiest way for me to get content from the Canon 5D into Cinelerra was using a conversion to MJPEG. However, the drawback with using mjpeg is that the image quality is lacking. Specifically, the output is darker than the original content. So over the past week, I found two solutions to convert the beautiful output of the Canon:

1) convert to H264 using this two pass string:
#CONVERT CANON USING H264, pass 1
ffmpeg -y -i INPUT.MOV -an -v 1 -threads 8 -vcodec libx264 -aspect 1.7777 -b 9000k -bt 7775k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 1 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 10000k -bufsize 2M -cmp 1 -f mp4 -pass 1 /dev/null

#CONVERT CANON USING H264, pass 2
ffmpeg -y -i INPUT.MOV -v 1 -threads 8 -vcodec libx264 -aspect 1.7777 -b 9000k -bt 7775k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 1 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 10000k -bufsize 2M -acodec libfaac -ab 160k -ar 48000 -ac 2 -cmp 1 -f mp4 -pass 2 OUTPUT.mp4

Now, this H264 content is beautiful, will import into Cinelerra and is editable. However, I found that when I went to render the final output, four minutes of the 1080p, H264 content took SIX HOURS to render!! That is unacceptable. I believe the lengthy render time has something to do with the color space or internal conversion that Cinelerra is doing. This bears further research.

If you're not familiar with H264 (x264 libraries on Linux), here's some useful H264 reference material.

2) convert to MPEG2-TS

Converting Canon to 1080p, MPEG2-TS
Now, there are a few steps here.

a. Take a file from the Canon and use ffmpeg to pass a lossless yuv4mpegpipe stream into mpeg2enc, with the result a video stream with no audio:
ffmpeg -i INPUT.MOV -threads 8 -s 1920x1088 -f yuv4mpegpipe - | mpeg2enc --multi-thread 8 --verbose 0 --aspect 3 --format 13 --frame-rate 5 --video-bitrate 24000 --nonvideo-bitrate 384 --interlace-mode 0 --force-b-b-p --video-buffer 448 --video-norm n --keep-hf --no-constraints --sequence-header-every-gop --min-gop-size 6 --max-gop-size 6 -o OUTPUT.m2v

Next, render out the audio:
ffmpeg -y -i INPUT.MOV -acodec mp2 -ar 44100 -ab 256k -ac 2 OUTPUT.m2a

Using mplex, mux the video and audio streams together:
mplex -f 3 -b 2000 OUTPUT.m2a OUTPUT.m2v -o OUTPUT.ps

Using VLC, convert the MPEG2-PS into an MPEG2-TS:
cvlc OUTPUT.ps --sout '#duplicate{dst=std{access=file,mux=ts,dst="OUTPUT.m2t"}}' vlc://quit

Update 2009/02/13
I've found that VLC is not writing proper keyframes at the beginning of the converted MPEG-PS video output from mplex. This is only for 1080p video. The VLC command for 720p video still works. For the 1080p, I've found a workaround using our savior, ffmpeg:
ffmpeg -y -i OUTPUT.ps -acodec copy -f mpegts -qscale 1 OUTPUT.m2t
*** end update ***

I used this method to output a new version of my Water video from Cinelerra to Vimeo here:
/2009/01/water-new-canon-5d-video.html

The quality and the colors are definetly improved upon over the old version. However, the larger file size is a drawback (479MB for 4m16s of video). So I'd like to get the H264 output without compression artifacts during the scenes with a lot of motion. So now its time to figure that out. Erg.

In general though, I think this is some good news!

Until the next time,
the mule

cam compatible HDV edit chain, part II (short)

I posted a more detailed version of this two posts ago (http://crazedmuleproductions.blogspot.com/2006_10_01_archive.html), but I thought I'd reiterate it in a shorter format:
1) from Cinelerra, export video using YUV4MPEG using mpeg2enc (.m2v) compression scheme with special params:
mpeg2enc --verbose 0 --aspect 3 --format 3 --frame-rate 4 --video-bitrate 18300 --nonvideo-bitrate 384 --interlace-mode 0 --force-b-b-p --video-buffer 448 --video-norm n --keep-hf --no-constraints --sequence-header-every-gop --min-gop-size 6 --max-gop-size 6 -o %
2) from Cinelerra, export audio using mpeg audio, layer II (384Kbps)
3) multiplex audio and video streams to MPEGPS (Program Stream) format using mplex:
mplex -f 3 -b 2000 video.m2v audio.m2a -o output.ps
4) convert .ps to .ts, MPEGTS (Transport Stream) format using VLC
5) export to cam, consider using 10sec blank leader

Update 5/17/2009
Note that if you bring the video into Cinelerra and the thumbnail is green, this means that the first frame of the video is not a keyframe. This is a problem with VLC. You can avoid this problem by using this ffmpeg command in place of Step 4 above:
ffmpeg -y -i OUTPUT.ps -acodec copy -f mpegts -qscale 1 OUTPUT.m2t
*** end update ***

Update 3/14/2008
Once the video is converted, you may then output to the camera using test-mpeg2 from libiec61883:
/2006/10/success-and-failure-in-land-of-dvhdv.html

Also, take note of a possible 2GB limit in libiec61883 uploading your content to the cam:
/2006/10/libiec61883-now-exports-files-greater.html
*** end update ***

Update 11/17/06
Last night, I found that doing performing other operations on your system like surfing the web or playing audio while running mplex or VLC convert may cause problems with the video as it uploads to the camera. So don't do anything while you convert your videos.

In my case, I was 28 minutes into an hour long video uploading to the cam when the import stopped. I no longer had uploading video displayed on the LCD screen of the cam. I only saw blue, the upload seemed to continue, but I got no final output. The solution was to remux the original streams using mplex and rebuild the MPEGTS using VLC. The important point being that one of these two processes is very sensitive to fluctuations in CPU or hard drive performance. So leave your PC alone while these processes are active.
*** end update ***

That's all she wrote for today. Enjoy!
the mule

rendering cam-compatible HDV MPEGTS files

I have been successful in exporting HDV back out to my cam. I haven't posted detailed output from the individual steps before, so here it is:

General Steps
1) export video from Cinelerra as mpeg2 using mpeg2enc
2) export audio from Cinelerra as mp3
3) mux in mplex
4) open MPEGPS in VLC and convert to TS.

Detail
1) export video from Cinelerra as mpeg2 using mpeg2enc:
mpeg2enc --verbose 0 --aspect 3 --format 3 --frame-rate 4 --video-bitrate 18300 --nonvideo-bitrate 384 --interlace-mode 0 --force-b-b-p --video-buffer 448 --video-norm n --keep-hf --no-constraints --sequence-header-every-gop --min-gop-size 6 --max-gop-size 6 -o /mnt/videos/20050721/20050721.m2v

2) export audio from Cinelerra as mp3:
--------------------------------------------
Input File : 'stdin' 48.0 kHz
Output File: '/mnt/videos/20050721/test.mp3'
384 kbps MPEG-1 Layer II j-stereo Psy model 1
[De-emph:Off Copyright:No Original:No CRC:Off]
[Padding:Normal Byte-swap:Off Chanswap:Off DAB:Off]
ATH adjustment 0.000000
--------------------------------------------
encode_init: using tablenum 0 with sblimit 27
Hit end of audio data
Avg slots/frame = 1152.000; b/smp = 8.00; bitrate = 384.000 kbps

3) mux in mplex
[root@computer 20050721]# mplex -f 3 -b 2000 -o test.ps test.mp3 20050721.m2v
INFO: [mplex] mplex version 1.8.0 (2.2.4 $Date: 2005/08/28 17:50:54 $)
INFO: [mplex] File test.mp3 looks like an MPEG Audio stream.
INFO: [mplex] File 20050721.m2v looks like an MPEG Video stream.
INFO: [mplex] Found 1 audio streams and 1 video streams
INFO: [mplex] Selecting generic MPEG2 output profile
INFO: [mplex] Multiplexing video program stream!
INFO: [mplex] Scanning for header info: Audio stream c0 (test.mp3)
INFO: [mplex] MPEG AUDIO STREAM: c0
INFO: [mplex] Audio version : 1.0
INFO: [mplex] Layer : 2
INFO: [mplex] CRC checksums : no
INFO: [mplex] Bit rate : 49152 bytes/sec (384 kbit/sec)
INFO: [mplex] Frequency : 48000 Hz
INFO: [mplex] Mode : 0 stereo
INFO: [mplex] Mode extension : 0
INFO: [mplex] Copyright bit : 0 no copyright
INFO: [mplex] Original/Copy : 0 copy
INFO: [mplex] Emphasis : 0 none
INFO: [mplex] Scanning for header info: Video stream e0 (20050721.m2v)
INFO: [mplex] VIDEO STREAM: e0
INFO: [mplex] Frame width : 1280
INFO: [mplex] Frame height : 720
INFO: [mplex] Aspect ratio : 16:9 display
INFO: [mplex] Picture rate : 29.970 frames/sec
INFO: [mplex] Bit rate : 18300000 bits/sec
INFO: [mplex] Vbv buffer size : 229376 bytes
INFO: [mplex] CSPF : 0
INFO: [mplex] SYSTEMS/PROGRAM stream:
INFO: [mplex] rough-guess multiplexed stream data rate : 19077648
INFO: [mplex] Setting best-guess data rate.
INFO: [mplex] Run-in Sectors = 752 Video delay = 58123 Audio delay = 61126
INFO: [mplex] New sequence commences...
INFO: [mplex] Audio c0: buf= 0 frame=000000 sector=00000000
INFO: [mplex] Video e0: buf= 0 frame=000000 sector=00000000
INFO: [mplex] Scanned to end AU 39469
INFO: [mplex] STREAM e0 completed @ frame 39469.
INFO: [mplex] STREAM c0 completed @ frame 54874.
INFO: [mplex] Multiplex completion at SCR=118584775.
INFO: [mplex] Audio c0: buf= 2304 frame=054874 sector=00031342
INFO: [mplex] Video e0: buf= 0 frame=054843 sector=00509538
INFO: [mplex] AUDIO_STATISTICS: c0
INFO: [mplex] Audio stream length 63216000 bytes.
INFO: [mplex] Syncwords : 54875
INFO: [mplex] Frames : 54875 padded
INFO: [mplex] Frames : 0 unpadded
INFO: [mplex] BUFFERING min 18 Buf max 1169
INFO: [mplex] VIDEO_STATISTICS: e0
INFO: [mplex] Video Stream length: 1031417371 bytes
INFO: [mplex] Sequence headers: 6579
INFO: [mplex] Sequence ends : 1
INFO: [mplex] No. Pictures : 39470
INFO: [mplex] No. Groups : 6579
INFO: [mplex] No. I Frames : 6579 avg. size 58215 bytes
INFO: [mplex] No. P Frames : 32891 avg. size 19714 bytes
INFO: [mplex] No. B Frames : 0 avg. size 0 bytes
INFO: [mplex] Average bit-rate : 6264800 bits/sec
INFO: [mplex] Peak bit-rate : 9203600 bits/sec
INFO: [mplex] BUFFERING min 943996 Buf max 1944874
INFO: [mplex] MUX STATUS: no under-runs detected.
[root@computer 20050721]#


4) open MPEGPS in VLC and convert to MPEGTS.

Update 3/14/2008
Once the video is converted, you may then output to the camera using test-mpeg2 from libiec61883:
/2006/10/success-and-failure-in-land-of-dvhdv.html

Also, take note of a possible 2GB limit in libiec61883 uploading your content to the cam:
/2006/10/libiec61883-now-exports-files-greater.html

HDV MPEG2 transport stream file sizes/render rates

The data rate of the 720P MPEG2-TS files output from my cam is about 108.95MB/min or 1.82MB/s. Here is a table of video length-to-size conversions.

duration size

12m 1.32GB
15m 1.65GB
18m35s 2.07GB
19m12s 2.12GB
20m01s 2.18GB
34m 3.70GB
Exporting 720P HDV from Cinelerra takes two processes:
1) render the video
2) render the audio

Here are some rendering times using mpeg2enc and mpeg layer 2 audio compression:

duration mpeg2enc render rate
63m 310m 4.92min per min of video

duration mp2 render rate
63m 6m 0.09min per min of audio

Mplex takes about 7 minutes to mux about an hour of audio and video.

hard work is paying off / HDV workflow

OK! I've got the latest video up on iTunes and now I just need to edit the XML for it. Editing the damn XML always takes too long. And of course, I want to get the track list right with the correct track times and witty comments. Ufff. OK! So the track list is done and now to send the boys the email for the latest video! Hurrah!

More for my benefit than anyone elses, here is my HDV workflow using Cinelerra:
1) import video
2) edit video
3) for multiple segments, make sure you set "align to frames" and audio and video segments end at the exact same time.
4) for HD output, render using mpeg2enc settings
5) for DVD output, render using ffmpeg -target DVD from Cinelerra, something like this:
ffmpeg -f yuv4mpegpipe -i - -y -i /mnt/videos/20050721/20050721.wav -target dvd %
6) for iPod output, render using ffmpeg -target DVD, then output to mpeg4 video/aac audio mov container:
ffmpeg -i inputdvd.mpg -f mov -vcodec mpeg4 -qscale 7 -s 320x180 -r 29.97 -aspect 16:9 -acodec aac -ac 2 -ab 128 output.mov
7) import .mov to XP iTunes
8) upload .mov to website
9) edit RSS XML
10) final test in iTunes

exporting HD content to the cam: IT IS ALIIIIVE!! ALIVE!!

After a month of testing and experimentation with many inter-OS softwares, exporting HD content to my camera is working! YAHOO!! The greatest part is..I am Microsoft FREE!

Most of my work was based off of this original thread:
http://www.dvinfo.net/conf/archive/index.php/t-19007.html

But a number of steps (ffmpeg/DVHSCap) did not work for me.

Here are the steps I used to get HD content from Cinelerra back to my cam in full resolution, HD 720P format:

1) install the latest 2.6.18 kernel according to specs on http://www.linux1394.org
2) you'll need the latest libiec61883 and libraw1394 libraries from the same place . I needed to manually tweak the ./configure procedure for libiec61883 with the following environment variable:
PKG_CONFIG_PATH=/usr/local/lib/pkconfig
Another caveat to this step is that I needed to remove libraw1394 first, which was a nightmare because of all its dependencies. I created a yum install script to reinstall everything after this was complete.
3) you'll need to make a change to tsbuffer.c and recompile libiec61883 (don't have it now..email me if you need it).
4) you'll need to recompile mpeg2enc to allow for bitrates larger than 14000. This is referenced in the thread from way back when:
http://www.dvinfo.net/conf/archive/index.php/t-19007.html
--------------------
A kludgy hack allowed me to get bit rates higher than 15Mbs in mpeg2enc

I removed the following lines from conform.cc
if (bit_rate> 1.0e6 * maxval->bit_rate)
mjpeg_error_exit1("Bit rate is greater than permitted in specified Level");
---------------------
5) here are the mpeg2enc switches that were successful for me:
mpeg2enc --verbose 0 --aspect 3 --format 3 --frame-rate 4 --video-bitrate 18300 --nonvideo-bitrate 384 --interlace-mode 0 --force-b-b-p --video-buffer 448 --video-norm n --keep-hf --no-constraints --sequence-header-every-gop --min-gop-size 6 --max-gop-size 6 -o mpeg2.mpv
6) mplex params:
mplex -f 3 -b 2000 -o programstream.ps audio.mp2 mpeg2.mpv
7) Use VLC to convert the file to a transport stream. By the way, the VLC is a hellish snake pit of install dependencies. Go to http://www.videolan.org to download the latest source.

Here are the instructions for VLC from Paul St. Denis' original steps (http://www.celt.sunysb.edu/paul/mpeg2enc_HDTV.zip):
------------------
Start VLC, from the "file" menu select "open file", click on the "Browse..." button and find "programstream.ps", check on the "advanced output" check box and then click on the "Settings.." button. Make sure the "file" radio button is picked click on the "Browse..." button save the file on your desktop as "transport.ts".

Make sure that the "Encapsulation Method" is MPEG TS, leave everything else blank. Click OK, which brings you back to the other screen click OK again.
------------------

Yes, it's a lot of work. But to be Microsoft free is a great feeling.

Update: as of this writing (10/17), I'm only able to upload less than 2GB of video. For my 720P video cam, this is about 19.7 minutes of video. I am working with the linux1394 community to resolve this.