recording desktop with ffmpeg

I had this idea that I'm going to start interviewing folks on different technical topics via Skype or Google Hangout.  Normally, I would use Cinelerra to perform this task, but for some reason, the recordings in Cinelerra showed some irritating sound artifacts.  Testing in audacity showed no such audio problems.  I'll bring this up on the Cin mailing list soon.

FFmpeg to the Rescue
So with Cinelerra not working for desktop recordings, I needed an alternate way to record my desktop screen as well as the audio from the person being interviewed.  After some searching a good deal of trial and error, I found from the link in the reference section at the bottom of the page, a very nice ffmpeg command.  You can save to any format you want.  The example below shows how to save the microphone audio (hw:0,0) in PCM format.  The desktop video is brought in via the xllgrab format specifier :
ffmpeg -f alsa -i hw:0,0 -ac 2 -ar 48000 -acodec pcm_s16le -f x11grab -r 24 -s 1280x720 -i :0.0 -aspect 16:9 -vcodec libx264 -vpre lossless_ultrafast -threads 8 -y output.mov

I later tweaked the command to do a bit more, using pulse audio as input as well as saving out to an iPod compatible format:
ffmpeg -f alsa -i pulse -f x11grab -s 1280x720 -r 23.98 -i :0.0 -vcodec libx264 -acodec pcm_s16le -ab 128k -ar 48000 -ac 1 -b 2000k -bt 750k -refs 1 -deblockalpha 0 -deblockbeta 0 -subq 1 -me_range 21 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 2000k -bufsize 2M -cmp 1 -threads 8 output.mov

Errors
Not sure how yet to resolve these ffmpeg errors:
[alsa @ 0x16512a0] Estimating duration from bitrate, this may be inaccurate
Input #0, alsa, from 'hw:0,0':
  Duration: N/A, start: 17479.447401, bitrate: N/A
    Stream #0.0: Audio: pcm_s16le, 32000 Hz, 2 channels, s16, 1024 kb/s
[x11grab @ 0x1650240] device: :0.0 -> display: :0.0 x: 0 y: 0 width: 1280 height: 720
[x11grab @ 0x1650240] shared memory extension found
[x11grab @ 0x1650240] Estimating duration from bitrate, this may be inaccurate
Input #1, x11grab, from ':0.0':
  Duration: N/A, start: 1327197224.425685, bitrate: 707788 kb/s
    Stream #1.0: Video: rawvideo, bgra, 1280x720, 707788 kb/s, 24 tbr, 1000k tbn, 24 tbc
Incompatible pixel format 'bgra' for codec 'libx264', auto-selecting format 'yuv420p'

Along the way, I received an error:
"unknown field slave"

Finding a bit more about ALSA
This ended up being because I had edited my /etc/asound.conf file from the default.  This corrupted my sound card settings.  I was reacquainted with some debugging commands and places where "sound stuff" lives:
[sodo@computer etc]$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: U0x46d0x821 [USB Device 0x46d:0x821], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0
card 2: MobilePre [MobilePre], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

[sodo@computer etc]$ amixer -c 2
Simple mixer control 'Speaker',0
  Capabilities: pvolume pswitch pswitch-joined penum
  Playback channels: Front Left - Front Right
  Limits: Playback 0 - 31
  Mono:
  Front Left: Playback 31 [100%] [0.00dB] [on]
  Front Right: Playback 31 [100%] [0.00dB] [on]
Simple mixer control 'PCM',0
  Capabilities: pvolume pswitch pswitch-joined penum
  Playback channels: Front Left - Front Right
  Limits: Playback 0 - 31
  Mono:
  Front Left: Playback 25 [81%] [0.00dB] [on]
  Front Right: Playback 25 [81%] [0.00dB] [on]
Simple mixer control 'PCM',1
  Capabilities: volume volume-joined penum
  Playback channels: Mono
  Capture channels: Mono
  Limits: 0 - 2
  Mono: 2 [100%]
Simple mixer control 'Analog In',0
  Capabilities: pvolume cvolume pswitch pswitch-joined cswitch cswitch-joined penum
  Playback channels: Front Left - Front Right
  Capture channels: Front Left - Front Right
  Limits: Playback 0 - 31 Capture 0 - 31
  Front Left: Playback 28 [90%] [6.00dB] [off] Capture 31 [100%] [12.00dB] [on]
  Front Right: Playback 28 [90%] [6.00dB] [off] Capture 31 [100%] [12.00dB] [on]

[sodo@computer ~]$ cat /proc/asound/cards
 0 [U0x46d0x821    ]: USB-Audio - USB Device 0x46d:0x821
                      USB Device 0x46d:0x821 at usb-0000:00:1d.7-6, high speed
 1 [MobilePre      ]: USB-Audio - MobilePre
                      M Audio MobilePre at usb-0000:00:1d.0-1, full speed

[sodo@computer etc]$ ls /dev/snd

by-id  by-path  controlC0  controlC2  pcmC0D0c  pcmC2D0c  pcmC2D0p  seq  timer

[sodo@computer etc]$ cat /etc/asound.conf

#

# Place your global alsa-lib configuration here...

#

@hooks [

{

func load

files [

"/etc/alsa/pulse-default.conf"

]

errors false

}

]

XWinInfo
Also, I learned about xwininfo, a cool command that shows you your desktop settings:
[sodo@computer videoConfTest]$ xwininfo
xwininfo: Window id: 0x320003b "Hak5 - Linux Screen Recording, Boxee Python Development and Qnext - YouTube - Google Chrome"

  Absolute upper-left X:  0
  Absolute upper-left Y:  53
  Relative upper-left X:  0
  Relative upper-left Y:  22
  Width: 1280
  Height: 971
  Depth: 24
  Visual: 0x21
  Visual Class: TrueColor
  Border width: 0
  Class: InputOutput
  Colormap: 0x20 (installed)
  Bit Gravity State: NorthWestGravity
  Window Gravity State: NorthWestGravity
  Backing Store State: NotUseful
  Save Under State: no
  Map State: IsViewable
  Override Redirect State: no
  Corners:  +0+53  -1280+53  -1280-0  +0-0
  -geometry 1280x971+0-0

Finally, I installed Skype and dealt with resolving a bunch of dependency problems (as usual):

http://kezhong.wordpress.com/2011/09/02/installing-skype-on-fedora-15x86_64/

VideoCamming
And a simple way to trigger mplayer to open up your videocam:
mplayer tv:// -tv driver=v4l2 device=/dev/video0

Another simple way to capture the output:
ffmpeg -f alsa -i hw:0 -f video4linux2 -s 320x240 -i /dev/video0 out.mpg

Or a more complex example, saving the video into an iPod friendly format:
ffmpeg -f alsa -i pulse -f video4linux2 -i /dev/video0 -aspect 16:9 -s 1280x720 -r 23.98 -vcodec libx264 -acodec libfaac -ab 128k -ar 48000 -ac 1 -b 2000k -bt 750k -refs 1 -deblockalpha 0 -deblockbeta 0 -subq 1 -me_range 21 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 2000k -bufsize 2M -cmp 1 -threads 8 output.mov

Finally, guvcview is a very nice program to control one's videocam:

Reference
http://www.linuxquestions.org/questions/linux-newbie-8/recording-desktop-with-ffmpeg-877580/
http://www.youtube.com/watch?v=mNz5Lrc06_s
http://forums.fedoraforum.org/showthread.php?t=256681
http://ffmpeg.org/ffprobe.html#pulse
http://alien.slackbook.org/blog/adding-an-alsa-software-pre-amp-to-fix-low-sound-levels/
http://forums.opensuse.org/english/get-technical-help-here/multimedia/471522-record-my-desktop-using-ffmpeg-post2433494.html
http://superuser.com/questions/251338/mencoder-capture-from-camera-and-microphone
http://ubuntuforums.org/showthread.php?t=1556038 (using jack)
http://wiki.linuxaudio.org/wiki/screencasttutorial (jack_capture)
https://help.ubuntu.com/community/Webcam (webcam)
http://howto-pages.org/ffmpeg/ (a nice, human readable version of the FFmpeg doc)
http://www.tldp.org/HOWTO/Alsa-sound-6.html (amixer settings)

audio sync, nudge, avidemux2 and a reminder about archiving

I spent most of the night trying to debug an audio synchronization problem in Cinelerra. The video was trailing the audio by a little bit. So I used the Nudge feature to slide the audio .5 seconds behind to the right.

A word about nudge:
If the audio is running behind the video, use POSITIVE values for nudge.
If the video is running behind the audio, use NEGATIVE values for nudge.

Another way to put it:
If your audio is lagging and is behind the video (more to the RIGHT on the timeline than the video), push it FORWARD using positive values.
If your audio is running too fast and is ahead of the video (more to the LEFT on the timeline than the video), push it BACKWARD using negative values.

2/17/2008
Since nudge is good for fixing an entire track, one way to fix synchronization problems that occur on only part of a track is to use the Delay Video video effect or the Delay Audio audio effect:

• The Delay Video track works when audio lags behind the video.
• The Delay Audio track works when the video lags behind the audio.

The two effects take positive values for the delay in seconds. So they only work "one way", so to speak.
:)

This syncronization problem was after another discovery that Cinelerra on the Fedora Core 6 box I just built is not letting me drag and drop video. So I couldn't manually slide my video around. I'm not sure why this is not working for this particular project, as the Record button is enabled and my editing Preferences seem correct. The funny thing is that it does not happen when I create a new project. My gut feeling is that this version of Cinelerra I am using is SLIGHTLY newer than the one on my main box. So the XML is probably screwed up in some way. Ugh. Tiring.

But not tired enough to write a reminder to everybody that if they want the MOST PORTABLE video files, keep them below 4GB. That way they will fit on a DVD for archiving or transferring, fit on a FAT32 file system and be welcomed on any modern file system. Any larger, and it becomes a nightmare to transfer or archive. As I edit in HDV, file sizes get huge pretty quickly. But keep 'em below 4GB kids! It'll save you much pain in the long run.

Your friend in the battle to trim files down to size is AVIDEMUX2, a wonderful little program I've mentioned before that chops up large video files in no time flat. On my system, a Dell 400SC, 3.2Ghz, 2GB RAM, multiple drives, it took about four minutes total to chop up a 5GB HDV file into two 2.5 chunks. Not bad! One thing I do is to make sure that when you are saving out your file, use Copy for both the Audio and Video tracks and then save to your final format. The Copy feature will save out a new file to the exact same specs as your source file. Very nice feature!

MediaGate - Ximeta NDAS troubles with Linux

The MediaGate MG-350HD is a little wireless device that allows me to view my Cinelerra creations on my HDTV.


So, I'm trying to get it to work smoothly with Linux, as that's where my HD source files are. However, HD files will only play on the MediaGate if the files are either:
1) accessed via network share over wired connection or
2) copied via NDAS drivers to the hard drive installed in the thing

Ximeta (http://www.ximeta.com/) are the makers of the network direct attached storage (NDAS) that the MediaGate uses. I don't have a hard run down to my entertainment center, so I've been pulling my hair out in order to get the NDAS drivers working. The problem with this is that the drivers seem to work on XP only. The Linux drivers on Ximeta's site don't work worth a damn. They fail on my fedora and redhat enterprise boxes and on the two Ubuntu installs (6.0.6/6.1.0) that I did this morning. Yarg. In addition, I've tried upgrading the firmware to no avail. The frustrating thing is that the firmware is supposed to fix a problem where the MG can't do wired and wireless connections at the same time. In point of fact, the firmware upgrade has BROKEN wireless. Double YARG!!

2/11 Update: I've since fixed the wireless by first doing an AP (access point) scan of the enviroment first, and then changing the wireless mode to Ad Hoc to configure the security settings of my specific router. Gads!

So I think I'm just going to leave it wired upstairs for now, copy over the HD content when its ready and bring it back down to the HDTV. What a waste of time..I'm going to send the bozos an email.

2/19/2007 Update: A wirelessly mounted MediaGate is now working!!

Cinelerra hanging after I move audio on the timeline

This is a bit off topic, but is related to troubleshooting Cinelerra. I have been having a problem with Cinelerra hanging for three or four minutes after I zoom in and slide audio tracks around on the timeline. To replicate the problem, I grabbed and moved the audio track (a wav file) around quite a bit, perhaps 10 times. In order to find the hanging thread, I started gdb ("gdb cinelerra pid").
[root@localhost ~]# ps -ef | grep cin
root 3671 3571 90 20:05 pts/2 00:17:55 cinelerra
root 4263 4178 0 20:25 pts/4 00:00:00 grep cin
[root@localhost ~]# gdb cinelerra 3671

Scrolling through the thread list, I identified the offending thread:

> Thread 65 (Thread -1520473168 (LWP 5674)):
> #0 0x0813fceb in CacheBase::get_oldest (this=0x8c04068) at cachebase.C:106
> #1 0x081f00bb in MWindow::age_caches (this=0xbfc5a288) at
> mwindow.C:1569
> #2 0x081f5e7b in MWindow::move_edits (this=0xbfc5a288, edits=0x8539b70,
> track=0x920f238, position=49.724958333333333, behaviour=1) at
> mwindowedit.C:830
> #3 0x08289ed6 in TrackCanvas::drag_stop (this=0xa9e1e7e0) at
> trackcanvas.C:575
> #4 0x0828a13d in TrackCanvas::drag_stop_event (this=0xa9e1e7e0) at
> trackcanvas.C:325
> #5 0xb7b73029 in BC_WindowBase::dispatch_drag_stop (this=0xa9e1e7e0) at
> bcwindowbase.C:1316
> #6 0xb7b7300e in BC_WindowBase::dispatch_drag_stop (this=0x8c44da0) at
> bcwindowbase.C:1311
> #7 0xb7b73122 in BC_WindowBase::dispatch_button_release
> (this=0x8c44da0) at bcwindowbase.C:1162
> #8 0xb7b778b1 in BC_WindowBase::dispatch_event (this=0x8c44da0) at
> bcwindowbase.C:787
> #9 0xb7b78739 in BC_WindowBase::run_window (this=0x8c44da0) at
> bcwindowbase.C:614
> #10 0xb7b88bb8 in Thread::entrypoint (parameters=0xbfc5a288) at
> thread.C:48
> #11 0x00da0b80 in start_thread () from /lib/libpthread.so.0
> #12 0x00c11dee in clone () from /lib/libc.so.6

I then exited gdb and found that the process hang was still occurring.
In order to step through the program code, I use kdbg, a graphical debugger. I started up kdbg ("kdbg -p /usr/local/bin/cinelerra") and
selected "View -> Threads". I then used an alternating combination of
F8 and F10 to step in and out of the code. I found that kdbg hung at
this line of code: MWindow.C:1561. It took a while, maybe five minutes
for kdbg to free up. This indicates where the hang is occurring. After the release, I exited the debugger.

Here is the snippet of hvirtual/cinelerra/mwindow.C code starting at
line 1561:

> if(memory_usage > preferences->cache_size)
> {
> int target = 1;
> int oldest1 = audio_cache->get_oldest();
> int oldest2 = video_cache->get_oldest();
> if(oldest2 < target =" 2;"> int oldest3 = frame_cache->get_oldest();
> if(oldest3 <> target = 3;
> int oldest4 = wave_cache->get_oldest();
> if(oldest4 <> oldest4 < target =" 4;"> switch(target)
> {
> case 1: audio_cache->delete_oldest();
> break;
> case 2: video_cache->delete_oldest();
> break;
> case 3: frame_cache->delete_oldest();
> break;
> case 4: wave_cache->delete_oldest();
> break;
> }
> }

I'm not a very experienced C programmer, but inferring from the if statement, it looks like I only go into this control structure if cin's memory usage is
greater than the value stated in Preferences -> Cache Size. I have it
set to the default value of 10MB.

I have not yet found the cause of this particular hang, but I will update my blog once I have an answer.

trials of Cinelerra rendering, deja vu all over again

Well, four hours of overnight rendering yields a video I can't playback. I get this error:
int YUVStream::read_header(): y4m_read_stream_header() failed: bad header magic
int YUVStream::open_read(char*): Bad YUV4MPEG2 header: parameter out of range

Now what the F is wrong? Is it a problem with:
- my rendering options I chose?
- this version of ffmpeg in general?
- my project?

I wasted a number of hours on the first two possibilities. I finally saw two issues in my project:
1) for some reason, probably my big fat fingers, an audio transition was placed right at the beginning of the project (Timecode 00:00:00). I deleted this.
2) I noticed that for joins between separate videos on the same a/v tracks, the joins between the audio and video segments did not end at the exact same place:
http://content.serveftp.net/video/avmisaligned.png

To stop this, I went to the end of each segment, zoomed in all the way, and made sure that each video and audio segment ended at the exact same time. If they didn't end at the exact same time, I clipped off a bit of whatever audio or video track is dangling by using the mark in/mark out indicators and doing a "cut". After doing the cut, the segments aligned perfectly.
Even with these changes, however, I still get a similar version of my original error when I load the project:
int YUVStream::read_header(): y4m_read_stream_header() failed: bad stream or frame header
int YUVStream::open_read(char*): Bad YUV4MPEG2 header: parameter out of range
However, I can output it to a usable format, so I don't mind the rather ugly error messages. You'd thought I would have known better by now. ARGH!

Along the way tonight, I've been trying to help the Cinelerra CVS community debug a couple of errors:
1) the intermittent project hangs
2) MS MPEG4 render crashes

I synchronized my hvirtual directory twice and recompiled, once for v932 for the project hang problem and then v940 for the MS MPEG4 render crashes. Both took a couple interations, but it seems like the programmers have the problems licked. That's good. We've been pushing them pretty hard lately.

So, finally around 11pm, I've rendered the video to a usable DVD mpg. Now it's time to render once more; this time to iPod format. It looks like FFMPEG has removed the "-b" bitrate switch in favor of some quality scale switch, "-qscale". The higher the qscale, the lower the quality. To create videos with the same quality as the original, you can use the "-sameq" switch. Otherwise, a qscale of 4 will yield a decent quality video with some significant compression. I'd use h264, but I haven't been able to install it successfully. I may try VLC..

Update 2009/06/09
My understanding of the "-sameq" switch was a little naive. Robert Swain enlightens here:
http://rob.opendot.cl/index.php/2009/05/01/bit-rate-file-size-quality-misunderstandings/

You'll probably need to read it a couple of times like I did.
:)
*** end update ***

Cinelerra compile problem solved, more or less..

To follow up on this thread from a few weeks back, through a fair bit of reading and educated(?) guessing, I was able to get around this problem that stymied me.

One of the guys on the Cinelerra boards did clue me into the fact that the aclocal.m4 file was much too short, as it should be a superset of /usr/share/aclocal/libtool.m4. So that gave me a start. After reading a bit of doc and problem postings in Google regarding aclocal.m4, I saw that the file listing of hvirtual/m4 was about half as long as the list of *.m4 files in /usr/share/aclocal. So, I edited autogen.sh to point aclocal to build aclocal.m4 from the files in /usr/share/aclocal, rather than using the m4 directory under hvirtual. After running autogen, I then had a complete aclocal.m4 file with a libtool section. I did not have the libtool section in the file before building from /usr/share/aclocal.

I don't know what side effects this has, but the CVS compiled and I can playback and render video from my saved projects, so I assume this is a valid solution. The only side effect I've noticed so far is that I get a few more ALSA underruns like this:
AudioALSA::write_buffer underrun

ps - here's some doc I found that may help someone in a similar predicament:
re automake:
http://www.delorie.com/gnu/docs/automake/automake.html#SEC_Top
re libtool:
http://people.debian.org/~keybuk/libtool-updating.html