Thursday, January 26, 2012

recording desktop with ffmpeg

I had this idea that I'm going to start interviewing folks on different technical topics via Skype or Google Hangout.  Normally, I would use Cinelerra to perform this task, but for some reason, the recordings in Cinelerra showed some irritating sound artifacts.  Testing in audacity showed no such audio problems.  I'll bring this up on the Cin mailing list soon.

FFmpeg to the Rescue
So with Cinelerra not working for desktop recordings, I needed an alternate way to record my desktop screen as well as the audio from the person being interviewed.  After some searching a good deal of trial and error, I found from the link in the reference section at the bottom of the page, a very nice ffmpeg command.  You can save to any format you want.  The example below shows how to save the microphone audio (hw:0,0) in PCM format.  The desktop video is brought in via the xllgrab format specifier :
ffmpeg -f alsa -i hw:0,0 -ac 2 -ar 48000 -acodec pcm_s16le -f x11grab -r 24 -s 1280x720 -i :0.0 -aspect 16:9 -vcodec libx264 -vpre lossless_ultrafast -threads 8 -y output.mov

I later tweaked the command to do a bit more, using pulse audio as input as well as saving out to an iPod compatible format:
ffmpeg -f alsa -i pulse -f x11grab -s 1280x720 -r 23.98 -i :0.0 -vcodec libx264 -acodec pcm_s16le -ab 128k -ar 48000 -ac 1 -b 2000k -bt 750k -refs 1 -deblockalpha 0 -deblockbeta 0 -subq 1 -me_range 21 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 2000k -bufsize 2M -cmp 1 -threads 8 output.mov

Errors
Not sure how yet to resolve these ffmpeg errors:
[alsa @ 0x16512a0] Estimating duration from bitrate, this may be inaccurate
Input #0, alsa, from 'hw:0,0':
  Duration: N/A, start: 17479.447401, bitrate: N/A
    Stream #0.0: Audio: pcm_s16le, 32000 Hz, 2 channels, s16, 1024 kb/s
[x11grab @ 0x1650240] device: :0.0 -> display: :0.0 x: 0 y: 0 width: 1280 height: 720
[x11grab @ 0x1650240] shared memory extension found
[x11grab @ 0x1650240] Estimating duration from bitrate, this may be inaccurate
Input #1, x11grab, from ':0.0':
  Duration: N/A, start: 1327197224.425685, bitrate: 707788 kb/s
    Stream #1.0: Video: rawvideo, bgra, 1280x720, 707788 kb/s, 24 tbr, 1000k tbn, 24 tbc
Incompatible pixel format 'bgra' for codec 'libx264', auto-selecting format 'yuv420p'

Along the way, I received an error:
"unknown field slave"

Finding a bit more about ALSA
This ended up being because I had edited my /etc/asound.conf file from the default.  This corrupted my sound card settings.  I was reacquainted with some debugging commands and places where "sound stuff" lives:
[sodo@computer etc]$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: U0x46d0x821 [USB Device 0x46d:0x821], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0
card 2: MobilePre [MobilePre], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

[sodo@computer etc]$ amixer -c 2
Simple mixer control 'Speaker',0
  Capabilities: pvolume pswitch pswitch-joined penum
  Playback channels: Front Left - Front Right
  Limits: Playback 0 - 31
  Mono:
  Front Left: Playback 31 [100%] [0.00dB] [on]
  Front Right: Playback 31 [100%] [0.00dB] [on]
Simple mixer control 'PCM',0
  Capabilities: pvolume pswitch pswitch-joined penum
  Playback channels: Front Left - Front Right
  Limits: Playback 0 - 31
  Mono:
  Front Left: Playback 25 [81%] [0.00dB] [on]
  Front Right: Playback 25 [81%] [0.00dB] [on]
Simple mixer control 'PCM',1
  Capabilities: volume volume-joined penum
  Playback channels: Mono
  Capture channels: Mono
  Limits: 0 - 2
  Mono: 2 [100%]
Simple mixer control 'Analog In',0
  Capabilities: pvolume cvolume pswitch pswitch-joined cswitch cswitch-joined penum
  Playback channels: Front Left - Front Right
  Capture channels: Front Left - Front Right
  Limits: Playback 0 - 31 Capture 0 - 31
  Front Left: Playback 28 [90%] [6.00dB] [off] Capture 31 [100%] [12.00dB] [on]
  Front Right: Playback 28 [90%] [6.00dB] [off] Capture 31 [100%] [12.00dB] [on]


[sodo@computer ~]$ cat /proc/asound/cards
 0 [U0x46d0x821    ]: USB-Audio - USB Device 0x46d:0x821
                      USB Device 0x46d:0x821 at usb-0000:00:1d.7-6, high speed
 1 [MobilePre      ]: USB-Audio - MobilePre
                      M Audio MobilePre at usb-0000:00:1d.0-1, full speed

[sodo@computer etc]$ ls /dev/snd
by-id  by-path  controlC0  controlC2  pcmC0D0c  pcmC2D0c  pcmC2D0p  seq  timer

[sodo@computer etc]$ cat /etc/asound.conf
#
# Place your global alsa-lib configuration here...
#

@hooks [
{
func load
files [
"/etc/alsa/pulse-default.conf"
]
errors false
}
]

XWinInfo
Also, I learned about xwininfo, a cool command that shows you your desktop settings:
[sodo@computer videoConfTest]$ xwininfo
xwininfo: Window id: 0x320003b "Hak5 - Linux Screen Recording, Boxee Python Development and Qnext - YouTube - Google Chrome"

  Absolute upper-left X:  0
  Absolute upper-left Y:  53
  Relative upper-left X:  0
  Relative upper-left Y:  22
  Width: 1280
  Height: 971
  Depth: 24
  Visual: 0x21
  Visual Class: TrueColor
  Border width: 0
  Class: InputOutput
  Colormap: 0x20 (installed)
  Bit Gravity State: NorthWestGravity
  Window Gravity State: NorthWestGravity
  Backing Store State: NotUseful
  Save Under State: no
  Map State: IsViewable
  Override Redirect State: no
  Corners:  +0+53  -1280+53  -1280-0  +0-0
  -geometry 1280x971+0-0

Finally, I installed Skype and dealt with resolving a bunch of dependency problems (as usual):

VideoCamming
And a simple way to trigger mplayer to open up your videocam:
mplayer tv:// -tv driver=v4l2 device=/dev/video0

Another simple way to capture the output:
ffmpeg -f alsa -i hw:0 -f video4linux2 -s 320x240 -i /dev/video0 out.mpg

Or a more complex example, saving the video into an iPod friendly format:
ffmpeg -f alsa -i pulse -f video4linux2 -i /dev/video0 -aspect 16:9 -s 1280x720 -r 23.98 -vcodec libx264 -acodec libfaac -ab 128k -ar 48000 -ac 1 -b 2000k -bt 750k -refs 1 -deblockalpha 0 -deblockbeta 0 -subq 1 -me_range 21 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 2000k -bufsize 2M -cmp 1 -threads 8 output.mov

Finally, guvcview is a very nice program to control one's videocam:

Reference
http://www.linuxquestions.org/questions/linux-newbie-8/recording-desktop-with-ffmpeg-877580/
http://www.youtube.com/watch?v=mNz5Lrc06_s
http://forums.fedoraforum.org/showthread.php?t=256681
http://ffmpeg.org/ffprobe.html#pulse
http://alien.slackbook.org/blog/adding-an-alsa-software-pre-amp-to-fix-low-sound-levels/
http://forums.opensuse.org/english/get-technical-help-here/multimedia/471522-record-my-desktop-using-ffmpeg-post2433494.html
http://superuser.com/questions/251338/mencoder-capture-from-camera-and-microphone
http://ubuntuforums.org/showthread.php?t=1556038 (using jack)
http://wiki.linuxaudio.org/wiki/screencasttutorial (jack_capture)
https://help.ubuntu.com/community/Webcam (webcam)
http://howto-pages.org/ffmpeg/ (a nice, human readable version of the FFmpeg doc)
http://www.tldp.org/HOWTO/Alsa-sound-6.html (amixer settings)