Sunday, December 08, 2013

lossless encoding from H264 to MPEGTS

For editing purposes, I needed a good way to losslessly encode video from my Canon 5D Mark II's H264 source to a format that allows me to concatenate multiple video files into one large one.  Only a few video formats have the ability to concatenate multiple files together: mainly mpg and mpegts.  FFmpeg is always getting better (thank you, guys!), and has a nice formula for doing this here:
https://trac.ffmpeg.org/wiki/How%20to%20concatenate%20(join,%20merge)%20media%20files

The formula relies on a specification from the H264 standard known as AnnexB.  Here are a couple articles about Annex B, though both are above my head:
http://aviadr1.blogspot.com/2010/05/h264-extradata-partially-explained-for.html
http://wiki.multimedia.cx/index.php?title=H.264

The H.264 spec itself:
http://www.itu.int/rec/T-REC-H.264-201304-I

My goal was to losslessly extract the video and audio from the individual videos I shot using the Canon and concatenate them all into one long-length video.

Here's what the Canon video file container looked like to FFprobe:
ffprobe $VIDEO 2>&1 | grep Stream
    Stream #0:0(eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuvj420p(pc, smpte170m), 1920x1080, 45694 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 60k tbc (default)
    Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)

Note the video stream is H264 and the audio stream is waveform PCM.

Extract the Video
I did this with an FFmpeg command line:
ffmpeg -i $VIDEO -c copy -bsf:v h264_mp4toannexb -f mpegts $NEWVIDEO

Note that you'll need the latest and greatest FFmpeg.  Using an older version of FFmpeg from RPM Fusion's repo )FFmpeg 1.0.8), I got a bunch of these errors when I tried to concatenate individual MPEGTS files.
[aac @ 0x6727e0] channel element 1.10 is not allocated
[aac @ 0x6727e0] More than one AAC RDB per ADTS frame is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x6727e0] channel element 3.15 is not allocated
[aac @ 0x6727e0] Error decoding AAC frame header.
[aac @ 0x6727e0] Reserved bit set.
[aac @ 0x6727e0] channel element 1.8 is not allocated
[aac @ 0x6727e0] channel element 2.0 is not allocated
[aac @ 0x6727e0] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x6727e0] Reserved bit set.
    Last message repeated 1 times
[aac @ 0x6727e0] Number of bands (4) exceeds limit (2).
[aac @ 0x6727e0] SSR not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[mpegts @ 0x676640] decoding for stream 1 failed
[mpegts @ 0x676640] Could not find codec parameters for stream 1 (Audio: aac ([6][0][0][0] / 0x0006), 8 channels (FL+FR+FC+LFE+BL+BR+FLC+FRC), s16, 548 kb/s): unspecified sample rate
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mpegts, from 'concat:test.ts|test2.ts':
  Duration: 00:03:14.17, start: 1.400000, bitrate: 131495 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuvj420p, 1920x1088, 29.97 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101](eng): Audio: aac ([6][0][0][0] / 0x0006), 8 channels (FL+FR+FC+LFE+BL+BR+FLC+FRC), s16, 548 kb/s
[mpegts @ 0x672c20] sample rate not set

Once I downloaded the latest FFmpeg from FFmpeg's git repo

the errors disappeared.  Hooray for updated code!

Extract the Audio
My second task was to yank the PCM audio track from the Canon and save that off to a file for concatenation later on.  I did so using this command to extract only the audio stream (-c:a) from the source video file the Canon outputs:
ffmpeg -i $VIDEO -c:a copy $NEWAUDIO

Concatenate the Individual Video Files
Next, I combined the extracted video streams that were saved to MPEGTS format into one large file.  That command looked like this:
ffmpeg -i "concat:file1.ts|file2.ts|file3.ts" -c copy $FINAL.ts

Concatenate the Individual Audio Files
I did the same for the audio:
ffmpeg -i "concat:file1.wav|file2.wav|file3.wav" -c copy $FINAL.wav

Edit to Taste
Knowing my goal was to have as close as lossless video and audio, I did some audio sweetening in Audacity and reencoded the $FINAL.wav as an MP2 audio file.  This is lossy, but
1) the MPEGTS container prefers MPEG audio codecs.
2) FFmpeg doesn't yet have an encoder for MPEG-4 ALS lossless just yet.

Combine the Long-length Video and Audio Streams
Lastly, I combined both lossless video and lossy audio streams using this command into a:
ffmpeg -i $FINAL.mp2 -i $FINAL.ts -c:a copy -c:v copy -map 0:0 -map 0:1 $COMBINED.ts

The -map feature in FFmpeg is the best thing since sliced bread:
http://ffmpeg.org/ffmpeg.html#Advanced-options

You can take an audio or video stream from one file and combine it with an audio or video stream from another file to save into a new file.  This assumes the container format supports those codecs.  Sweet!

Sync Issue Averted
I noticed that if I combined the individual audio and video streams into files that were greater than an hour and a half long, FFmpeg would introduce audio-visual sync issues towards the end of that long duration video.  My solution was to combine the audio and video streams into smaller transport stream files first and then combine those smaller, complete files into one big file.  This way, I had no a/v sync issue.

Anyway, that's my latest adventure into the world of lossless video editing using my Canon and FFmpeg.

Enjoy.
CM

Saturday, April 20, 2013

a simple way to render H264 content directly from Cinelerra

Use presets!
The FFmpeg encoding guide has excellent information on using presets:
http://ffmpeg.org/trac/ffmpeg/wiki/x264EncodingGuide

At the time of this writing, by running:
ffmpeg -i input -c:v libx264 -preset -tune dummy.mp4

I see that the available presets are:
ultrafast
superfast
veryfast
faster
fast
medium
slow
slower
veryslow
placebo

So really, all you need to do is translate that simple one liner into the Cinelerra YUV4MPEG Stream:
 ffmpeg -y -i - -threads 12 -c:v libx264 -preset medium %

Like so:





































WahLah!  Easy-peasy conversion to H264.
CM

Reference:
FFmpeg x264 Encoding Guide

Monday, January 14, 2013

proper dmesg output for a high-speed USB device

It's nice when things work the way they are supposed to.  For instance, it's nice to see dmesg output that tells you that you're connected USB device is not connected at full speed.  In the below dmesg output, I see that notification when I connect my 5D Mark II, a high-speed USB device, to a low speed USB port:

[14381.652509] usb 4-2.3: new full-speed USB device number 4 using uhci_hcd
[14381.755506] usb 4-2.3: not running at top speed; connect to a high speed hub
[14381.778506] usb 4-2.3: New USB device found, idVendor=04a9, idProduct=3199
[14381.778510] usb 4-2.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[14381.778513] usb 4-2.3: Product: Canon Digital Camera
[14381.778516] usb 4-2.3: Manufacturer: Canon Inc.

So, I disconnect the camera:
[14436.920306] usb 4-2.3: USB disconnect, device number 4

And then reconnect it to a USB port that I know is high speed:
[14487.803026] usb 1-6: new high-speed USB device number 7 using ehci_hcd
[14487.922212] usb 1-6: New USB device found, idVendor=04a9, idProduct=3199
[14487.922217] usb 1-6: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[14487.922220] usb 1-6: Product: Canon Digital Camera
[14487.922223] usb 1-6: Manufacturer: Canon Inc

The second dmesg output does not indicate that it has negotiated at full speed, but then again, it does not give me the "not running" error.  So in this case, we assume that the device is connected at the highest speed.

Cool.  I can now move on to the more important stuff..downloading my media in Shotwell at the highest speed possible!

-da mule-

Saturday, December 29, 2012

convert video into an animated gif

Why would someone want to do that?  One use would be to use the animated gif as a profile pic on a social media site or an avatar for a user forum..those are two cases I can think of.

I wanted to convert this short video into an animated gif loop.  The below timelapse was acheived by using the FPS override function of the MagicLantern firmware upgrade for the Canon 5D:


In order to convert the movie, I did two things:
1) in Cinelerra, export a JPG sequence
2) use ImageMagick "convert" to convert to an animated gif

Export a JPG sequence
1) In Cinelerra, you can export your movie as a JPG sequence (among other choices like an EXR/PNG/TGA or TIFF image sequence):

Depending on the number of frames you're exporting, you'll get a whole bunch of files saved to your hard drive.  Since my project was set to 24fps, I had about 120 files saved to my filesystem.  The file names start and end with the number of the frames you selected to export (shown in this screen cap of the Cinelerra timeline):

Here's how the sequence of JPEGs looks in the filesystem:
-rw-rw-r--. 1 sodo sodo   1302637 Dec 28 19:50 seq000373.jpg
-rw-rw-r--. 1 sodo sodo   1304670 Dec 28 19:50 seq000374.jpg
...
-rw-rw-r--. 1 sodo sodo   1296049 Dec 28 19:50 seq000490.jpg
-rw-rw-r--. 1 sodo sodo   1297281 Dec 28 19:50 seq000491.jpg
-rw-rw-r--. 1 sodo sodo   1298527 Dec 28 19:50 seq000492.jpg


Once you've export all the single images from Cinelerra, use ImageMagick's "convert" program to assemble the individual images into an animated gif.  Since I wanted to resize the 1080P resolution images (1920x1080) into 1/8th the size of the original, I used 240x135 as the resize value.  You can see I've used other specifiers as well:
-delay 0 (no delay between frames)
-loop 0 (infinitely loop the gif)
seq*.jpg (convert is smart enough to assemble the jpgs in ordered sequence)
the other specifiers are essentially "high quality" output settings

[sodo@computer 2012_12_28]$ convert -resize 240x135 -colorspace RGB +sigmoidal-contrast 11.6933 -sigmoidal-contrast 11.6933 -colorspace sRGB -define filter:filter=Sinc -define filter:window=Jinc -define filter:lobes=3 -delay 0 -loop 0 seq*.jpg anim2.gif

The result is a thumbnail of the gif:

Neato.
da mule

Saturday, March 03, 2012

joining, concatenating video files

I was trying to combine four MP4's and had a bit of a learning experience, so I thought I'd share.

Here are the four files I was trying to combine, a total of roughly 1.5GB:
[sodo@computer tmp]$ ll part?.mp4
-rw-------. 1 sodo sodo 282308395 Feb 29 19:19 part1.mp4
-rw-------. 1 sodo sodo 547860894 Feb 29 19:39 part2.mp4
-rw-------. 1 sodo sodo 429237647 Mar  3 13:54 part3.mp4
-rw-------. 1 sodo sodo 428161483 Feb 29 20:12 part4.mp4

The Few, The Proud
After reading some FFmpeg doc (linked in the references), I found out that MP4's can't be concatenated in their native form.  There are only a few video formats that CAN be concatenated in their native form: MPEG-1, MPEG-2 PS, DV.  After some experimentation and viewing the results, I decided to use mpeg2video as my intermediate format with this command:
ffmpeg -threads 8 -i part1.mp4 -sameq -vcodec mpeg2video -acodec mp2 -ac 2 -ar 44100 -ab 256k 1.mpg

Converting en masse
Since I had four of these files to convert, it made sense for me to use some quick shell control flow to get the job done in one shot (input/output streams in bold below):
for i in 1 2 3 4;do ffmpeg -threads 8 -i part$i.mp4 -sameq -vcodec mpeg2video -acodec mp2 -ac 2 -ar 44100 -ab 256k $i.mpg;done

When running, the output looks like this:
[sodo@computer tmp]$ for i in 1 2 3 4;do ffmpeg -threads 8 -i part$i.mp4 -sameq -vcodec mpeg2video -acodec mp2 -ac 2 -ar 44100 -ab 256k $i.mpg;done
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'part1.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2012-02-18 19:44:00
  Duration: 00:12:08.29, start: 0.000000, bitrate: 3101 kb/s
    Stream #0.0(und): Video: h264 (High), yuv420p, 1280x720, 2946 kb/s, 29.97 fps, 29.97 tbr, 60k tbn, 59.94 tbc
    Metadata:
      creation_time   : 1970-01-01 00:00:00
    Stream #0.1(und): Audio: aac, 44100 Hz, stereo, s16, 151 kb/s
    Metadata:
      creation_time   : 2012-02-18 19:44:26
[buffer @ 0x2329fe0] w:1280 h:720 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param:
[mpeg @ 0x2322280] VBV buffer size not set, muxing may fail
Output #0, mpeg, to '1.mpg':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2012-02-18 19:44:00
    encoder         : Lavf52.111.0
    Stream #0.0(und): Video: mpeg2video, yuv420p, 1280x720, q=2-31, 200 kb/s, 90k tbn, 29.97 tbc
    Metadata:
      creation_time   : 1970-01-01 00:00:00
    Stream #0.1(und): Audio: mp2, 44100 Hz, stereo, s16, 256 kb/s
    Metadata:
      creation_time   : 2012-02-18 19:44:26
Stream mapping:
  Stream #0.0 -> #0.0
  Stream #0.1 -> #0.1

frame=21827 fps=117 q=0.0 Lsize= 1385478kB time=00:12:08.26 bitrate=15584.8kbits/s  
video:1357365kB audio:22759kB global headers:0kB muxing overhead 0.387953%


The intermediate blues
The file size of the intermediates ballooned, expectedly:
[sodo@computer tmp]$ ll ?.mpg
-rw-rw-r--. 1 sodo sodo 1418729472 Mar  3 12:59 1.mpg
-rw-rw-r--. 1 sodo sodo 2290866176 Mar  3 13:05 2.mpg
-rw-rw-r--. 1 sodo sodo 2143287296 Mar  3 13:06 3.mpg
-rw-rw-r--. 1 sodo sodo 1918547968 Mar  3 13:11 4.mpg

Simple filesystem concatenation
After I converted the base files into files of a type that could be concatenated, I used this command to concatenate the files:
cat 1.mpg 2.mpg 3.mpg 4.mpg > all.mpg


The resulting file is pretty large ~6GB:
[sodo@computer tmp]$ ll all.mpg 
-rw-rw-r--. 1 sodo sodo 6165041152 Mar  3 13:16 all.mpg

This would be a quite serviceable intermediate file, but upon reflection, I thought that I could probably do the concatenation and conversion to a final format in one step, rather than two.  FFmpeg to the rescue!

FFmpeg's "concat" feature
Because I like my iDevices, I want the file format to be an MP4 container using H264/AAC as my video and audio formats.  So instead of doing a two-step conversion:
1) concatenate the files in the filesystem
2) transcode the video to a file format

I decided to combine both of those steps into one by using the "concat" feature of ffmpeg.  The command looks like this:
ffmpeg -i concat:"/tmp/1.mpg|/tmp/2.mpg|/tmp/3.mpg|/tmp/4.mpg" -acodec libfaac -aq 100 -ac 2 -vcodec libx264 -vpre slow -crf 24 output.mp4

You can change the output specifiers to taste.  Also note that you'll need a presets file defined if you are going to use the "-vpre slow" specifier.

There was no way to tell from FFmpeg's output that the concatenation command was working except for this small line:
Input #0, mpeg, from 'concat:/tmp/1.mpg|/tmp/2.mpg|/tmp/3.mpg|/tmp/4.mpg'
as well as the fact that the "frame=" counter at the bottom of the FFmpeg output incremented beyond the length of the first video.  In this case, 21827.

Verify the total number of frames
Redirecting standard error to standard output
As a sanity check, I like to verify the total number of frames in the output.  I can do this by capturing the text information that FFmpeg outputs when FFmpeg runs.  That text information is not "standard output" in the Unix sense.  The text from an FFmpeg command actually outputs "standard error".  So instead of trying to grep with a command like this:
ffmpeg -i 1.mpg -an -vcodec copy -f mpeg2video -y NUL  | grep 'frame'

You actually need a command like this:
ffmpeg -i 1.mpg -an -vcodec copy -f mpeg2video -y NUL 2>&1 | grep 'frame'

Or save the output to a file and then grep:
ffmpeg -i 1.mpg -an -vcodec copy -f mpeg2video -y NUL 2>&1 | tee test.txt ; grep 'frame' test.txt

The 2>&1 redirects (the ">") standard error (the "2") to standard output (the "1").  For some reason, we need an ampersand in there for the command to work correctly.

Unfortunately, I encountered a nasty little problem when I tried to read the output from that file, explained here:
http://www.techanswerguy.com/2012/03/redirecting-ffmpeg-output-performing.html

If you don't want to worry about the details, here is the solution I used to grab the number of frames in the output:
ffmpeg -i 1.mpg -an -vcodec copy -f mpeg2video -y NUL 2>&1 | awk -F"\015" 'NF > 1 {lf=NF-1;print $lf}' | awk '{print $1}' | awk -F= '{print $2}'

Until then..there you have it!  Appending multiple video files together using FFmpeg's concat feature. Sweet!
*da mule*

References
http://www.ffmpeg.org/faq.html#toc-How-can-I-join-video-files_003f
http://ffmpeg.org/faq.html

get detailed information about a video file

Here are a few ways to get detailed information about a video file using mplayer, ffmpeg or mencoder.  Pardon the format weirdness..blogger tries to overthink things and ends up screwing things up.
- mplayer
- ffmpeg
- mencoder
- ffprobe

1) mplayer: simple inspection
[sodo@computer tmp]$ mplayer -vo null -benchmark part1.mp4 
MPlayer SVN-r33996-4.6.1 (C) 2000-2011 MPlayer Team
mplayer: could not connect to socket
mplayer: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

# PLAYBACK STARTS
Playing part1.mp4.
# CONTAINER INSPECTION
libavformat file format detected.
# MPLAYER USES libavformat PLAYBACK for this VIDEO STREAM
[lavf] stream 0: video (h264), -vid 0
[lavf] stream 1: audio (aac), -aid 0, -alang und
VIDEO:  [H264]  1280x720  24bpp  29.970 fps  2946.5 kbps (359.7 kbyte/s)
Clip info:
 major_brand: mp42
 minor_version: 0
 compatible_brands: isommp42
 creation_time: 2012-02-18 19:44:00
# CHECK FOR SUBTITLES
Load subtitles in ./
# DECODE VIDEO USING FFMPEG
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
# DECODE AUDIO USING FFMPEG

==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
AUDIO: 44100 Hz, 2 ch, s16le, 152.0 kbit/10.77% (ratio: 18997->176400)
Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
==========================================================================
# USE DISCOVERED AUDIO DEVICE
AO: [pulse] 44100Hz 2ch s16le (2 bytes per sample)
Starting playback...
Unsupported PixelFormat 61
Unsupported PixelFormat 53
Movie-Aspect is undefined - no prescaling applied.
VO: [null] 1280x720 => 1280x720 Planar YV12 
A:   1.3 V:   1.3 A-V:  0.001 ct:  0.033   0/  0 23%  0%  0.4% 0 0                                                                                                                
Exiting... (Quit)

2) mplayer: detailed information
[sodo@computer tmp]$ mplayer -v part1.mp4 -frames 0
MPlayer SVN-r33996-4.6.1 (C) 2000-2011 MPlayer Team
# YOUR COMPUTER STATS
CPU vendor name: GenuineIntel  max cpuid level: 10
CPU: Intel(R) Xeon(R) CPU           E5310  @ 1.60GHz (Family: 6, Model: 15, Stepping: 7)
extended cpuid-level: 8
extended cache-info: 268468288
Detected cache-line size is 64 bytes
CPUflags:  MMX: 1 MMX2: 1 3DNow: 0 3DNowExt: 0 SSE: 1 SSE2: 1 SSSE3: 1
# MPLAYER BUILD STATS
Compiled with runtime CPU detection.
get_path('codecs.conf') -> '/home/sodo/.mplayer/codecs.conf'
Reading optional codecs config file /home/sodo/.mplayer/codecs.conf: No such file or directory
Reading optional codecs config file /etc/mplayer/codecs.conf: No such file or directory
Using built-in default codecs.conf.
init_freetype
Using MMX (with tiny bit MMX2) Optimized OnScreenDisplay
get_path('fonts') -> '/home/sodo/.mplayer/fonts'
Configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/mplayer --mandir=/usr/share/man --confdir=/etc/mplayer --libdir=/usr/lib64 --codecsdir=/usr/lib64/codecs --extra-cflags=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic --language=all --enable-joystick --enable-lirc --enable-menu --enable-radio --enable-radio-capture --enable-runtime-cpudetection --enable-unrarexec --disable-dvdread-internal --disable-libdvdcss-internal --disable-nemesi --disable-smb --disable-ffmpeg_a --disable-faac --disable-mad --disable-libmpeg2-internal --disable-tremor-internal --disable-bitmap-font --disable-dga1 --disable-dga2 --disable-directfb --disable-svga --disable-termcap --enable-xvmc --with-xvmclib=XvMCW --disable-arts --disable-esd --disable-jack --disable-openal
CommandLine: '-v' 'part1.mp4' '-nosound' '-frames' '0'
Using nanosleep() timing
# MPLAYER CONFIG PATH SEARCH
get_path('input.conf') -> '/home/sodo/.mplayer/input.conf'
Reading optional input config file /home/sodo/.mplayer/input.conf: No such file or directory
Parsing input config file /etc/mplayer/input.conf
Input config file /etc/mplayer/input.conf parsed: 92 binds
Setting up LIRC support...
mplayer: could not connect to socket
mplayer: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.
get_path('part1.mp4.conf') -> '/home/sodo/.mplayer/part1.mp4.conf'

# PLAYBACK STARTS
Playing part1.mp4.
get_path('sub/') -> '/home/sodo/.mplayer/sub/'
[file] File size is 282308395 bytes
# META INFORMATION
STREAM: [file] part1.mp4
STREAM: Description: File
STREAM: Author: Albeu
STREAM: Comment: based on the code from ??? (probably Arpi)
# CONTAINER INSPECTION
LAVF_check: QuickTime/MPEG-4/Motion JPEG 2000 format
libavformat file format detected.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x35a00e4c00]ISO: File Type Major Brand: mp42
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x35a00e4c00]All info found
# VIDEO STREAM INSPECTION, frame sizes/compression type
==> Found video stream: 0
======= VIDEO Format ======
  biSize 79
  biWidth 1280
  biHeight 720
  biPlanes 0
  biBitCount 24
  biCompression 875967048='H264'
  biSizeImage 2764800
Unknown extra header dump: [1] [64] [0] [1f] [ff] [e1] [0] [18] [67] [64] [0] [1f] [ac] [24] [88] [5] [0] [5b] [a1] [0] [0] [3] [3] [e9] [0] [0] [ea] [60] [f] [18] [32] [a0] [1] [0] [4] [68] [ee] [3c] [b0] 
===========================
[lavf] stream 0: video (h264), -vid 0
# AUDIO STREAM INSPECTION
==> Found audio stream: 1
======= WAVE Format =======
Format Tag: 20557 (0x504D)
Channels: 2
Samplerate: 44100
avg byte/sec: 18997
Block align: 1
bits/sample: 16
cbSize: 2
Unknown extra header dump: [12] [10] 
==========================================================================
# MPLAYER USES libavformat PLAYBACK for this VIDEO STREAM
[lavf] stream 1: audio (aac), -aid 0, -alang und
LAVF: 1 audio and 1 video streams found
LAVF: build 3436288
VIDEO:  [H264]  1280x720  24bpp  29.970 fps  2946.5 kbps (359.7 kbyte/s)
[V] filefmt:44  fourcc:0x34363248  size:1280x720  fps:29.970  ftime:=0.0334
Clip info:
 major_brand: mp42
 minor_version: 0
 compatible_brands: isommp42
 creation_time: 2012-02-18 19:44:00
# CHECK FOR SUBTITLES
Load subtitles in ./
get_path('sub/') -> '/home/sodo/.mplayer/sub/'
# CHECK X11 STATS
X11 opening display: :0
vo: X11 color mask:  FFFFFF  (R:FF0000 G:FF00 B:FF)
vo: X11 running at 2560x1024 with depth 24 and 32 bpp (":0" => local display)
[x11] Detected wm supports layers.
[x11] Using workaround for Metacity bugs.
[x11] Detected wm supports NetWM.
[x11] Detected wm supports ABOVE state.
[x11] Detected wm supports BELOW state.
[x11] Detected wm supports FULLSCREEN state.
[x11] Current fstype setting honours FULLSCREEN ABOVE BELOW X atoms
# DECODE VIDEO USING FFMPEG

==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
INFO: libavcodec init OK!
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
==========================================================================
# DECODE AUDIO USING FFMPEG
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
dec_audio: Allocating 192000 + 65536 = 257536 bytes for output buffer.
FFmpeg's libavcodec audio codec
INFO: libavcodec "aac" init OK!
AUDIO: 44100 Hz, 2 ch, s16le, 152.0 kbit/10.77% (ratio: 18997->176400)
Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
==========================================================================
# PREPARE TO PLAYBACK AUDIO
Building audio filter chain for 44100Hz/2ch/s16le -> 0Hz/0ch/??...
[libaf] Adding filter dummy 
[dummy] Was reinitialized: 44100Hz/2ch/s16le
[dummy] Was reinitialized: 44100Hz/2ch/s16le
# FIND SUITABLE AUDIO DEVICE
Trying preferred audio driver 'pulse', options '[none]'
AO: [pulse] 44100Hz 2ch s16le (2 bytes per sample)
AO: Description: PulseAudio audio output
AO: Author: Lennart Poettering
Building audio filter chain for 44100Hz/2ch/s16le -> 44100Hz/2ch/s16le...
[dummy] Was reinitialized: 44100Hz/2ch/s16le
[dummy] Was reinitialized: 44100Hz/2ch/s16le
Starting playback...
Increasing filtered audio buffer size from 0 to 46144
Unsupported PixelFormat 61
Unsupported PixelFormat 53
[ffmpeg] aspect_ratio: 0.000000
# FIND SUITABLE PLAYBACK COLORSPACE
VDec: vo config request - 1280 x 720 (preferred colorspace: Planar YV12)
Trying filter chain: vo
VDec: using Planar YV12 as output csp (no 0)
Movie-Aspect is undefined - no prescaling applied.
# VIDEO OUTPUT CONFIGURATION (using NVIDIA)
VO Config (1280x720->1280x720,flags=0,'MPlayer',0x32315659)
VO: [vdpau] 1280x720 => 1280x720 Planar YV12 
VO: Description: VDPAU with X11
VO: Author: Rajib Mahapatra and others
[vdpau] Updating CSC matrix for BT.601
*** [vo] Exporting mp_image_t, 1280x720x12bpp YUV planar, 1382400 bytes
Unicode font: 5179 glyphs.
Unicode font: 5179 glyphs.
A:   1.0 V:   1.0 A-V:  0.002 ct:  0.032   0/  0 33% 10%  0.4% 0 0                                                                                                                
Uninit audio filters...
[libaf] Removing filter dummy 
Uninit audio: ffmpeg
Uninit video: ffmpeg
vo: uninit ..

3) ffmpeg
This option is nice because it gives you a count of the frames and bitrates.
[sodo@computer tmp]$  ffmpeg -i part1.mp4 -an -vcodec copy -f avi -y NUL
# FFMPEG BUILD STATS
ffmpeg version 0.7.11-rpmfusion, Copyright (c) 2000-2011 the FFmpeg developers
  built on Feb 25 2012 08:39:28 with gcc 4.6.1 20110908 (Red Hat 4.6.1-9)
  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --extra-cflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --extra-version=rpmfusion --enable-bzlib --enable-libcelt --enable-libdc1394 --enable-libdirac --enable-libfaac --enable-nonfree --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab --enable-avfilter --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-runtime-cpudetect
  libavutil    50. 43. 0 / 50. 43. 0
  libavcodec   52.123. 0 / 52.123. 0
  libavformat  52.111. 0 / 52.111. 0
  libavdevice  52.  5. 0 / 52.  5. 0
  libavfilter   1. 80. 0 /  1. 80. 0
  libswscale    0. 14. 1 /  0. 14. 1
  libpostproc  51.  2. 0 / 51.  2. 0

# CONTAINER INSPECTION
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'part1.mp4':
# META INFORMATION: INPUT
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2012-02-18 19:44:00
  Duration: 00:12:08.29, start: 0.000000, bitrate: 3101 kb/s
# VIDEO STREAM INSPECTION: INPUT
    Stream #0.0(und): Video: h264 (High), yuv420p, 1280x720, 2946 kb/s, 29.97 fps, 29.97 tbr, 60k tbn, 59.94 tbc
    Metadata:
      creation_time   : 1970-01-01 00:00:00
# AUDIO STREAM INSPECTION: INPUT
    Stream #0.1(und): Audio: aac, 44100 Hz, stereo, s16, 151 kb/s
    Metadata:
      creation_time   : 2012-02-18 19:44:26
# META INFORMATION: OUTPUT
Output #0, avi, to 'NUL':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2012-02-18 19:44:00
    ISFT            : Lavf52.111.0
# VIDEO STREAM INSPECTION: OUTPUT
    Stream #0.0(und): Video: libx264, yuv420p, 1280x720, q=2-31, 2946 kb/s, 29.97 tbn, 29.97 tbc
    Metadata:
      creation_time   : 1970-01-01 00:00:00
Stream mapping:
  Stream #0.0 -> #0.0
Press [q] to stop, [?] for help
# SUMMARY STATS: frames, bitrates
frame=21827 fps=  0 q=-1.0 Lsize=  262481kB time=00:12:08.29 bitrate=2952.4kbits/s    
video:261953kB audio:0kB global headers:0kB muxing overhead 0.201505%

4) mencoder
Mencoder's kbit/s differs from ffmpegs..hmmm?
[sodo@computer tmp]$ mencoder -nosound -ovc frameno -vc null -o /dev/null part1.mp4 
MEncoder SVN-r33996-4.6.1 (C) 2000-2011 MPlayer Team
# STARTUP STATUS
success: format: 0  data: 0x0 - 0x10d3af2b
# META INFORMATION: INPUT
libavformat file format detected.
[lavf] stream 0: video (h264), -vid 0
[lavf] stream 1: audio (aac), -aid 0, -alang und
# VIDEO STREAM INSPECTION: INPUT
VIDEO:  [H264]  1280x720  24bpp  29.970 fps  2946.5 kbps (359.7 kbyte/s)
[V] filefmt:44  fourcc:0x34363248  size:1280x720  fps:29.970  ftime:=0.0334
Writing header...
ODML: Aspect information not (yet?) available or unspecified, not writing vprp header.
Writing header...
ODML: Aspect information not (yet?) available or unspecified, not writing vprp header.
Pos: 728.3s  21827f (100%) 7165.79fps Trem:   0min   0mb  A-V:0.000 [0:0]
Writing index...
Writing header...
ODML: Aspect information not (yet?) available or unspecified, not writing vprp header.

# BITRATE CALCULATIONS FOR MEDIA BURN
Recommended video bitrate for 650MB CD: 7480
Recommended video bitrate for 700MB CD: 8055
Recommended video bitrate for 800MB CD: 9207
Recommended video bitrate for 2 x 650MB CD: 14966
Recommended video bitrate for 2 x 700MB CD: 16118
Recommended video bitrate for 2 x 800MB CD: 18422


# SUMMARY STATS: frames, bitrates
Video stream:    0.959 kbit/s  (119 B/s)  size: 87308 bytes  728.294 secs  21827 frames


5) ffprobe

[sodo@computer tmp]$  ffprobe -loglevel error -show_streams GOPR0078.MP4 
ffprobe version 0.7.13, Copyright (c) 2007-2011 the FFmpeg developers
  built on Aug  1 2012 21:08:35 with clang 3.1 (tags/Apple/clang-318.0.58)
  configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-libmp3lame --enable-libvorbis --enable-libtheora --enable-libdirac --enable-libschroedinger --enable-libopenjpeg --enable-libvpx --enable-libspeex --disable-libopencore-amrnb --disable-libopencore-amrwb --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --arch=x86_64 --enable-yasm --enable-gpl --enable-postproc --enable-libx264 --enable-libxvid
  libavutil    50. 43. 0 / 50. 43. 0
  libavcodec   52.123. 0 / 52.123. 0
  libavformat  52.111. 0 / 52.111. 0
  libavdevice  52.  5. 0 / 52.  5. 0
  libavfilter   1. 80. 0 /  1. 80. 0
  libswscale    0. 14. 1 /  0. 14. 1
  libpostproc  51.  2. 0 / 51.  2. 0
[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
codec_type=video
codec_time_base=1001/120000
codec_tag_string=avc1
codec_tag=0x31637661
width=1280
height=720
has_b_frames=1
sample_aspect_ratio=1:1
display_aspect_ratio=16:9
pix_fmt=yuv420p
r_frame_rate=60000/1001
avg_frame_rate=60000/1001
time_base=1/90000
start_time=0.000000 
duration=489.655833 
nb_frames=29350
TAG:creation_time=2012-01-08 12:43:14
TAG:language=eng
[/STREAM]
[STREAM]
index=1
codec_name=aac
codec_long_name=Advanced Audio Coding
codec_type=audio
codec_time_base=0/1
codec_tag_string=mp4a
codec_tag=0x6134706d
sample_rate=48000.000000 
channels=2
bits_per_sample=0
r_frame_rate=0/0
avg_frame_rate=375/8
time_base=1/48000
start_time=0.000000 
duration=489.642667 
nb_frames=22952
TAG:creation_time=2012-01-08 12:43:14
TAG:language=eng
[/STREAM]



Appendix
Comparing the meta info of different output video streams
a. get frame and bitrate stats for a video, ffmpeg using -f avi
[sodo@computer tmp]$  ffmpeg -i part1.mp4 -an -vcodec copy -f avi -y NUL
# META INFORMATION: OUTPUT

Output #0, avi, to 'NUL':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2012-02-18 19:44:00
    ISFT            : Lavf52.111.0
    Stream #0.0(und): Video: libx264, yuv420p, 1280x720, q=2-31, 2946 kb/s, 29.97 tbn, 29.97 tbc
    Metadata:
      creation_time   : 1970-01-01 00:00:00
..
frame=21827 fps=  0 q=-1.0 Lsize=  262481kB time=00:12:08.29 bitrate=2952.4kbits/s    
video:261953kB audio:0kB global headers:0kB muxing overhead 0.201505%

b. get frame and bitrate stats for a video, ffmpeg, using -f mpeg2video
Output will be mostly the same as above, but I suspect the differences in Lsize and bitrate is due to my forcing the format of the output stream to mpeg2video
[sodo@computer tmp]$  ffmpeg -i part1.mp4 -an -vcodec copy -f mpeg2video -y NUL
..
# META INFORMATION: OUTPUT
Output #0, mpeg2video, to 'NUL':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2012-02-18 19:44:00
    encoder         : Lavf52.111.0
    Stream #0.0(und): Video: libx264, yuv420p, 1280x720, q=2-31, 2946 kb/s, 90k tbn, 29.97 tbc
    Metadata:
      creation_time   : 1970-01-01 00:00:00

..
frame=21827 fps=  0 q=-1.0 Lsize=  261953kB time=00:12:08.29 bitrate=2946.5kbits/s    
video:261953kB audio:0kB global headers:0kB muxing overhead 0.000000%

c. ffmpeg, using rawvideo, -f yuv4mpegpipe
This one takes some time, as I've removed the option specifying ffmpeg to copy the same video stream so that ffmpeg actually transcodes the video stream into the raw video format, yuv4mpegpipe.
[sodo@computer tmp]$  ffmpeg -threads 8 -i part1.mp4 -an -f yuv4mpegpipe -y NUL
..
# META INFORMATION: OUTPUT

Output #0, yuv4mpegpipe, to 'NUL':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2012-02-18 19:44:00
    encoder         : Lavf52.111.0
    Stream #0.0(und): Video: rawvideo, yuv420p, 1280x720, q=2-31, 200 kb/s, 90k tbn, 29.97 tbc
    Metadata:
      creation_time   : 1970-01-01 00:00:00
..
frame=21827 fps= 46 q=0.0 Lsize=29466578kB time=00:12:08.29 bitrate=331446.0kbits/s    
video:0kB audio:0kB global headers:0kB muxing overhead inf%



References
Mplayer FAQ
# convert/transcode a video to rawvideo format, but just inspect the header
ffmpeg -i part1.mp4 -pix_fmt yuv420p -f yuv4mpegpipe - | head -1

# convert/transcode a video to rawvideo format
ffmpeg -i part1.mp4 -an -f yuv4mpegpipe -y NUL

http://ffmpeg.org/faq.html
http://ffmpeg.org/ffprobe.html

Thursday, January 26, 2012

recording desktop with ffmpeg

I had this idea that I'm going to start interviewing folks on different technical topics via Skype or Google Hangout.  Normally, I would use Cinelerra to perform this task, but for some reason, the recordings in Cinelerra showed some irritating sound artifacts.  Testing in audacity showed no such audio problems.  I'll bring this up on the Cin mailing list soon.

FFmpeg to the Rescue
So with Cinelerra not working for desktop recordings, I needed an alternate way to record my desktop screen as well as the audio from the person being interviewed.  After some searching a good deal of trial and error, I found from the link in the reference section at the bottom of the page, a very nice ffmpeg command.  You can save to any format you want.  The example below shows how to save the microphone audio (hw:0,0) in PCM format.  The desktop video is brought in via the xllgrab format specifier :
ffmpeg -f alsa -i hw:0,0 -ac 2 -ar 48000 -acodec pcm_s16le -f x11grab -r 24 -s 1280x720 -i :0.0 -aspect 16:9 -vcodec libx264 -vpre lossless_ultrafast -threads 8 -y output.mov

I later tweaked the command to do a bit more, using pulse audio as input as well as saving out to an iPod compatible format:
ffmpeg -f alsa -i pulse -f x11grab -s 1280x720 -r 23.98 -i :0.0 -vcodec libx264 -acodec pcm_s16le -ab 128k -ar 48000 -ac 1 -b 2000k -bt 750k -refs 1 -deblockalpha 0 -deblockbeta 0 -subq 1 -me_range 21 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 2000k -bufsize 2M -cmp 1 -threads 8 output.mov

Errors
Not sure how yet to resolve these ffmpeg errors:
[alsa @ 0x16512a0] Estimating duration from bitrate, this may be inaccurate
Input #0, alsa, from 'hw:0,0':
  Duration: N/A, start: 17479.447401, bitrate: N/A
    Stream #0.0: Audio: pcm_s16le, 32000 Hz, 2 channels, s16, 1024 kb/s
[x11grab @ 0x1650240] device: :0.0 -> display: :0.0 x: 0 y: 0 width: 1280 height: 720
[x11grab @ 0x1650240] shared memory extension found
[x11grab @ 0x1650240] Estimating duration from bitrate, this may be inaccurate
Input #1, x11grab, from ':0.0':
  Duration: N/A, start: 1327197224.425685, bitrate: 707788 kb/s
    Stream #1.0: Video: rawvideo, bgra, 1280x720, 707788 kb/s, 24 tbr, 1000k tbn, 24 tbc
Incompatible pixel format 'bgra' for codec 'libx264', auto-selecting format 'yuv420p'

Along the way, I received an error:
"unknown field slave"

Finding a bit more about ALSA
This ended up being because I had edited my /etc/asound.conf file from the default.  This corrupted my sound card settings.  I was reacquainted with some debugging commands and places where "sound stuff" lives:
[sodo@computer etc]$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: U0x46d0x821 [USB Device 0x46d:0x821], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0
card 2: MobilePre [MobilePre], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

[sodo@computer etc]$ amixer -c 2
Simple mixer control 'Speaker',0
  Capabilities: pvolume pswitch pswitch-joined penum
  Playback channels: Front Left - Front Right
  Limits: Playback 0 - 31
  Mono:
  Front Left: Playback 31 [100%] [0.00dB] [on]
  Front Right: Playback 31 [100%] [0.00dB] [on]
Simple mixer control 'PCM',0
  Capabilities: pvolume pswitch pswitch-joined penum
  Playback channels: Front Left - Front Right
  Limits: Playback 0 - 31
  Mono:
  Front Left: Playback 25 [81%] [0.00dB] [on]
  Front Right: Playback 25 [81%] [0.00dB] [on]
Simple mixer control 'PCM',1
  Capabilities: volume volume-joined penum
  Playback channels: Mono
  Capture channels: Mono
  Limits: 0 - 2
  Mono: 2 [100%]
Simple mixer control 'Analog In',0
  Capabilities: pvolume cvolume pswitch pswitch-joined cswitch cswitch-joined penum
  Playback channels: Front Left - Front Right
  Capture channels: Front Left - Front Right
  Limits: Playback 0 - 31 Capture 0 - 31
  Front Left: Playback 28 [90%] [6.00dB] [off] Capture 31 [100%] [12.00dB] [on]
  Front Right: Playback 28 [90%] [6.00dB] [off] Capture 31 [100%] [12.00dB] [on]


[sodo@computer ~]$ cat /proc/asound/cards
 0 [U0x46d0x821    ]: USB-Audio - USB Device 0x46d:0x821
                      USB Device 0x46d:0x821 at usb-0000:00:1d.7-6, high speed
 1 [MobilePre      ]: USB-Audio - MobilePre
                      M Audio MobilePre at usb-0000:00:1d.0-1, full speed

[sodo@computer etc]$ ls /dev/snd
by-id  by-path  controlC0  controlC2  pcmC0D0c  pcmC2D0c  pcmC2D0p  seq  timer

[sodo@computer etc]$ cat /etc/asound.conf
#
# Place your global alsa-lib configuration here...
#

@hooks [
{
func load
files [
"/etc/alsa/pulse-default.conf"
]
errors false
}
]

XWinInfo
Also, I learned about xwininfo, a cool command that shows you your desktop settings:
[sodo@computer videoConfTest]$ xwininfo
xwininfo: Window id: 0x320003b "Hak5 - Linux Screen Recording, Boxee Python Development and Qnext - YouTube - Google Chrome"

  Absolute upper-left X:  0
  Absolute upper-left Y:  53
  Relative upper-left X:  0
  Relative upper-left Y:  22
  Width: 1280
  Height: 971
  Depth: 24
  Visual: 0x21
  Visual Class: TrueColor
  Border width: 0
  Class: InputOutput
  Colormap: 0x20 (installed)
  Bit Gravity State: NorthWestGravity
  Window Gravity State: NorthWestGravity
  Backing Store State: NotUseful
  Save Under State: no
  Map State: IsViewable
  Override Redirect State: no
  Corners:  +0+53  -1280+53  -1280-0  +0-0
  -geometry 1280x971+0-0

Finally, I installed Skype and dealt with resolving a bunch of dependency problems (as usual):

VideoCamming
And a simple way to trigger mplayer to open up your videocam:
mplayer tv:// -tv driver=v4l2 device=/dev/video0

Another simple way to capture the output:
ffmpeg -f alsa -i hw:0 -f video4linux2 -s 320x240 -i /dev/video0 out.mpg

Or a more complex example, saving the video into an iPod friendly format:
ffmpeg -f alsa -i pulse -f video4linux2 -i /dev/video0 -aspect 16:9 -s 1280x720 -r 23.98 -vcodec libx264 -acodec libfaac -ab 128k -ar 48000 -ac 1 -b 2000k -bt 750k -refs 1 -deblockalpha 0 -deblockbeta 0 -subq 1 -me_range 21 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 2000k -bufsize 2M -cmp 1 -threads 8 output.mov

Finally, guvcview is a very nice program to control one's videocam:

Reference
http://www.linuxquestions.org/questions/linux-newbie-8/recording-desktop-with-ffmpeg-877580/
http://www.youtube.com/watch?v=mNz5Lrc06_s
http://forums.fedoraforum.org/showthread.php?t=256681
http://ffmpeg.org/ffprobe.html#pulse
http://alien.slackbook.org/blog/adding-an-alsa-software-pre-amp-to-fix-low-sound-levels/
http://forums.opensuse.org/english/get-technical-help-here/multimedia/471522-record-my-desktop-using-ffmpeg-post2433494.html
http://superuser.com/questions/251338/mencoder-capture-from-camera-and-microphone
http://ubuntuforums.org/showthread.php?t=1556038 (using jack)
http://wiki.linuxaudio.org/wiki/screencasttutorial (jack_capture)
https://help.ubuntu.com/community/Webcam (webcam)
http://howto-pages.org/ffmpeg/ (a nice, human readable version of the FFmpeg doc)
http://www.tldp.org/HOWTO/Alsa-sound-6.html (amixer settings)