Lifshitz: /* GStreamer */

2014-02-16T11:45:17Z

‎GStreamer

Lifshitz: /* See also */

2012-11-15T09:25:06Z

‎See also

Lifshitz at 08:49, 25 October 2012

2012-10-25T08:49:23Z

Grinberg: /* FFmpeg */

2012-10-24T09:41:50Z

‎FFmpeg

New page

== Overview ==

The CM-T3730 features DM3730 Digital Media System on Chip (SoC) with High Performance Image, Video, Audio (IVA) Accelerator Subsystem. It includes TMS320C64x+™ DSP Core (up to 800-MHz), Enhanced Direct Memory Access (EDMA) Controller and Video Hardware Accelerators.

DM3730 supports hardware acceleration of the following Video/Audio codecs:

:Video (up to 720p) : H.264 - Baseline, Medium, and High profiles, JPEG, MPEG2 - Medium profile, MPEG4 - Simple profile, VC1
:Audio : AAC - High-Efficiency and Low-Complexity profiles, G.711, MP3, WMA

CM-T3730 Linux package supports DaVinci Multimedia Application Interface (DMAI).
DMAI is a utility layer on top of the operating system and the Codec Engine (CE).
It is able to interact with the OMAP DSP or acceleration modules for encoding and decoding tasks.
It allows offloading the processing of the applications running on MPU to OMAP hardware accelerators.
The TI plugin for GStreamer multimedia framework is built on top of DMAI.
In this article we describe sample application of GStreamer to demonstrate hardware acceleration features for video and audio stream playback.

We also provide an overview of the FFmpeg application to prepare media files compatible with CM-T3730 hardware decoding capabilities.

== Boot Loader and Linux Kernel ==

To set the maximum permitted frequency of the DSP processor core, please run the following command in U-Boot before booting the kernel:

<pre>
mw 48004040 0x0009900c
</pre>

Linux kernel boot options require adjustment to enable DSP support. Following example modifies default CM-T3730 boot options by designating additional memory and OMAP frame-buffer related parameters:

<pre>
omapfb.mode=dvi:1024x768-24@60 omapdss.def_disp=dvi vram=18M omapfb.vram=0:6M,1:6M,2:6M mem=55M@0x80000000 mem=128M@0x88000000
</pre>

Please refer to [[CM-T3730: Linux: Angstrom#Digital_Video_SDK|CM-T3730: Linux: Angstrom]] article for Linux DMAI support on CM-T3730.

All provided playback examples were tested with 1024x768 DVI Monitor.
If you want to test the playback of 720p video, please change kernel command line options accordingly, for example by setting display resolution to 1280x1024:

<pre>
omapfb.mode=dvi:1280x1024-24@60
</pre>

== GStreamer ==

GStreamer is a pipeline-based multimedia framework that allows creating a wide variety of media-handling software components.
This includes simple audio playback, audio and video playback, recording, streaming, and editing.
The pipeline design allows easy creation of any type of streaming multimedia application.
GStreamer capabilities can be extended through plugins.
For instance, TI GStreamer plugin uses DMAI to make access to the underlying hardware and DSP to perform tasks, such as video encoding/decoding, using hardware acceleration.
This article demonstrates media manipulations by using of hardware acceleration.
GStreamer package and required plugins are included in the CM-T3730 Angstrom Linux root filesystem image.

=== Validation ===

[[Image:Gstreamer-videotestsrc.jpeg|thumb|right|200px|GStreamer test video stream output]]

To validate the installation of the GStreamer components, launch {{cmd|gst-inspect}} GStreamer utility.

Testing the basic streaming to OMAP frame buffer:
<pre>
gst-launch videotestsrc ! omapdmaifbsink
</pre>

GStreamer pipeline parameters in details:
{|class="wikitable" cellpadding="3" border="1" style="border: 1px solid rgb(85, 85, 85); border-collapse: collapse;"
|{{parameter|videotestsrc}} || Creates a test video stream.
|-
|{{parameter|omapdmaifbsink}} || Define the OMAP frame buffer as the output sink.
|}

<br style="clear: both" />

Test basic encoding/decoding capabilities by using synthetic video test pattern and measure MPU/DPS performance.
The same applies to any playback example in this article.

Following pipeline encodes 640x480 test video stream using H.264 codec:

<pre>
gst-launch videotestsrc ! 'video/x-raw-yuv, width=640, height=480' ! TIVidenc1 codecName=h264enc engineName=codecServer ! \
dmaiperf print-arm-load=true engine-name=codecServer ! filesink location=testsrc_sample.264
</pre>
Press '''Ctrl'''+'''C''' to stop streaming.<br>
The following pipeline plays the recorded video:

<pre>
gst-launch filesrc location=testsrc_sample.264 ! typefind ! TIViddec2 ! \
dmaiperf print-arm-load=true engine-name=codecServer ! omapdmaifbsink
</pre>

GStreamer pipeline parameters in detail:
{|class="wikitable" cellpadding="3" border="1" style="border: 1px solid rgb(85, 85, 85); border-collapse: collapse;"
|{{parameter|1=video/x-raw-yuv, width=640, height=480}} || Link the source to a ''capability'', which is defined by a mime type and a few optional properties. The properties are: video and resolution.
|-
|{{parameter|typefind}} || Find the media type of a stream
|-
|{{parameter|TIVidenc1}} || Encode video stream.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=codecName=h264enc}}
|}
|| Specify H.264 codec. Another possible values are: {{parameter|mpeg2enc}}, {{parameter|mpeg4enc}}.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=engineName=codecServer}}
|}
|| Specify codec server.
|-
|{{parameter|TIViddec2}} || Decode video stream.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=codecName=h264dec}}
|}
|| Specify H.264 decoder. Another possible values are: {{parameter|mpeg2dec}}, {{parameter|mpeg4dec}}. If the upstream element (i.e. demuxer or typefind element) is able to negotiate with the decoder element and expose its capabilities, the {{parameter|engineName}} and {{parameter|codecName}} properties will be auto-detected based on the mime type requested.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=engineName=codecServer}}
|}
|| Specify codec server.
|-
|{{parameter|dmaiperf}} || Capture pipeline performance data.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=print-arm-load=true}}
|}
|| Print the ARM CPU load information.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=engine-name=codecServer}}
|}
|| Specify codec server.
|-
|{{parameter|filesrc}} || Read data from a file.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=location=testsrc_sample.264}}
|}
|| Specify input file name.
|-
|{{parameter|filesink}} || Write incoming data to a file.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=location=testsrc_sample.264}}
|}
|| Specify output file name.
|}

To output a video in a separate window in X windows environment, the {{parameter|DISPLAY}} environment variable must be set:

<pre>
export DISPLAY=:0
</pre>

=== Playback ===
[[Image:Dsp_playback_3730.jpeg|thumb|right|200px|720x480 H.264/AAC movie playback.]]

This section provides several examples of GStreamer video playback pipelines.
More TI DMAI GStreamer Plugin applications and use cases can be found in [[CM-T3730: Linux: Video Playback#See also|See also]] section.
Encoding and network streaming of live camera video can be found in the [[CM-T3730: Linux: Camera|Camera]] article.

* Auto-playback of all supported video / audio formats.
<pre>
gst-launch playbin2 uri=file:///home/root/test.avi video-sink=omapdmaifbsink
</pre>

* Playback of plain (not muxed into container) MPEG-2/MPEG-4/H.264 video file.
<pre>
gst-launch filesrc location=test.bin ! typefind ! TIViddec2 ! omapdmaifbsink
</pre>

* Playback of plain MPEG-4 video file
<pre>
gst-launch filesrc location=test.bin ! TIViddec2 codecName=mpeg4dec engineName=codecServer ! omapdmaifbsink
</pre>

* Playback of all supported video muxed into AVI container
<pre>
gst-launch filesrc location=test.avi ! avidemux name=demux demux. ! TIViddec2 ! omapdmaifbsink
</pre>

* Playback of all supported video / audio muxed into MOV container
<pre>
gst-launch filesrc location=test.mov ! qtdemux name=demux demux. ! queue ! TIViddec2 ! omapdmaifbsink demux. ! \
queue max-size-time=0 ! TIAuddec1 ! alsasink
</pre>

GStreamer pipeline parameters in detail:
{|class="wikitable" cellpadding="3" border="1" style="border: 1px solid rgb(85, 85, 85); border-collapse: collapse;"
|{{parameter|playbin2}} || Auto-plug and play media from an uri. {{Parameter|playbin2}} builds a playback pipeline using automatic file type recognition and automatic selection of the right audio/video/subtitle demuxers/decoders. The resulting pipeline is a bit more complicated than explicit custom pipelines and can increase the MPU load. This element is very simple to use and supports HW accelerated playback.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=uri=file:///home/root/test.avi}}
|}
|| Specify a media file for playback.
|-
|{{parameter|queue}} || Provide a buffer for the next element in the pipeline. It also required to force the use of separate threads for audio and video decoding.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=max-size-time=0}}
|}
|| Max. amount of data in the queue in nano seconds.
|-
|{{parameter|qtdemux}} || Demux a .mov file into audio and video streams. This element can be replaced by {{Parameter|avidemux}}, {{Parameter|oggdemux}}, {{Parameter|mpegtsdemux}} etc. depending on the input file container format.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|1=name=demux}}
|}
|| Specify demuxer name. Respective pads of {{Parameter|demux.}} are then linked to video and audio decoders.
|-
|{{parameter|TIAuddec1}} || Decode audio. Due to the licensing restriction DVSDK does not come with mp3 support integrated in the default codec server.
|-
|-
|{{parameter|alsasink}} || Output to a sound card via ALSA subsystem.
|}

{{Note | If a video or an audio decoder element doesn't support input media stream type, GStreamer will output a failure report.}}

== FFmpeg ==

FFmpeg is a very fast video and audio converter application. It can be installed on the desktop computer to prepare media files compatible with CM-3730 hardware accelerated playback capabilities.
This article describes FFmpeg installation and usage for Ubuntu 11.10 Linux distribution.

=== Installation ===

This section describes the compilation and installation of the FFmpeg from the packages upstream repositories.
Installation from sources allows to benefit from bug-fixes, enhancements and most recent FFmpeg version.
For example, Ubutnu repository FFmpeg package application is precompiled with restriction to use of proprietary or commercial encoders like MP3, AAC, MPEG4 etc.

* Remove previously installed applications and libraries
<pre>
sudo apt-get remove ffmpeg x264 libx264-dev libva-dev
</pre>

* Install dependancies
<pre>
sudo apt-get -y install build-essential checkinstall git libfaac-dev libgpac-dev libjack-jackd2-dev libmp3lame-dev \
libopencore-amrnb-dev libopencore-amrwb-dev librtmp-dev libsdl1.2-dev libtheora-dev libva-dev \
libvdpau-dev libvorbis-dev libx11-dev libxfixes-dev texi2html yasm zlib1g-dev
</pre>

* x264 library
<pre>
git clone git://git.videolan.org/x264
cd x264
./configure --prefix=/usr --enable-static && make
sudo checkinstall --pkgname=x264 --pkgversion="3:$(./version.sh | awk -F'[" ]' '/POINT/{print $4"+git"$5}')" \
--backup=no --deldoc=yes --fstrans=no --default
</pre>

* FFmpeg
<pre>
git clone --depth 1 git://source.ffmpeg.org/ffmpeg
cd ffmpeg
./configure --prefix=/usr --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb \
--enable-libopencore-amrwb --enable-librtmp --enable-libtheora --enable-libvorbis \
--enable-libx264 --enable-nonfree --enable-version3 --enable-x11grab && make
sudo checkinstall --pkgname=ffmpeg --pkgversion="5:$(date +%Y%m%d%H%M)-git" --backup=no --deldoc=yes \
--fstrans=no --default
</pre>
<pre>hash x264 ffmpeg ffplay ffprobe</pre>

=== Usage ===

* Convert to H.264 (High Profile) 720x480 video
<pre>
ffmpeg -i sample_1080p.mov -vcodec libx264 -vprofile high -preset slow -r 30 -s 720:480 -threads 0 -an -f h264 test.bin
</pre>

* Convert to MPEG4 (Simple Profile) 720x480 video
<pre>
ffmpeg -i sample_1080p.mov -vcodec mpeg4 -b:v 1024k -r 30 -s 720:480 -threads 0 -an -f m4v test.bin
</pre>

* Convert to H.264 (High Profile) 720x480 video, AAC 128Kbit/s audio muxed into MOV container
<pre>
ffmpeg -i sample_1080p.mov -vcodec libx264 -vprofile high -preset slow -s 720:480 -threads 0 \
-acodec libfaac -ab 128k test.mov
</pre>

FFmpeg command line parameters in detail:
{|class="wikitable" cellpadding="3" border="1" style="border: 1px solid rgb(85, 85, 85); border-collapse: collapse;"
|width="200px"| {{parameter|-i sample_1080p.mov}} || Specify input file name.
|-
|{{parameter|-vcodec libx264}} || Encode video to H.264 format using libx264 library.
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|-vprofile high}}
|}
|| Force the limits of the H.264 profile to {{parameter|high}}. Other valid options are {{parameter|baseline}} and {{parameter|main}} (supported by CM-T3730).
|-
|
{|cellpadding="0" cellspacing="0" width="100%"
|width="30px"| || {{parameter|-preset slow}}
|}
|| Set encoding preset for x264 - slower presets give more quality at the same bitrate, but take more time to encode. {{parameter|slow}} is a good balance between encoding time and quality. Other valid options are: {{parameter|ultrafast}}, {{parameter|superfast}}, {{parameter|veryfast}}, {{parameter|faster}}, {{parameter|fast}}, {{parameter|medium}}, {{parameter|slower}}, {{parameter|veryslow}}, {{parameter|placebo}}.
|-
|{{parameter|-vcodec mpeg4}} || Encode video to MPEG4.
|-
|{{parameter|-b:v 1024k}} || Set video bitrate to 1024 kbits/s.
|-
|{{parameter|-r 30}} || Set video framerate to 30 fps.
|-
|{{parameter|-s 720:480}} || Specify output frame size - 720x480.
|-
|{{parameter|-acodec libfaac}} || Encode audio to AAC format using libfaac library.
|-
|{{parameter|-ab 128k}} || Set audio bitrate to 128 kbits/s.
|-
|{{parameter|-an}} || Disable audio.
|-
|{{parameter|-f m4v}} || Force setting of output file format to MPEG4.
|-
|{{parameter|-threads 0}} || Use an optimal number of threads when encoding.
|-
|{{parameter|test.mov}} || Specify output file and the mov muxer (autodetected from extension). Anything found on the command line which cannot be interpreted as an option is considered to be an output filename.
|}

== See also ==

* [http://processors.wiki.ti.com/index.php/GstTIPlugin_Elements/ DMAI GStreamer Plug-In Elements]
* [http://processors.wiki.ti.com/index.php/DMAI_GStreamer_Plug-In DMAI GStreamer Plug-In]
* [http://gstreamer.freedesktop.org/ GStreamer official resource]
* [http://www.ffmpeg.org/index.html FFmpeg official resource]

* [[CM-T3730: Linux: Angstrom#Digital_Video_SDK | CM-T3730: Linux: Angstrom - Digital Video SDK]]
* [[CM-T3730: Linux: Camera]]
* [[CM-T3730: Linux: Getting started]]
* [[CM-T3730: Linux: Angstrom]]
* [[CM-T3730: Linux: Kernel]]

[[Category:Linux]]
[[Category:CM-T3730]]

@@ Line 31: / Line 31: @@
 </pre>
-Please refer to [[CM-T3730: Linux: Angstrom#Digital_Video_SDK|CM-T3730: Linux: Angstrom]] article for Linux DMAI support on CM-T3730.
+Please refer to [[CM-T3730: Linux: Angstrom#Digital_Video_SDK|CM-T3730: Linux: Angstrom]] and [[CM-T3730: Linux: Debian#Digital_Video_SDK|CM-T3730: Linux: Debian]] articles for Linux DMAI support on CM-T3730.
 All provided playback examples were tested with 1024x768 DVI Monitor.
@@ Line 48: / Line 48: @@
 For instance, TI GStreamer plugin uses DMAI to make access to the underlying hardware and DSP to perform tasks, such as video encoding/decoding, using hardware acceleration.
 This article demonstrates media manipulations by using of hardware acceleration.
-GStreamer package and required plugins are included in the CM-T3730 Angstrom Linux root filesystem image.
+GStreamer package and required plugins are included in the CM-T3730 Angstrom and Debian Linux root filesystem image.
 === Validation ===
@@ Line 334: / Line 334: @@
 * [[CM-T3730: Linux: Angstrom#Digital_Video_SDK | CM-T3730: Linux: Angstrom - Digital Video SDK]]
 * [[CM-T3730: Linux: Camera]]
 * [[CM-T3730: Linux: Getting started]]
 * [[CM-T3730: Linux: Angstrom]]
 * [[CM-T3730: Linux: Kernel]]

@@ Line 328: / Line 328: @@
 == See also ==
-* [http://processors.wiki.ti.com/index.php/GstTIPlugin_Elements/ DMAI GStreamer Plug-In Elements]
+* [http://processors.wiki.ti.com/index.php/GstTIPlugin_Elements DMAI GStreamer Plug-In Elements]
 * [http://processors.wiki.ti.com/index.php/DMAI_GStreamer_Plug-In DMAI GStreamer Plug-In]
 * [http://gstreamer.freedesktop.org/ GStreamer official resource]

@@ Line 159: / Line 159: @@
 === Playback ===
-[[Image:Dsp_playback_3730.jpeg|thumb|right|200px|720x480 H.264/AAC movie playback.]]
+[[Image:Dsp_playback_3730_scaled.jpeg|thumb|right|200px|720x480 H.264/AAC movie playback.]]
 This section provides several examples of GStreamer video playback pipelines.

CM-T3730: Linux: Video Playback - Revision history

Lifshitz: /* GStreamer */

Lifshitz: /* See also */

Lifshitz at 08:49, 25 October 2012

Grinberg: /* FFmpeg */