i.MX_6_Series_Ubuntu_Multimedia_User's_Guide
i.MX 6 Series Ubuntu Multimedia
User's Guide
Document Number: IMX6UBUNTUUG
Rev L3.0.35_4.1.0, 09/2013
Contents
Section number Title Page
Chapter 1
About This Book
1.1Audience (5)
1.2Organization (5)
1.3Conventions (5)
1.4References (6)
1.5Definitions, Acronyms, and Abbreviations (6)
Chapter 2
Installing and Building Plug-Ins
2.1Building Plug-Ins by Using LTIB (7)
2.1.1System Requirements for Installing LTIB (7)
2.1.2Building Plug-Ins (8)
2.2Installing/Building Plug-Ins on Ubuntu (11)
2.2.1System Requirements for Ubuntu (11)
2.2.2Installing/Building Plug-Ins (12)
Chapter 3
Testing Installation
3.1Testing Multimedia Environment Setting (17)
3.1.1Audio Output Setting (17)
3.1.2Audio Input Setting (18)
3.1.3Video Setting (19)
3.2Testing Codecs with GStreamer (19)
3.2.1gst-inspect Tool (19)
3.2.2gst-launch Tool (21)
3.2.2.1Playback with playbin2 (21)
3.2.2.2Audio Playback (21)
3.2.2.3Video only Playback (22)
3.2.2.4Audio/Video File Playback (22)
Section number Title Page
3.2.2.5Audio Encoder Record (23)
3.2.2.6VPU-Based Video Encoder Record (24)
3.2.2.7SPDIF Transmit and Receive Converter (25)
3.2.2.8Audio Post-Process (25)
3.2.2.9Dual Display (26)
3.2.2.10Transcoding (26)
3.2.3gplay Player (26)
3.2.4Totem Player (27)
3.3Testing Core Codec Libraries (27)
3.4Debug Exception in Multimedia Plug-In (27)
Chapter 4
Multi-Overlay Support
4.1How to Use mfw_isink (29)
4.1.1Using mfw_isink with gst-launch (30)
4.1.2Using mfw_isink with Totem Player (31)
Chapter 5
Streaming Support
5.1HTTP Support (33)
5.2DLNA/UPnP Support (33)
Chapter 1
About This Book
This document describes the package contents and provides instructions for building the libraries that are based on the GStreamer architecture. GStreamer is a powerful, versatile framework for creating streaming media applications.
1.1Audience
This document is intended for software, hardware, and system engineers who are planning to use multimedia codecs with GStreamer architecture and for anyone who wants to understand more about multimedia codecs. The document assumes that the user has a basic understanding of GStreamer and LTIB architecture.
1.2Organization
This document consists of the following chapters:
?Chapter 1: Provides an introduction of the document.
?Chapter 2: Identifies the system requirements for installing and building plug-ins, and explains how to build multimedia components using LTIB or install multimedia components on Ubuntu operating system.
?Chapter 3: Explains how to test and use multimedia codecs.
?Chapter 4: Describes how to add multi-overlay support for video playback.?Chapter 5: Describes how to add streaming support.
1.3Conventions
This document uses the following conventions:
?Courier New font: This font is used to identify commands, explicit command parameters, code examples, expressions, data types, and directives.
?$ sign: It is used to specify replaceable command parameters.
1.4References
The following documents were referenced to build this document:
?
i.MX 6Dual/6Quad SABRE-SD Linux User's Guide ?
i.MX 6 Series Ubuntu Multimedia Release Notes ?
GStreamer Command-Line Player Application Specification ?i.MX Advanced Toolkit Standard Version User's Guide
1.5Definitions, Acronyms, and Abbreviations
Following is a list of abbreviations used in this document:
?
FSL: Freescale ?
GUI: Graphic user interface ?
SoC: System-on-chip ?
Codec: Coder/decoder ?
HTTP: Hypertext Transfer Protocol ?
LTIB: Linux Target Image Builder ?
ARM: Advanced RISC Machine ?
ASRC: Asynchronous Sample Rate Converter ?
APT: Advanced Packaging Tool ?
Gst: GStreamer (open source multimedia framework)?
gplay: Freescale command-line player with GStreamer backend
Chapter 2
Installing and Building Plug-Ins
This chapter describes how to build/install Freescale multimedia core libraries and Freescale GStreamer plug-ins. Freescale multimedia core libraries are released only in binary format. Freescale GStreamer plug-ins contain source code.
Freescale provides the following two types of release packages:
?LTIB packages: These packages are for Freescale core libraries and GStreamer plug-ins. They include Freescale multimedia core binary libraries and Freescale
GStreamer plug-in source code.
?Debian packages: These packages are for Ubuntu system. Debian binary packages are used to install Freescale core libraries and GStreamer plug-in binaries on an i.MX
6 series board running Ubuntu operating system. Debian source packages are used to
build Freescale GStreamer plug-ins on an i.MX 6 series board.
2.1Building Plug-Ins by Using LTIB
2.1.1System Requirements for Installing LTIB
?Hardware requirements:
?i.MX 6Dual/Quad/6Solo/6DualLite/6SoloLite SABRE-SD/SABRE-AI/EVK
board
?Software requirements:
?Board Support Package (BSP):
?i.MX Linux BSP version L3.0.35_4.1.0 or above
?GStreamer:
?gstreamer (version 0.10.35)
?gstreamer-plugins-base (version 0.10.35)
?gstreamer-plugins-good (version 0.10.30)
NOTE
Freescale GStreamer plug-ins are dependent on the GStreamer
framework, including the GStreamer core, gstreamer-plugins-
base, and gstreamer-plugins-good.
2.1.2Building Plug-Ins
To implement the procedures given in this section, use a computer (x86) running a Linux operating system.
To install LTIB and extract the package files, follow these steps:
1.Install LTIB on your computer.
./
NOTE
To install LTIB, see i.MX Linux User's Guide of the target
platform.
2.Obtain the following packages included in the release:
?gst-fsl-plugins-$VERSION.tar.gz: It is a GStreamer plug-in source package that
contains source code for the multimedia GStreamer-based plug-in for the i.MX
application processor.
?libfslcodec-$VERSION.tar.gz: It is a codec binary package that contains
Freescale multimedia core codec libraries for the i.MX application processor.
?libfslparser-$VERSION.tar.gz: It is a parser binary package that contains
Freescale multimedia core parser libraries for the i.MX application processor.
?libfslvpuwrap-$VERSION.tar.gz: It is a VPU-wrap source package.
?gst-plugins-gl-[version].tar.gz: It is a GL plug-in source package.
?fsl-alsa-plugins-[version].tar.gz: It is an ALSA plug-in source package.
NOTE
The libfslvpuwrap, gst-plugins-gl, and fsl-alsa-plugins
packages are only available on i.MX 6Dual/6Quad/
6DualLite platform. For details, see "Release Contents"
section of i.MX 6 Series Ubuntu Multimedia Release
Notes.
3.Copy these standard packages to the LPP directory, which, by default, is set to /var/
tmp/pkgs (see litb/.ltibrc %ldirs).
NOTE
If you are installing LTIB for the first time, you need to
create this directory manually.
To build the package, follow these steps:
1.Select the platform.
NOTE
For details, see "Building the Linux Platform" chapter of
i.MX 6Dual/6Quad SABRE-SD Linux User Guide and i.MX
6Solo/6DualLite SABRE-SD Linux User Guide .
2.Select Package List --> Freescale Multimedia Plugins/Codecs, as shown in the below
figure.
Figure 2-1. LTIB Package Selection Menu
3.Select libfslcodec, libfslparser, libfslvpuwrap, gst-fsl-plugins, gst-plugins-gl, and fsl-
alsa-plugins, as shown in the below figure.
Figure 2-2. Selecting Plug-Ins
4.
Select gstreamer-plugins-good package, as shown in the below figure.
Figure 2-3. Selecting gstreamer-plugins-good Package
5.Follow the LTIB compilation instructions.After a successful build, uImage and rootfs will be created. The rootfs is located in LTIB directory. It includes Freescale multimedia Linux codecs and Freescale GStreamer plug-ins. The uImage is located in rootfs/boot in LTIB build directory.
NOTE
For detailed information on a platform, see i.MX Linux User's
Guide for that platform.
2.2Installing/Building Plug-Ins on Ubuntu
2.2.1System Requirements for Ubuntu
?
i.MX 6 series board runs Ubuntu operating system (11.10 Oneiric).
?imx-lib Debian package (version>=4.1.0).
?firmware-imx Debian package (version>=4.1.0).
?GStreamer:
?gstreamer (version=0.10.35 on Ubuntu 11.10)
?gstreamer-plugins-good (version=0.10.30 on Ubuntu 11.10)
2.2.2Installing/Building Plug-Ins
This section describes how to build and generate Debian packages on the i.MX 6 series board natively.
Following steps illustrate how to install Freescale multimedia plug-ins on Ubuntu and also how to build a Debian package from the source.
1.Prepare an i.MX 6 series board running Ubuntu operating system (11.10 Oneiric).
2.Install BSP support libraries package, which is released with the BSP. You need to
make the .deb files available to the board or a USB key. Copying the files to the
board in some other way is also fine.
sudo dpkg -i imx-lib-$VERSION-$RELEASE.deb
sudo dpkg -i kernel_$VERSION-imx_$RELEASE_armel.deb
NOTE
It is desired to install all other BSP Debian packages as
well.
3.Obtain the Debian binary packages given in the below two tables from the
multimedia release package. You can copy them to a USB key that will be connected to the board. The packages listed in the first table are repackaged to avoid any
toolchain compatibility issue.
Use the dpkg command to install these packages:
sudo dpkg -i *.deb
4.To build the codec and plug-in deb packages from source, install the below tools and
dependencies:
sudo apt-get update
sudo apt-get install aptitude
sudo aptitude install dpkg-dev
sudo aptitude install devscripts
sudo aptitude install dh-autoreconf
For fundamental packages:
sudo aptitude install ntp cdbs dh-autoreconf gnupg-agent keychain pbuilder quilt
For GStreamer plug-ins:
sudo apt-get install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev gstreamer-tools
For gst-plugins-gl:
sudo apt-get install mesa-common-dev libgtk2.0-dev libjpeg-dev libpng12-dev
For fsl-alsa-plugins:
sudo apt-get install libasound2-dev
It is required to install the dev Debian packages from the multimedia release package to solve the build dependency.
5.Create a directory, and copy all the source Debian packages given in the below table
to this directory.
6.Run the following command to build each of the above packages:
dpkg-source -x
cd
debuild -i -uc -us
The Debian binaries will be created at the above directory. You can install them by using the following command:
dpkg -i
Chapter 3
Testing Installation
This chapter explains how to check and test the multimedia codecs (audio decoder, audio encoder, video decoder, and video encoder). It also explains how to enable the post-process filter to the pipeline to be created in the GStreamer architecture.
NOTE
Each platform provides a certain set of codecs. See i.MX 6
Series Ubuntu Multimedia Release Notes to determine which
codecs are included in the BSP release package.
3.1Testing Multimedia Environment Setting
If the test rootfs is Ubuntu, some audio/video input/output may need to be configured before testing. In below description, it is assumed that pulseaudio is installed.
3.1.1Audio Output Setting
Use "pactl" command to list all available audio sinks:
pactl list sinks
A list of available audio sinks will be displayed as shown below:
Sink #0
State: SUSPENDED
Name: alsa_output.platform-soc-audio.1.analog-stereo
Description: sgtl5000-audio Analog Stereo
...
...
Sink #1
State: SUSPENDED
Name: alsa_output.platform-soc-audio.4.analog-stereo
Description: imx-hdmi-soc Analog Stereo
...
...
Use "pacmd" command to set the default audio sink according to the sink number given above:
pacmd set-default-sink $sink-number (e.g. $sink-number could be 0 or 1 in above list)After setting the default sink, use below command to verify the audio path:
gst-launch audiotestsrc ! pulsesink
NOTE
The pulseaudio is only available for Ubuntu rootfs. For LTIB
environment HDMI output, use "aplay -l" to check the audio
device. Use "gst-launch playbin2 uri=
sink="alsasink device=plughw:$audio_device_number"" to
output to the specific device.
3.1.2Audio Input Setting
Use "pactl" command to list all available audio sources:
pactl list sources
A list of available audio sources will be displayed as shown below:
Source #0 State: SUSPENDED Name: alsa_output.platform-soc-audio.1.analog-stereo.monitor Description: Monitor of sgtl5000-audio Analog Stereo ... ...Source #1 State: SUSPENDED Name: alsa_input.platform-soc-audio.1.analog-stereo Description: sgtl5000-audio Analog Stereo ... ... ...
Use "pacmd" command to set default audio source according to the source number given above:
pacmd set-default-source $source-number (e.g. $source-number could be 0 or 1 in above list)
NOTE
If record and playback are not needed at the same time, there is
no need to set to monitor mode.
Use "pactl" command to see the current status of audio input/output path setting:
pactl stat
A list will be displayed as given below:
Currently in use: 1 blocks containing 64.0 KiB bytes total.
Allocated during whole lifetime: 2931 blocks containing 5.3 MiB bytes total.
Sample cache size: 0 B Server String: /home/linaro/.pulse/82aa6303a555980d8320686e000e1e89-runtime/native Library Protocol Version: 24Server Protocol Version: 24Is Local: yes Client Index: 11Tile Size: 65496User Name: linaro Host Name: linaro-ubuntu-desktop Server Name: pulseaudio Server Version: 1.0Default Sample Specification: s16le 2ch 44100Hz Default Channel Map: front-left,front-right Default Sink: alsa_output.platform-soc-audio.1.analog-stereo Default Source: alsa_input.platform-soc-audio.1.analog-stereo Cookie: e9a5:dcd9
3.1.3Video Setting
See i.MX Linux User's Guide to know how to set boot cmd in U-Boot for different video output (HDMI, LVDS, and so on).
3.2Testing Codecs with GStreamer
GStreamer provides two useful applications for testing multimedia codecs: gst-inspect and gst-launch.
3.2.1gst-inspect Tool
The gst-inspect tool can provide information about an available GStreamer plug-in, a particular plug-in, or a particular element.
To view the list of installed Freescale multimedia codec plug-ins, use the following command:
gst-inspect | grep imx
A list is displayed as shown below:
wmadec.imx: mfw_wma10decoder: wma audio decoder aiur.imx: aiurdemux: aiur universal demuxer v4lsink.imx: mfw_v4lsink: v4l2 video sink amrdec.imx: mfw_amrdecoder: amr audio decoder beep.imx: beepdec: beep audio decoder wmvdec.imx: mfw_wmvdecoder: wmv video decoder aacpdec.imx: mfw_aacplusdecoder: aac plus audio decoder mp3enc.imx: mfw_mp3encoder: mp3 audio encoder v4lsrc.imx: mfw_v4lsrc: v4l2 based src mp3dec.imx: mfw_mp3decoder: mp3 audio decoder isink.imx: mfw_isink: IPU-based video sink adownmix.imx: mfw_downmixer: audio down mixer
vorbisdec.imx: mfw_vorbisdecoder: vorbis audio decoder
wma8enc.imx: mfw_wma8encoder: wma8 audio encoder
aacdec.imx: mfw_aacdecoder: aac audio decoder
audiopeq.imx: mfw_audio_pp: audio post equalizer
vpu.imx: vpudec: VPU-based video decoder
The elements contained in this list may be different depending on the target platform. For example:
vpu.imx: vpudec: VPU-based video decoder
Here, "vpu.imx" is plug-in name, "vpudec" is element name, and "VPU-based video decoder" is long name of the element.
Use the following gst-inspect command to view the detailed information of an element: gst-inspect $ELEMENT_NAME
For example, use the below command to display the detailed information of the vpudec element:
gst-inspect vpudec
All these plug-ins can be classified into audio decoder/encoder, video decoder/encoder,
demuxer, and so on (as shown in the below table).