I gave the following presentations during the Seminar on MPEG-4 BIFS and MPEG-2 TS for the WorldDMB Technical Committee. The first one explains the basics of MPEG-4 BIFS and the second one shows the latest developments for Digital Radio Services, the new BIFS tools and profiles and the modification for the proposed carriage of those services over MPEG-2 TS. As usual, if you have any comment please contact me. Continue reading MPEG-4 BIFS and MPEG-2 TS Seminar @ WorldDMB
ISO/IEC 14496-11:2005/Amd.7:2010 is an amendment to the MPEG-4 BIFS standard which defines a new profile, called ExtendedCore2D. This amendment is intended to improve services such as T-DMB mobile television as deployed in South Korea or Digital Radio services as shown in Figure 1. The goal of this profile is to enable richer services with reduced bandwidth requirements. It is based on the existing Core2D profile, extends it with existing tools from the MPEG-4 BIFS standard not yet included in the Core2D profile, and with newly defined tools.
Figure 1 – Example of BIFS Digital Radio Service – Live and audio-synchronized screen
(courtesy of VizionR)
Among the extensions of the Core2D profile from existing BIFS technologies, the ExtendedCore2D allows defining services with rich 2D vector graphics primitives. In particular, authors can now use Bézier curves (with the Curve2D or XCurve2D nodes) or ellipses. Additionally, the profile supports tools to stroke vector graphics objects, for instance with the LineProperties or XLineProperties nodes; and to fill or stroke objects with gradients using the LinearGradient or RadialGradient nodes. Using these nodes will save bandwidth as some raster images can be replaced by more compact binary vector graphics representations. Finally, the visual quality of the new services can now remain intact when a scene designed for a smartphone screen is displayed on a larger tablet screen. An example of Digital Radio Service on tablet is shown in Figure 2.
Figure 2 – Example of BIFS Digital Radio Service – Usage of vector graphics
(courtesy of VizionR and Fun Radio)
New and richer services defined with the ExtendedCore2D profile also need more compact and more efficient tools for scene graph management including layout description. For better scene management, the new profile includes the PROTO tool, to help reduce scene complexity and improve the coding efficiency. In term of layout, the new profile adds the ability to position objects, including text and vector graphics, in paragraphs with the Layout node. It also adds the possibility to transform, reuse, view and animate parts of the layout efficiently to achieve compact and appealing animations with the TransformMatrix2D, CompositeTexture2D and Viewport nodes.
The BIFS amendment also defines several new tools to enrich services. Some of them target a reduction of the bandwidth required by BIFS services. In particular, the CacheTexture node enables the carriage of compressed images within the BIFS stream, removing the use and signaling of some elementary streams dedicated for raster image delivery. The bandwidth gain can be important when the service contains many images, as shown in Figure 3. Some new tools provide new features for service authors. The EnvironmentTest node enables the design of services that adapt to viewing conditions, e.g. to the screen size, to the presence of keypad or to the viewing in cars. The KeyNavigator node facilitates the design of the navigation within rich scenes. With the Storage node, services can be designed to be persistent across activations and deactivations, offering the user a restoration of its settings. Finally, in T-DMB environments, it may be interesting to reuse, within the BIFS service, data carried outside the BIFS streams, such as Electronic Program Guides. For this purpose, a new type of BIFS update, called ReplaceToExternalData, is defined.
Figure 3 – Example of BIFS Digital Radio Service – News and non-synchronized screen with multiple images (courtesy of VizionR)
This video shows how some of the GPAC tools can be used to encode, deliver and play Digital Radio Services according to the T-DMB standard. It also shows GPAC running on iPad.
As some of you know, the GPAC player is compliant Digital Radio player, according to the T-DMB standard (i.e. according to MPEG-2 TS, MPEG-4 SL, MPEG-4 AAC and MPEG-4 BIFS). Given the fact that GPAC is big and implements many standards, the question on its footprint was raised when considering embedded platforms. In the GPAC code, there is a possibility to compile the code, removing a lot of unnecessary features. On Windows, you have to tweak the include/gpac/configuration.h file. On linux, you have to deal with the configure script. Here is an example:
Using SVN revision 2136.
concolat@virtualubuntu:~/svn/gpac/bin/gcc$ ./configure --use-js=no
--use-mad=no --use-xvid=no --use-ffmpeg=no --use-ogg=no --use-vorbis=no
--use-theora=no --use-openjpeg=no --disable-streaming --disable-isoff-frag
--disable-isoff-hint --disable-isoff-write --disable-loader-xmt
--disable-loader-bt --disable-loader-isoff --disable-scene-encode
--disable-mcrypt --disable-od-dump --disable-scene-dump --disable-scene-stats
--disable-swf --disable-export --disable-import --disable-m2ps --disable-ogg
-disable-avi --disable-qtvr --disable-beng --disable-x3d --disable-3d
--disable-ssl --disable-jack --disable-pulse --use-a52=no --disable-odf --disable-isoff
With this you can compile, just the MP4Client application and the few plugins needed (MPEG-2 TS, BIFS, ODF, AAC, Video ouput, audio output, rasterizer and font engine) and you will be able to play T-DMB files (no MP4, no BT or XMT).
The result of a Windows XP 32 bits compilation is the following:
MP4Client application: 76 KB
GPAC Core library: 1240 KB
AAC decoder plugin: 244 KB
BIFS decoder plugin: 7.5 KB
ODF decoder plugin: 8.5 KB
MPEG-2 TS plugin: 16.5 KB
DirectX Audio/Video plugin: 64 KB
FreeType font engine plugin: 360 KB
Rasterizer plugin: 32.5 KB
JPEG & PNG decoder plugin: 72 KB
This make a total T-DMB player for 2.07 MB (759 KB as a 7z archive !!). There is no other dependency (except the C runtime, Direct X runtime). This is good but could be improved (e.g. the BIFS decoder could be simplified to support only the nodes in the T-DMB profile, all unnecessary functions in the core library are not excluded from the compilation …).
On Ubuntu 10.10, 32 bits, compiled with -Os and stripped, the results are:
MP4Client application: 63 KB
GPAC Core library: 1726 KB
AAC decoder plugin: 9.5 KB
BIFS decoder plugin: 5.5 KB
ODF decoder plugin: 5.5 KB
MPEG-2 TS plugin: 26 KB
OSS Audio plugin: 9.5 KB
X11 Video plugin: 22 KB
FreeType font engine plugin: 34 KB
Rasterizer plugin: 52 KB
JPEG & PNG decoder plugin: 13.5 KB
This make a TAR package of 1.9 MB and a TGZ of 743 KB. Very light, I think, even if in this case some dependencies are hidden on my system (libm, libz, libjpeg, libpng12, libpthread, libdl, and libc).