Category Archives: GPAC

MPEG-4 BIFS and MPEG-2 TS Seminar @ WorldDMB

I gave the following presentations during the Seminar on MPEG-4 BIFS and MPEG-2 TS for the WorldDMB Technical Committee. The first one explains the basics of MPEG-4 BIFS and the second one shows the latest developments for Digital Radio Services, the new BIFS tools and profiles and the modification for the proposed carriage of those services over MPEG-2 TS. As usual, if you have any comment please contact me. Continue reading MPEG-4 BIFS and MPEG-2 TS Seminar @ WorldDMB

MPEG-4 BIFS ExtendedCore2D profile

ISO/IEC 14496-11:2005/Amd.7:2010 is an amendment to the MPEG-4 BIFS standard which defines a new profile, called ExtendedCore2D. This amendment is intended to improve services such as T-DMB mobile television as deployed in South Korea or Digital Radio services as shown in Figure 1. The goal of this profile is to enable richer services with reduced bandwidth requirements. It is based on the existing Core2D profile, extends it with existing tools from the MPEG-4 BIFS standard not yet included in the Core2D profile, and with newly defined tools.

Figure 1 – Example of BIFS Digital Radio Service – Live and audio-synchronized screen
(courtesy of VizionR)

Among the extensions of the Core2D profile from existing BIFS technologies, the ExtendedCore2D allows defining services with rich 2D vector graphics primitives. In particular, authors can now use Bézier curves (with the Curve2D or XCurve2D nodes) or ellipses. Additionally, the profile supports tools to stroke vector graphics objects, for instance with the LineProperties or XLineProperties nodes; and to fill or stroke objects with gradients using the LinearGradient or RadialGradient nodes. Using these nodes will save bandwidth as some raster images can be replaced by more compact binary vector graphics representations. Finally, the visual quality of the new services can now remain intact when a scene designed for a smartphone screen is displayed on a larger tablet screen. An example of Digital Radio Service on tablet is shown in Figure 2.

Figure 2 – Example of BIFS Digital Radio Service – Usage of vector graphics
(courtesy of VizionR and Fun Radio)

New and richer services defined with the ExtendedCore2D profile also need more compact and more efficient tools for scene graph management including layout description. For better scene management, the new profile includes the PROTO tool, to help reduce scene complexity and improve the coding efficiency. In term of layout, the new profile adds the ability to position objects, including text and vector graphics, in paragraphs with the Layout node. It also adds the possibility to transform, reuse, view and animate parts of the layout efficiently to achieve compact and appealing animations with the TransformMatrix2D, CompositeTexture2D and Viewport nodes.

The BIFS amendment also defines several new tools to enrich services. Some of them target a reduction of the bandwidth required by BIFS services. In particular, the CacheTexture node enables the carriage of compressed images within the BIFS stream, removing the use and signaling of some elementary streams dedicated for raster image delivery. The bandwidth gain can be important when the service contains many images, as shown in Figure 3. Some new tools provide new features for service authors. The EnvironmentTest node enables the design of services that adapt to viewing conditions, e.g. to the screen size, to the presence of keypad or to the viewing in cars. The KeyNavigator node facilitates the design of the navigation within rich scenes. With the Storage node, services can be designed to be persistent across activations and deactivations, offering the user a restoration of its settings. Finally, in T-DMB environments, it may be interesting to reuse, within the BIFS service, data carried outside the BIFS streams, such as Electronic Program Guides. For this purpose, a new type of BIFS update, called ReplaceToExternalData, is defined.

Figure 3 – Example of BIFS Digital Radio Service – News and non-synchronized screen with multiple images (courtesy of VizionR)

Test suite and validation

Most standards produce test suites to demonstrate the features of the standard. This is also very useful to test implementations. GPAC also has a (very limited) test suite to test that the code does not regress. However, when the tests need to be clicked on or when there is animation, the validation of a test may get quite complex.

There are several ways to solve this problem. Erik Dahlström from Opera told me that they use additional JavaScript with specific APIs to do their regression testing. This is interesting but I’m concerned by the time it takes to author this Javascript. So I  thought about another way.

Within GPAC, I’ve implemented a small plugin which enables two things:

  • to play a test, record the interactions (in an XML file) and take PNG snapshots upon specific events;
  • and then to replay the content, reproduce the interactions and compare the snapshots to indicate if the result is valid or not.

A playlist of test sequences can then be created and the validation can be automatic. It doesn’t take much time to record the interactions, what I called the validation script.

You should see this soon on GPAC SVN.