As of version 5017 of GPAC, I’ve added a first preliminary support of storage of TTML in MP4. The support is according to the ISOBMFF spec (MPEG-4 Part 30, to appear) and to the DECE CFF format. This post describes why and how to use this implementation. Continue reading First attempt at storing TTML in MP4
Category Archives: GPAC
Using WebVTT to carry media streams
Some of you may know the WebVTT format as a subtitling format. But WebVTT can also be used to carry video-synchronized metadata, i.e. any data that is not meant to be displayed, as is (or at all), but that should be processed at a given time in the video by some JavaScript code in the HTML page.
In this post, I describe the results of the modifications I made to MP4Box to export any MP4 tracks in WebVTT, including audio or video data. The idea here is not to replace the MP4 file format by WebVTT because WebVTT was not meant for that and is not really good at that (for one it is a textual format). The idea is more to provide a (temporary) way for people to experiment with JavaScript decoders for their data. You can already start using MP4Box to test but this work is still preliminary and is subject to changes.
HTML 5 Media Source Extensions in SVG
In a previous post, I described a bit the support in GPAC for the HTML 5 video and audio elements in SVG documents. My initial idea was to be able to play, with the Media Source Extensions API, adaptive streaming content (such as DASH) in SVG. So I started implementing MSE and it is now possible to play DASH content within an SVG presentation using GPAC. This post details how it was implemented. Continue reading HTML 5 Media Source Extensions in SVG
WebVTT, MP4 files, DASH and GPAC
In a previous post, I described how to package and manipulate WebVTT content in MP4 files according to the latest ISO standard using MP4Box. Basic import of WebVTT or SRT file is as follows:
MP4Box -add file1.vtt:lang=en subtitle1.mp4
or
MP4Box -add file2.srt:FMT=VTT:lang=en subtitle2.mp4
and then, the basic usage to create DASH subtitle segments of 10 seconds:
MP4Box -dash 10000 subtitle.mp4:role=subtitle video.mp4
It is now possible to play these MP4 files with the GPAC players (on all supported platforms: Win, Mac, Linux, Android, iOS). So, try it out and let me know if it has bugs. You can for instance test this file or its DASH version.
MP4Client http://download.tsi.telecom-paristech.fr/gpac/webvtt/counter-vtt.mp4 MP4Client http://download.tsi.telecom-paristech.fr/gpac/webvtt/dash/counter-subtitles.mpd
This post describes some details on how the rendering was achieved.
[slideshare id=24802693&doc=m30301-reportonwebvttimplementation-130731093103-phpapp01]
WebVTT in MP4
As you probably know from my previous posts, MPEG is standardizing a way to deliver WebVTT files as streams (or tracks) in the ISO Base Media File Format (e.g. MP4 files). The standard is still at draft stage but I recently added VTT-in-MP4 support to GPAC (no playback support yet), as reported in the contribution below. Not that this support might change in the future depending on how the draft standard progresses, in particular in response to some comments we’ve made, as reported in the contribution referenced in this post. Continue reading WebVTT in MP4
HTML 5 media elements in SVG documents
As a follow-up of the work I did some time ago on the implementation of SVG Tiny 1.2 in GPAC (see this paper), and following the SVG WG decision to adopt the HTML 5 media elements in SVG 2.0, I decided to implement some support for these elements in GPAC, to better understand the differences between the old SVG Tiny 1.2 <video> element and the HTML 5 <video> element. As a result you can now run for example the following markup in GPAC:
function init() {
var v = document.getElementById("v");
v.src = "video.mp4";
v.play();
}
Presentation of GPAC Research Activities
Here are the slides I gave for the presentation of the research activities of the GPAC team of Telecom ParisTech during the plenary session of the “Réseau Thématique 4” of the Mines-Telecom Institute.
[slideshare id=18460316&doc=gpacimt-rt4-130409033821-phpapp01]
Live streaming of video and subtitles using DASH
This post contains the presentation made at the MPEG meeting in Shanghai, China, in October 2012, related to the input contribution M26906. The presentation gives the details about the demonstration made during the meeting. This demonstration showed the use of the Google Chrome browser to display synchronized video and subtitles, using the Media Source Extension draft specification and the WebVTT subtitle format. The video and DASH content was prepared using GPAC MP4Box tool.
[slideshare id=15636451&doc=m26906-livestreamingofvideoandsubtitleswithdash-121214045244-phpapp01]
Automated Testing Script Generation
During a recent SVG Working group discussion, we talked about how to improve the SVG test suite. Part of the problems in the development of the test suite is the sheer number of tests to be produced (i.e. written + reviewed + approved) and then tested against every implementation. The purpose of the testing is to check if every feature passes in at least two independent implementation: that is the criteria to exit Candidate Recommendation. The SVG Tiny 1.2 test suite already contains 496 tests, the SVG 1.1 2nd edition has 526. The SVG 2 spec will probably have more. Having one person test all features is not realistic anymore.
The CSS WG has been working for a long time on this issue. CSS 2.1 has 9685 tests! Together with a team from W3C, the WG set up a test suite server, where one can browse the CSS Test Suite, test its viewer and report the results. All results are then aggregated. The W3C architecture allows for 2 types of tests:
- ref tests (see Mozilla’s use of ref tests), where a test is provided together with an alternative way of producing the same result (visually).
- JavaScript tests, where together with the test, one provides JavaScript calls to test that the DOM tree is in the expected state.
Both of these types require quite some work to produce either the reference image or the expected results. This can be tedious. Also, for interactive cases, one would have to create the script to test all interaction paths. And for animation, one would have to author the JavaScript to test changes at some point in time.
In GPAC, we experimented a different approach. Continue reading Automated Testing Script Generation
SVG Folded Maps in 3D environments
Some time ago, Erik Dahlström made an SVG version of a folded Google Map. That map uses images, masks and clippers to produce a 3D effect. It is adapted from an HTML/CSS/SVG folded map. I found both pretty cool but I thought I could make another interesting version using 3D tools. Here it is. It’s a mix of 3D shapes onto which I map an SVG content retrieved from OpenStreeetMap. Unfortunately, there’s no shading but interestingly the map folds/unfolds with animations. It’s written in VRML/MPEG-4 BIFS textual language. I guess it could easily be made with WebGL as well (and has been done in a very nice demo by Adobe with CSS Shaders). It works only in GPAC but you can see a resulting video here:
[youtube]http://www.youtube.com/watch?v=fbz98Yc4I_k[/youtube]