DASH, MPEG-2 TS, Synchronization

Towards Hybrid IP-based and MPEG-2 TS non-IP-based synchronized delivery of media streams

1 Comment

Hi all,

With some delay, I’ve put the slides that we presented at MPEG in October 2012, on how to extend MPEG-2 TS and MPEG-DASH to support synchronized playback of streams coming from IP based networks and from MPEG-2 TS based (non-IP) networks. The contribution is here:

[textimport http://biblio.telecom-paristech.fr/cgi-bin/ws/biblio.cgi?author=cyril+concolato&type=standardisation&etat=submitted&lang=en&dept=tsi&year=2012&id=12905]

and the slides here:

[slideshare id=16005665&doc=m26903-hybriddeliveryts-130115093207-phpapp02]

Diffusion Curves, Streaming, SVG, Vector Graphics

Presenting SVG and my research work at Klagenfurt university

Leave a comment

Dear followers,

Last week, while teaching a course on interactive multimedia applications at Klagenfurt University, Austria, I gave a talk about the upcoming SVG 2 standard and about my research work in this area. You can find below the slides of this presentation:

[slideshare id=15641011&doc=2012-klagenfurt-researchtalk-121214112913-phpapp02]

DASH, GPAC, HTML5, MPEG, Multimedia, Streaming, Synchronization, WebVTT

Live streaming of video and subtitles using DASH

1 Comment

This post contains the presentation made at the MPEG meeting in Shanghai, China, in October 2012, related to the input contribution M26906. The presentation gives the details about the demonstration made during the meeting. This demonstration showed the use of the Google Chrome browser to display synchronized video and subtitles, using the Media Source Extension draft specification and the WebVTT subtitle format. The video and DASH content was prepared using GPAC MP4Box tool.

[slideshare id=15636451&doc=m26906-livestreamingofvideoandsubtitleswithdash-121214045244-phpapp01]

HTML5, Streaming, WebVTT

WebVTT streaming

2 Comments

WebVTT is a new subtilting format that is becoming popular amongst browser implementors. Chrome (v23), Opera (v12.5), IE 10 already support it and soon Firefox will too. As opposed to previous formats for subtitles such as DVB subtitles or 3GPP Timed Text, it is being defined by the WHATWG/W3C primarily for the Web. However, the Web being almost ubiquitous, Web technologies now have to be usable in different delivery environment, not only in download and play mode. In particular, just like all the previous subtitle formats, WebVTT has to be also streamable, and for instance usable the context of Dynamic Streaming over HTTP (DASH).  This post is about my experiments on this topic. For those who don’t want to read the whole post, in summary, it seems possible to generate WebVTT streams, with good random access properties, that can be delivered in chunks and be processed by standard browsers. Continue reading WebVTT streaming →

Diffusion Curves, Vector Graphics

Freeform Vector Graphics Properties

Leave a comment

As a follow up of my previous post, I tried to check the behavior of the FVG approach with respect to closed curves. More precisely, I want to see if it would be possible for an author to edit a complex image by editing closed regions of the image. This would simplify the editing. The problem is that the closed regions have to be independent. One doesn’t want the outside of a region to change when tweaking the inside. This is valid for simple diffusion curves proposed by Orzan as they are solution to the Laplace equation and this property is actually used for culling and fast rendering of diffusion curves (see “Diffusion curve textures for resolution independent texture mapping”, by Sun et al.). How true is this for FVG curves, especially given that the solution is not the solution of a Laplace equation but to complex energy minimization? Continue reading Freeform Vector Graphics Properties →

DASH, SVG, Synchronization

Carriage of synchronized graphics and text in MP4 files

Leave a comment

Following my previous post on the synchronization of HTML 5 video and SVG content, we are now proposing a contribution to MPEG, for the upcoming meeting in Stockholm. This work is part of the activity on timed text (subtitling, caption …). We propose to store frame-based SVG, WebVTT, HTML … content as a particular timed metadata track. We take special care of secondary resources, i.e. resources pointed to by the SVG, HTML … The goal is to be able to use SVG, WebVTT or HTML timed tracks in Dynamic Adaptive Streaming over HTTP. Continue reading Carriage of synchronized graphics and text in MP4 files →

HTML5, SVG, Synchronization

HTML 5 Video and SVG Graphics Synchronization

1 Comment

My experience with multimedia standards, such as MPEG-4 BIFS, made me used to very good synchronization between video streams and graphics. So I wanted to check that the same was possible with web technologies, in particular with the plugin-less HTML5. To be more precise, I would like to display graphics on top of a video in a synchronized manner, with frame accuracy. The synchronization should be preserved when: playing, at different speed; paused; seeked; when the video stalls, …. I investigated several options and present some of the results. Continue reading HTML 5 Video and SVG Graphics Synchronization →

GPAC, SVG, Testing

Automated Testing Script Generation

Leave a comment

During a recent SVG Working group discussion, we talked about how to improve the SVG test suite. Part of the problems in the development of the test suite is the sheer number of tests to be produced (i.e. written + reviewed + approved) and then tested against every implementation. The purpose of the testing is to check if every feature passes in at least two independent implementation: that is the criteria to exit Candidate Recommendation. The SVG Tiny 1.2 test suite already contains 496 tests, the SVG 1.1 2nd edition has 526. The SVG 2 spec will probably have more. Having one person test all features is not realistic anymore.

The CSS WG has been working for a long time on this issue. CSS 2.1 has 9685 tests! Together with a team from W3C, the WG set up a test suite server, where one can browse the CSS Test Suite, test its viewer and report the results. All results are then aggregated. The W3C architecture allows for 2 types of tests:

  • ref tests (see Mozilla’s use of ref tests), where a test is provided together with an alternative way of producing the same result (visually).
  • JavaScript tests, where together with the test, one provides JavaScript calls to test that the DOM tree is in the expected state.

Both of these types require quite some work to produce either the reference image or the expected results. This can be tedious. Also, for interactive cases, one would have to create the script to test all interaction paths. And for animation, one would have to author the JavaScript to test changes at some point in time.

In GPAC, we experimented a different approach. Continue reading Automated Testing Script Generation →

Diffusion Curves, Vector Graphics

Experimenting with Freeform Vector Graphics

1 Comment

As part of my interest for Vector Graphics technologies, I had look at the Freeform Vector Graphics (FVG) technology proposed by Microsoft Research. After all, it differs from SVG only by one letter.

The paper describing the technology is available here. A video presenting the approach is also provided by the authors here. The approach is in general an extension or alternative to the work on Diffusion Curves. It enables the control of the interpolation of colors between diffusion curves (or points), with a higher order of smoothness if necessary.

This post reports on my understanding of the method and of its implementation, focusing on the use of the software and on some aspects not dealt with in the paper and is intended for people who have already a good understanding of the paper. If you notice any mistake, if it is unclear or if you disagree, please leave a comment. Continue reading Experimenting with Freeform Vector Graphics →

Groupe Multimédia / Multimedia Group