In a previous post, I described how to package and manipulate WebVTT content in MP4 files according to the latest ISO standard using MP4Box. Basic import of WebVTT or SRT file is as follows:
MP4Box -add file1.vtt:lang=en subtitle1.mp4
MP4Box -add file2.srt:FMT=VTT:lang=en subtitle2.mp4
and then, the basic usage to create DASH subtitle segments of 10 seconds:
MP4Box -dash 10000 subtitle.mp4:role=subtitle video.mp4
It is now possible to play these MP4 files with the GPAC players (on all supported platforms: Win, Mac, Linux, Android, iOS). So, try it out and let me know if it has bugs. You can for instance test this file or its DASH version.
MP4Client http://download.tsi.telecom-paristech.fr/gpac/webvtt/counter-vtt.mp4 MP4Client http://download.tsi.telecom-paristech.fr/gpac/webvtt/dash/counter-subtitles.mpd
This post describes some details on how the rendering was achieved.
The rendering of WebVTT cues is performed as follows. Upon detection of a WebVTT stream in an MP4 file, a dedicated decoding pipeline is created (by the vtt_in module) and a graphical layer is added on top of the video layer, if a video is also present in the MP4 file.
So far, WebVTT rendering in GPAC supports :
- Top-based line positioning, bottom-based line positioning, explicit positioning
- Line wrapping
- Multiline cues
- Multiple cues simultaneously
- Basic text styling: italic, bold, underlined
Support for regions has not been added.
With this player we have been able to validate the playback of WebVTT cues in the following conditions:
- Stored in MP4 files and read from disk or HTTP;
- Stored in fragmented MP4 files and read as part of a DASH presentation;
- With a video or without a video
- Seeking into the stream
Currently, WebVTT playback is limited to cues packaged in MP4 files. Raw WebVTT files are not yet played. Some of the reasons for that are:
- Storage of WebVTT cues in MP4 files ensures that each sample is a RAP; therefore, at each sample, previous cues can be safely discarded. Rendering is easier.
- Storage in MP4 also easily enables delivering WebVTT content in a DASH streaming session, which is of high interest at the moment.
The associated MPEG contribution is here:
One thought on “WebVTT, MP4 files, DASH and GPAC”