Tag Archives: GPAC

Multimedia

GPAC Widgets Video on YouTube

Leave a comment

I have finally had a bit of  time to post on YouTube some old videos that we did to demonstrate the combined use of the W3C Widgets Packaging and Configuration specification and of the MPEG-U Widgets standard. These videos show: simple widget managment (with different GUI styles), communication between widgets and widget migration. Let me know if you have comments. Continue reading GPAC Widgets Video on YouTube →

Multimedia

Digital Radio Player and Memory Footprint

Leave a comment

As some of you know, the GPAC player is compliant Digital Radio player, according to the T-DMB standard (i.e. according to MPEG-2 TS, MPEG-4 SL, MPEG-4 AAC and MPEG-4 BIFS). Given the fact that GPAC is big and implements many standards, the question on its footprint was raised when considering embedded platforms. In the GPAC code, there is a possibility to compile the code, removing a lot of unnecessary features. On Windows, you have to tweak the include/gpac/configuration.h file. On linux, you have to deal with the configure script. Here is an example:

Using SVN revision 2136.

concolat@virtualubuntu:~/svn/gpac/bin/gcc$ ./configure --use-js=no 
--use-mad=no --use-xvid=no --use-ffmpeg=no --use-ogg=no --use-vorbis=no
--use-theora=no --use-openjpeg=no --disable-streaming --disable-isoff-frag
--disable-isoff-hint --disable-isoff-write --disable-loader-xmt
--disable-loader-bt --disable-loader-isoff --disable-scene-encode
--disable-mcrypt --disable-od-dump --disable-scene-dump --disable-scene-stats
--disable-swf --disable-export --disable-import --disable-m2ps --disable-ogg
-disable-avi --disable-qtvr --disable-beng --disable-x3d --disable-3d
--disable-ssl --disable-jack --disable-pulse --use-a52=no --disable-odf --disable-isoff
--disable-parsers --disable-m2ts-mux

With this you can compile, just the MP4Client application and the few plugins needed (MPEG-2 TS, BIFS, ODF, AAC, Video ouput, audio output, rasterizer and font engine) and you will be able to play T-DMB files (no MP4, no BT or XMT).

The result of a Windows XP 32 bits compilation is the following:

MP4Client application: 76 KB
GPAC Core library: 1240 KB
AAC decoder plugin: 244 KB
BIFS decoder plugin: 7.5 KB
ODF decoder plugin: 8.5 KB
MPEG-2 TS plugin: 16.5 KB
DirectX Audio/Video plugin: 64 KB
FreeType font engine plugin: 360 KB
Rasterizer plugin: 32.5 KB
JPEG & PNG decoder plugin: 72 KB

This make a total T-DMB player for 2.07 MB (759 KB as a 7z archive !!). There is no other dependency (except the C runtime, Direct X runtime). This is good but could be improved (e.g. the BIFS decoder could be simplified to support only the nodes in the T-DMB profile, all unnecessary functions in the core library are not excluded from the compilation …).

On Ubuntu 10.10, 32 bits, compiled with -Os and stripped, the results are:

MP4Client application: 63 KB
GPAC Core library: 1726 KB
AAC decoder plugin: 9.5 KB
BIFS decoder plugin: 5.5 KB
ODF decoder plugin: 5.5 KB
MPEG-2 TS plugin: 26 KB
OSS Audio plugin: 9.5 KB
X11 Video plugin: 22 KB
FreeType font engine plugin: 34 KB
Rasterizer plugin: 52 KB
JPEG & PNG decoder plugin: 13.5 KB

This make a TAR package of 1.9 MB and a TGZ of 743 KB. Very light, I think, even if in this case some dependencies are hidden on my system (libm, libz, libjpeg, libpng12, libpthread, libdl, and libc).

Multimedia

InterMedia Final Review

Leave a comment

These two days (Oct. 5th and 6th) are the days for the final review of the European InterMedia Network of Excellence to which we participated. This project has given us the opportunity to collaborate with different european research centers and to prepare several demos:

Most of these demos led to publications and to modifications of GPAC:

  • Improved support for H264|AVC video
  • Addition of a new type of plugin for audio filters
  • Addition of a UPnP stack, first using the Intel Tools and then using Platinum
GPAC

Test suite and validation

Leave a comment

Most standards produce test suites to demonstrate the features of the standard. This is also very useful to test implementations. GPAC also has a (very limited) test suite to test that the code does not regress. However, when the tests need to be clicked on or when there is animation, the validation of a test may get quite complex.

There are several ways to solve this problem. Erik Dahlström from Opera told me that they use additional JavaScript with specific APIs to do their regression testing. This is interesting but I’m concerned by the time it takes to author this Javascript. So I  thought about another way.

Within GPAC, I’ve implemented a small plugin which enables two things:

  • to play a test, record the interactions (in an XML file) and take PNG snapshots upon specific events;
  • and then to replay the content, reproduce the interactions and compare the snapshots to indicate if the result is valid or not.

A playlist of test sequences can then be created and the validation can be automatic. It doesn’t take much time to record the interactions, what I called the validation script.

You should see this soon on GPAC SVN.

Multimedia

Video support in Android

Leave a comment

Today (Oct. 8, 2009), I started playing around with a new phone, an HTC hero, which runs Android (v. 1.5). I tested the video support and here are the preliminary results.

Streaming

Summary

Generic support for

  • RTSP (DESCRIBE, SETUP, PLAY) and reception of SDP, including with MPEG-4 IOD

Stream types supported

  • video/MP4V-ES
  • video/H263-1998
  • audio/AMR
  • audio/mpeg4-generic

Stream types not supported

  • video/H264
  • video/mpeg4-generic
  • video/X-SV3V-ES
  • audio/X-QDM

Darwin Streaming Server Details

sample_100kbit.mov: Media type not supported.

v=0
o=StreamingServer 3463999147 1214234604000 IN IP4 137.194.232.94
s=/sample_100kbit.mov
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:94
t=0 0
a=control:*
a=range:npt=0-  70.00000
m=video 0 RTP/AVP 96
b=AS:79
a=rtpmap:96 X-SV3V-ES/90000
a=control:trackID=3
m=audio 0 RTP/AVP 97
b=AS:14
a=rtpmap:97 X-QDM/22050/2
a=control:trackID=4
a=x-bufferdelay:4.97

sample_300kbit.mov: Media type not supported.

v=0
o=StreamingServer 3463999278 1214234604000 IN IP4 137.194.232.94
s=/sample_300kbit.mov
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:253
t=0 0
a=control:*
a=range:npt=0-  70.00000
m=video 0 RTP/AVP 96
b=AS:209
a=rtpmap:96 X-SV3V-ES/90000
a=control:trackID=3
m=audio 0 RTP/AVP 97
b=AS:44
a=rtpmap:97 X-QDM/32000/2
a=control:trackID=4
a=x-bufferdelay:3.66

sample_50kbit.3gp: Success audio+video

v=0
o=StreamingServer 3463999304 1214234604000 IN IP4 137.194.232.94
s=/sample_50kbit.3gp
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:50
t=0 0
a=control:*
a=range:npt=0-  70.00000
m=video 0 RTP/AVP 96
b=AS:41
a=rtpmap:96 MP4V-ES/90000
a=control:trackID=3
a=cliprect:0,0,144,176
a=framesize:96 176-144
a=fmtp:96 profile-level-id=1;config=000001B008000001B50EE040C0CF0000010000000120
008440FA282C2090A21F
m=audio 0 RTP/AVP 97
b=AS:8
a=rtpmap:97 AMR/8000/1
a=control:trackID=4
a=fmtp:97 octet-align=1

sample.mp3: Fail

sample_100kbit.mp4: Success audio+video

v=0
o=StreamingServer 3463999476 1214234604000 IN IP4 137.194.232.94
s=/sample_100kbit.mp4
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:96
t=0 0
a=control:*
a=mpeg4-iod:"data:application/mpeg4-iod;base64,AoJrAE///w/z/wOBdgABQNhkYXRhOmFwc
GxpY2F0aW9uL21wZWc0LW9kLWF1O2Jhc2U2NCxBWUVDQVV3Rkh3TklBTWtnQUdVRUx5QVJBRzNkQUFFa
ytBQUJKUGdGSUFBQUFiRHpBQUFCdFE3Z1FNRFBBQUFCQUFBQUFTQUFoRUQ2S0RBZzhxSWZCaEFBUkFBQ
UFsZ0FBQUFBSUFBQUFBQURBVElDbndNdUFHVUFCSUNBZ0JSQUZRQVlBQUFBVGlBQUFFNGdCWUNBZ0FJV
mtBWVFBRVFBQUI5QUFBQWZRQ0FnQUFBQUF3PT0EDQEFAADIAAAAAAAAAAAGCQEAAAAAAAAAAANpAAJAR
mRhdGE6YXBwbGljYXRpb24vbXBlZzQtYmlmcy1hdTtiYXNlNjQsd0JBU2daTUNvRmNtRUVIOEFBQUIvQ
UFBQkVLQ0tDbjQEEgINAABkAAAAAAAAAAAFAwAAYAYJAQAAAAAAAAAA"
a=isma-compliance:1,1.0,1
a=range:npt=0-  70.00000
m=video 0 RTP/AVP 96
b=AS:76
a=rtpmap:96 MP4V-ES/90000
a=control:trackID=3
a=cliprect:0,0,242,192
a=framesize:96 192-242
a=fmtp:96 profile-level-id=1;config=000001B0F3000001B50EE040C0CF0000010000000120
008440FA283020F2A21F
a=mpeg4-esid:201
m=audio 0 RTP/AVP 97
b=AS:20
a=rtpmap:97 mpeg4-generic/8000/2
a=control:trackID=4
a=fmtp:97 profile-level-id=15;mode=AAC-hbr;sizelength=13;indexlength=3;indexdelt
alength=3;config=1590
a=mpeg4-esid:101

sample_300kbit.mp4: Success audio+video

v=0
o=StreamingServer 3464001266 1214234604000 IN IP4 137.194.232.94
s=/sample_300kbit.mp4
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:258
t=0 0
a=control:*
a=mpeg4-iod:"data:application/mpeg4-iod;base64,AoJrAE///w/z/wOBdgABQNhkYXRhOmFwc
GxpY2F0aW9uL21wZWc0LW9kLWF1O2Jhc2U2NCxBWUVDQVV3Rkh3TklBTWtnQUdVRUx5QVJBVUpFQUFOY
llBQURXMkFGSUFBQUFiRHpBQUFCdFE3Z1FNRFBBQUFCQUFBQUFTQUFoRUQ2S0Y4aDRLSWZCaEFBUkFBQ
UFCNEFBQUFBSUFBQUFBQURBVElDbndNdUFHVUFCSUNBZ0JSQUZRQVlBQUFBdTRBQUFMdUFCWUNBZ0FJV
GtBWVFBRVFBQUZZaUFBQldJaUFnQUFBQUF3PT0EDQEFAADIAAAAAAAAAAAGCQEAAAAAAAAAAANpAAJAR
mRhdGE6YXBwbGljYXRpb24vbXBlZzQtYmlmcy1hdTtiYXNlNjQsd0JBU2daTUNvRmNtRUVIOEFBQUIvQ
UFBQkVLQ0tDbjQEEgINAABkAAAAAAAAAAAFAwAAYAYJAQAAAAAAAAAA"
a=isma-compliance:1,1.0,1
a=range:npt=0-  70.00000
m=video 0 RTP/AVP 96
b=AS:209
a=rtpmap:96 MP4V-ES/90000
a=control:trackID=3
a=cliprect:0,0,480,380
a=framesize:96 380-480
a=fmtp:96 profile-level-id=1;config=000001B0F3000001B50EE040C0CF0000010000000120
008440FA285F21E0A21F
a=mpeg4-esid:201
m=audio 0 RTP/AVP 97
b=AS:48
a=rtpmap:97 mpeg4-generic/22050/2
a=control:trackID=4
a=fmtp:97 profile-level-id=15;mode=AAC-hbr;sizelength=13;indexlength=3;indexdelt
alength=3;config=1390
a=mpeg4-esid:101

sample_h264_100kbit.mp4: Impossible to read this video

v=0
o=StreamingServer 3463999589 1214234604000 IN IP4 137.194.232.94
s=/sample_h264_100kbit.mp4
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:2097172
t=0 0
a=control:*
a=isma-compliance:2,2.0,2
a=range:npt=0-  70.00000
m=video 0 RTP/AVP 96
b=AS:2097151
a=rtpmap:96 H264/90000
a=control:trackID=3
a=cliprect:0,0,242,192
a=framesize:96 192-242
a=fmtp:96 packetization-mode=1;profile-level-id=4D400C;sprop-parameter-sets=J01A
DKkYYELxCA==,KM4JiA==
a=mpeg4-esid:201
m=audio 0 RTP/AVP 97
b=AS:20
a=rtpmap:97 mpeg4-generic/8000/2
a=control:trackID=4
a=fmtp:97 profile-level-id=15;mode=AAC-hbr;sizelength=13;indexlength=3;indexdelt
alength=3;config=1590
a=mpeg4-esid:101

sample_h264_300kbit.mp4: Impossible to read this video

v=0
o=StreamingServer 3463999675 1214234605000 IN IP4 137.194.232.94
s=/sample_h264_300kbit.mp4
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:2097200
t=0 0
a=control:*
a=isma-compliance:2,2.0,2
a=range:npt=0-  70.00000
m=video 0 RTP/AVP 96
b=AS:2097151
a=rtpmap:96 H264/90000
a=control:trackID=3
a=cliprect:0,0,480,380
a=framesize:96 380-480
a=fmtp:96 packetization-mode=1;profile-level-id=4D401E;sprop-parameter-sets=J01A
HqkYMB73oA==,KM4NiA==
a=mpeg4-esid:201
m=audio 0 RTP/AVP 97
b=AS:48
a=rtpmap:97 mpeg4-generic/22050/2
a=control:trackID=4
a=fmtp:97 profile-level-id=15;mode=AAC-hbr;sizelength=13;indexlength=3;indexdelt
alength=3;config=1390
a=mpeg4-esid:101

sample_h264_1mbit.mp4: Impossible to read this video

v=0
o=StreamingServer 3463999710 1214234605000 IN IP4 137.194.232.94
s=/sample_h264_1mbit.mp4
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:2097279
t=0 0
a=control:*
a=isma-compliance:2,2.0,2
a=range:npt=0-  70.00000
m=video 0 RTP/AVP 96
b=AS:2097151
a=rtpmap:96 H264/90000
a=control:trackID=3
a=cliprect:0,0,480,380
a=framesize:96 380-480
a=fmtp:96 packetization-mode=1;profile-level-id=4D401E;sprop-parameter-sets=J01A
HqkYMB73oA==,KM4C+IA=
a=mpeg4-esid:201
m=audio 0 RTP/AVP 97
b=AS:127
a=rtpmap:97 mpeg4-generic/48000/2
a=control:trackID=4
a=fmtp:97 profile-level-id=15;mode=AAC-hbr;sizelength=13;indexlength=3;indexdelt
alength=3;config=1190
a=mpeg4-esid:101

movie_av_hint.mp4: Media file type not supported

v=0
o=StreamingServer 3463999989 1214235184000 IN IP4 137.194.232.94
s=/movie_av_hint.mp4
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:401
t=0 0
a=control:*
a=x-copyright: MP4/3GP File hinted with GPAC 0.4.5-DEV (build 24) (C)2000-2005 -
 http://gpac.sourceforge.net
a=range:npt=0- 195.02333
m=video 0 RTP/AVP 96
b=AS:339
a=rtpmap:96 mpeg4-generic/90000
a=control:trackID=65536
a=fmtp:96 profile-level-id=245; config=000001b0f5000001b509000001000000012000868
7fffb0aad8482098a31000001b244697658393939623030306e000001b25876694430303239; str
eamType=4; mode=generic; objectType=32; DTSDeltaLength=14
a=framesize:96 288-152
m=audio 0 RTP/AVP 97
b=AS:62
a=rtpmap:97 mpeg4-generic/44100/2
a=control:trackID=65537
a=fmtp:97 profile-level-id=41; config=1210; streamType=5; mode=AAC-hbr; objectTy
pe=103; sizeLength=13; indexLength=3; indexDeltaLength=3

counter.mp4: Media file type not supported

v=0
o=StreamingServer 3464000048 1214234604000 IN IP4 137.194.232.94
s=/counter.mp4
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:117
t=0 0
a=control:*
a=x-copyright: MP4/3GP File hinted with GPAC 0.4.3-DEV (C)2000-2005 - http://gpa
c.sourceforge.net
a=range:npt=0-  10.00000
m=video 0 RTP/AVP 96
b=AS:117
a=rtpmap:96 mpeg4-generic/90000
a=control:trackID=65536
a=fmtp:96 profile-level-id=245; config=000001b0f5000001b509000001000000012000868
400670c0f1030518f000001b244697658393939623030306e000001b25876694430303239; strea
mType=4; mode=generic; objectType=32; DTSDeltaLength=14
a=framesize:96 120-96

cvid.mp4: Media file type not supported

v=0
o=StreamingServer 3464000094 1214327349000 IN IP4 137.194.232.94
s=/cvid.mp4
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:117
t=0 0
a=control:*
a=x-copyright: MP4/3GP File hinted with GPAC 0.4.5-DEV (build 24) (C)2000-2005 -
 http://gpac.sourceforge.net
a=isma-compliance=1,1.0,1
a=mpeg4-iod:"data:application/mpeg4-iod;base64,AoJJAE8BAf/1AQOBVwACQLBkYXRhOmFwc
GxpY2F0aW9uL21wZWc0LW9kLWF1O2Jhc2U2NCxBV2NCWlFVZkEyRUF5U0FBQWdSSUlCRUFEOGNBQWlaW
UFBSEtPQVU1QUFBQnNQVUFBQUcxQ1FBQUFRQUFBQUVnQUlhRUFHY01EeEF3VVk4QUFBR3lSR2wyV0RrN
U9XSXdNREJ1QUFBQnNsaDJhVVF3TURJNUJoQUFKQUFCWDVBQUFBQUFJQUFBQUFRRAQPAQUAAGkAAAAAA
AAAAAUABhAA1AAAAlgAAAAAIAAAAAADA2YAAWA2ZGF0YTphcHBsaWNhdGlvbi9tcGVnNC1iaWZzLWF1O
2Jhc2U2NCx3QkFTWUVLQ0tDblFUd0E9AAIEFgINAAALAAAAAAAAAAAFBwAAcAeABgAGEADUAAACWAAAA
AAgAAAAAAM="
a=range:npt=0-  10.00000
m=video 0 RTP/AVP 96
b=AS:117
a=rtpmap:96 mpeg4-generic/90000
a=mpeg4-esid:201
a=control:trackID=65736
a=fmtp:96 profile-level-id=245; config=000001b0f5000001b509000001000000012000868
400670c0f1030518f000001b244697658393939623030306e000001b25876694430303239; strea
mType=4; mode=generic; objectType=32; DTSDeltaLength=14
a=framesize:96 120-96

test.3gp: Success – Video only

v=0
o=StreamingServer 3464000121 1214234605000 IN IP4 137.194.232.94
s=/test.3gp
u=http:///
e=admin@
c=IN IP4 0.0.0.0
b=AS:91
t=0 0
a=control:*
a=x-copyright: MP4/3GP File hinted with GPAC 0.4.3-DEV (C)2000-2005 - http://gpa
c.sourceforge.net
a=range:npt=0- 158.60000
m=video 0 RTP/AVP 96
b=AS:91
a=rtpmap:96 H263-1998/90000
a=control:trackID=65536
a=cliprect:0,0,96,128
a=framesize:96 128-96

GPAC MP4Streamer Details

SDP Multicast session

  • local: not reckognised
  • remote, served as text/plain: not supported
  • remote, served as application/sdp: recognized but cannot connect ? Multicast problem ?

SDP Multicast session (simplified):

HTTP or local files Details

Video 1

  • local: success
  • remote, served as text/plain: not supported
  • remote, served as video/mp4: cannot open this file ??