How to listen for ID3 metadata using flowplayer

[stephendeyoung]

Using flowplayer I want to listen for ID3 metadata when it's embedded into the stream. I found this bit of ActionScript code that dispatches an event to the clip in flowplayer:

2cd5287#diff-c64e94b06201787635a3704e6e6cdcc2R96

How would I listen for ID3 events in flowplayer?

[phloxic]

You can access the clip's metadata by grabbing and evaluating the clip's metaData object from any clip event - http://flash.flowplayer.org/documentation/api/clip.html#properties
The onMetaDataChange clip event might also be helpful - http://flash.flowplayer.org/documentation/events/clip.html
Disclaimer: I have not experience re: ID3 events in particular.

[stephendeyoung]

I've tried onMetaDataChange but it doesn't fire on an ID3 event.

[mangui]

Hi @stephendeyoung , I would first advise you to test your stream with either chromelessplayer or bin/debug/flashlsFlowplayer.swf and enable debug logs and see if you can get any logs related to ID3 tags on the JS console : the following line should output something :
https://github.com/mangui/flashls/blob/dev/src/org/mangui/hls/stream/HLSNetStream.as#L94-L96
Cheers,
Mangui

[dusterio]

@mangui I'm segmenting using OS X's mediafilesegmenter, and then checking debug log in JS console – don't see onID3Data() being triggered :(

Has anybody tested ID3 functionality of flashls?

[mangui]

hi @dusterio
ID3 is an external contribution, not sure it has been validated/used with apple tool.
have you enable debug/debug2 logs ?
if ID3 PID is found in TS you should also see some debug logs printed during parsing :

https://github.com/mangui/flashls/blob/dev/src/org/mangui/hls/demux/TSDemuxer.as#L641-L674

plz share a log including debug/debug2
if you don't get any id3 match, plz share a link to your stream

Cheers
Mangui

[dusterio]

@mangui yes, I'm following debug/debug2 logs, there is only one mentioning of ID3:
VM196:1 DEBUG:Loading duration/RTT/length/speed:39/23/569076/113998 kb/s
VM197:1 DEBUG:TS: PAT found.PMT PID:256
VM198:1 DEBUG:TS: PMT found
VM199:1 DEBUG:TS: Selected video PID: 257
VM200:1 DEBUG:TS: Selected ID3 PID: 259
VM201:1 DEBUG:TS: Found 1 audio tracks

I don't see onID3Data() triggered anywhere, there are no errors mentioning ID3 either.

[mangui]

ok are you able to get metadata displayed on iOS ?
if yes plz share a fragment (just one is enough) holding some metadata on PID 259
Cheers,
Mangui

[dusterio]

Sorry for the late reply – I've been experimenting with different approaches.

I have this demo stream (VOD): http://demo.rapport.fm/test/prog_index.m3u8
It's created using Apple mediafilesegmenter, with several ID3 meta tags embedded.

I can play it fine in iOS and I can see/parse ID3 metadata. However, Flashls just halts (stops playback) once it reaches the first ID3 tag at 10 seconds. I checked debug logs – couldn't see anything special.

[mangui]

I got a bunch of parsing warning msg with your stream but it is playing fine and I can see logs with ID3 tags with latest flashls release (0.4.0.x)

[dusterio]

@mangui oh yes, I see that you have updated Flashls – I was using a 1-2 month old package. I tried playing the same stream directly from your official page: http://www.flashls.org/latest/examples/chromeless/, the stream plays fine but regarding the ID3s – I only see warnings:

[Log] DEBUG:TS: Selected ID3 PID: 259
[Log] DEBUG:TS: complete ID3 PES found, parse it
[Log] WARN:TS: Discarding ID3 packet with id 259 bad TS segmentation ?
[Log] DEBUG:id3:4944330400000000002a575858580000000f00000300687474703a2f2f7777772e72755450453100000007000003537061696e00

How do I get access to contents of the ID3 packet though?

[dusterio]

@mangui since Apple HLS tools work only under MAC OS and nobody ever made an open source analogue, I had to write my own: https://github.com/dusterio/hlsinjector

I'm creating Apple-compliant TS stream with metadata, it plays well in iOS.

However, Flashls still issues the warning:
[Log] WARN:TS: Discarding ID3 packet with id 259 bad TS segmentation ?

What's exactly meant by bad TS segmentation here? I cross-checked my stream several times – everything is accurate.

[mangui]

it means that at the beginning of your ts packet, there is some data found on ID3 pid before any payload start has been found on this pid

[dusterio]

@mangui well, I followed the format of Apple HLS tools. At this moment I'm going through Flashls source code – I will try to find the cause and revert here. PS. iOS and Safari desktop play the very same stream fine.

[mangui]

you need to check the header of your first Id3 TS packet.

[mangui]

it might not have a payload start unit indicator

[dusterio]

@mangui the indicator is set to 1 and there is only one TS packet because the whole ID3 content fits in one frame.

These are my TS & PES headers, following Apple standard:
8 bits: Sync byte
1 bit: Transport Error Indicator = 0
1 bit: Payload Unit Start Indicator = 1
1 bit: Transport Priority = 0
13 bits: PID (dec: 259 hex: 0x103)
2 bits: Scrambling control = 00
1 bit: Adaptation field exists = 0
1 bit: Payload exists = 1
4 bits: Continuity counter = 0000 (dec: 0)
==== PES HEADER ====
24 bits: Packet start prefix: OK
8 bits: Stream ID = 13 (dec) / 0xd (hex)
16 bits: PES packet length = 178 (bytes)
==== OPTIONAL PES HEADER ====
2 bits: Optional header marker
2 bits: Scrambling control = 00
1 bit: Priority = 0
1 bit: Data alignment indicator = 1
1 bit: Copyright = 0
1 bit: Original or copy = 0
2 bits: PTS & DTS indicator = 10
1 bit: ESCR flag = 0
1 bit: ES rate flag = 0
1 bit: DSM trick mode flag = 0
1 bit: Copy info flag = 0
1 bit: CRC flag = 0
1 bit: Extension flag = 0
8 bits: PES header length = 123 (bytes)
After this: PTS + filling bytes + content

Once again, iOS and Safari play it fine, and meta data is accessible. Therefore I tend to conclude there is a "bug" (more of a misinterpretation probably) in Flashls code.

[mangui]

strange. if all packets are like this one this should not happen. could u share a playlist with which i could repro ?

[dusterio]

@mangui Sure: http://demo.rapport.fm/test2.m3u8
There are ID3 tags at 10 and 30 seconds
By the way, this entry in the debug log:
DEBUG:id3:4944330400000000002a575858580000000f00000300687474703a2f2f7777772e72755450453100000007000003537061696e00

This is actually the full, correct ID3 tag

[dusterio]

Some further research:

If debug2 is on, I can see two lines in a row that shouldn't actually go together:
DEBUG:TS: complete ID3 PES found, parse it
VM3927:1 WARN:TS: Discarding ID3 packet with id 258 bad TS segmentation ?

Going to TSDemuxer.as, lines 699 and on:
CONFIG::LOGGING {
Log.debug2("TS: complete ID3 PES found, parse it");
}
// complete PES, parse and push into the queue
_parseID3PES(pes);
_curId3PES = null;

And then further from lines 712:
CONFIG::LOGGING {
if (!_curId3PES) {
Log.warn("TS: Discarding ID3 packet with id " + pid + " bad TS segmentation ?");
}
}

While I don't have any experience with ActionScript, it does look like in case of full, complete ID3 packet _curId3PES will be set to null and then later on a warning will be issued (like if there was no _curId3PES). So a warning is actually guaranteed!

If I'm right, this warning can simply be ignored.

I don't see ID3 events in ChromelessPlayer.as, so, I suppose, Javascript is not notified of incoming metadata at the moment. Do you plan to add it soon?

[mangui]

Hi @dusterio , indeed in your case this warning could be ignored. I will fix this spurious warning

[dusterio]

@mangui thanks, that was quick!

[stephendeyoung]

So with regards to my original issue is it possible to listen to timed metadata events using flowplayer? I can catch events using the onNetStreamEvent handler but this only works in browsers that can use flash. On iOS devices I'm not able to catch these events at all.

[dusterio]

@stephendeyoung as far as I understand, that's not related to Flashls/flowplayer anymore. if you play it on device that "naturally" plays HLS (Safari desktop, iOS, etc) – I don't think you will be able to catch the event. For iOS, I would suggest to implement it in an app using MPMoviePlayer or AVPlayer, then you can listen for ID3 metadata... Playing in the browser natively – I seriously doubt...

[stephendeyoung]

@dusterio It used to be possible to do this by embedding a Quicktime object and then using the JS API for Quicktime to listen for timed metadata events:

http://www.wowza.com/forums/content.php?355-How-to-debug-timed-data-events-%28ID3-tags%29-from-Apple-HLS-streams-in-iOS-devices

Unfortunately in iOS8 the Quicktime JS API no longer works so we were hoping we might be able to use flowplayer and the flashls plugin to catch the metadata events.

[dusterio]

@stephendeyoung please let me know if you find a solution ;-) I would be very happy to be able to parse metadata in browsers under iOS

[stephendeyoung]

@dusterio I've asked a question on Stack Overflow about this:

http://stackoverflow.com/questions/29772013/http-live-streaming-how-to-listen-for-timed-metadata-embedded-as-id3-tags-using

That explains a bit more about some of the information I've found.

[stephendeyoung]

@dusterio I've found a solution to this now. You can use HTML5 "in-band metadata text tracks" to catch the metadata:

http://stackoverflow.com/a/31433502/776472

I don't know if this could be wired into the flashls plugin in some way so that you can use the standard Flowplayer onMetaData event handler?

[dusterio]

@stephendeyoung wow interesting! thanks a lot for your research, I'm going to play with this functionality now, too :)

as far as I understand, you cannot do it in Flashls – simply because it's Flash. You have to do it somewhere beforehand, in JS – eg. Flowplayer or your own JS.