MPEG-TS subtitles

[balderys]

Hi @mangui,

is it possible to handle MPEG-TS subtitles (aka DVB/DVD subtitles) because I guess it is not supported by flash/OSMF itself? Note that I'm not talking about WebVTT subtitles added to HLS draft.

[mangui]

indeed it is not supported, although it would be feasible to retrieve subtitle PES packets and parse them, then notify events (with timestamps + subtitle bitmap / text) so that the player can render them on screen appropriately.

[balderys]

Thank you @mangui. Do you think it could be implemented in certain time?

[mangui]

I have no plan to implement this as I won't use such features, but anybody can volunteer. or eventually sponsorship might be an option.

[balderys]

Ok, what is your price expectation?

[mangui]

i was not thinking of me particularly ... looking at DVB subtitle decoder implementation, see https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/dvbsubdec.c for example, it seems quite complex ... i am not sure it is really the right way to go for HLS ...

for subtitling you can use timed metadata [https://developer.apple.com/library/ios/documentation/AudioVideo/Conceptual/HTTP_Live_Streaming_Metadata_Spec/Introduction/Introduction.html] , it is already implemented, flashls will fire events each time a new metadata comes.

[jlacivita]

most production-level end-user facing live HLS streams w/ captions use the binary MPEG2 TS subtitles. pre-recorded / VOD HLS streams can use WebVTT, but for live, it's not realistic since all of the live encoders out there just spit out binary streams.

Ideally flashls would parse those packets and surface them, just like the video tag does.

[jlacivita]

Hmm, i might be mistaken... I was under the impression this issue was asking the same thing as:
#11

which is 608/708 CCs.

[jlacivita]

@balderys: Turns out what i want is 608/708 CCs. They are embedded in the MPEG2 TS fragments, though. I'm not sure what you're asking about, but it could be the same thing or not? @mangui, do you know if they're the same?

If so, see my comment in #11, which suggests that you don't need to do the parsing of the 608/708 data since there are other plug-ins that do so.

[balderys]

@jlacivita 608CC are little bit different in comparison with standard full subtitles. I'm not sure how the 608CC are represented in TS but I guess it is the same format only in different PID.

[triwav]

The way I understand it, 608 captions are actually within the TS file itself whereas webvtt which seems to the alternative approach by Apple is a separate playlist referenced in the master playlist that is split up into chunks and is read in parallel. I think webvtt would be the easier route considering there aren't many open source tools that handle 608.

[jlacivita]

WebVTT is certainly easier, but most live encoders output 608 today.

I think flashls only needs to dispatch the onCaptionInfo events with the binary 608 info. There's no need for flashls to decode/render 608. That's something OSMF of jwPlayer should handle.

[kfeinUI]

Yes, this is a duplicate of #11 . 608/708 support for firing the onCaptionInfo events is the gap. Adobe packages a captioning lib for OSMF with its Media Server product. Sure it would make more sense to include such a valuable assistive technology with OSMF, but they instead are keeping it closed source and separate. Easy enough to work with at least as there is not a lot to their public interface. I wrote a simple OSMF plugin wrapper with JavaScript bridge (will see if I can share it).

Anyway...as mentioned previously, this feature is quite valuable within the industry. Essential for live events since this data gets passed through the encoder and streamed. This does work with HLS on Apple's players. Not to be confused with the HLS support for WebVTT which may be streamed, but doesn't really have the encoder support.