Distinguish between subtitle and caption tracks

[sarge]

As I understand it there is some work planned to look at subtitles, TTML and subtitle playlists. Dash itself has the concept of an adaption set for subtitles.

I was looking to add subtitles and potentially captions as well. While the implementation for each is essentially the same there is a semantic difference that I would like to capture.

I currently pass something like html5 tracks structure to our player and then inturn pass that to the shakaplayer.

tracks: [
          {
            kind: 'subtitles',
            src: '/subtitles-en.vtt',
            srclang: 'en'
          },
          {
            kind: 'captions',
            src: '/captions-en.vtt',
            srclang: 'en'
          }
        ]

The shakaplayer accepts a call like this

addExternalCaptions('/subtitles-en.vtt', 'en', 'text/vtt');
addExternalCaptions('/captions-en.vtt', 'en', 'text/vtt');

But once these are added there is no easy way to distinguish between them. As calling getTextTracks() will return

0: {
active: true,
enabled: false,
id: 2,
lang: "en"
},
1: {
active: false,
enabled: false,
id: 3,
lang: "en"
}

There is no information that indicates that one is a subtitle and the other is a caption. Even if I call addExternalCaptions with a mime type extension like 'text/vtt+caption' that information is not available.

Ultimately I would like to create a UI to allow the user to select a subtitle or a caption in their language.

I could manage the association between my tracks structure and the getTextTracks but there a couple of assumptions I need to make. Either I assume that the call to getTextTracks will return the results in the same order that I called addExternalCaptions which is probably a safe thing to do, or I take the index of my tracks list and add +2 before calling selectTextTrack(index + 2); which seems a little mysterious (I'm experienced in off by one errors but off by two are less common:)).

Perhaps the cleanest way would be to return the TextTrack object from the call to addExternalCaptions so I know which text track this refers to. Currently addExternalCaptions returns undefined.

An alternative would be to enrich the current text track model with a friendly name and perhaps even the kind into the call addExternalCaptions('/subtitles-en.vtt', 'en', 'text/vtt', 'English'); which can be queried via getTextTracks(). This better aligns with the HTML5 specification.

I hope that made sense

[joeyparrish]

Thanks, Mark. In the short term (v1.6.0), we could return the track object from addExternalCaptions. Would this work for now?

Long term, I'd like to make sure that a future redesign of Shaka Player provides some more elegant way to represent the semantics you're looking for. Can you help me understand the difference between captions and subtitles?

[sarge]

Sure that will work.

Captions are targeted at the hearing impaired. For example if you are hearing impaired, and an English speaker the captions will contain English dialogue but also environmental noises that might influence the actors. [bang!] to indicate a loud noise, [creaking floorboards] to add suspense, etc...

Subtitles are dialog translated to a different language, potentially you could have captions in different languages.

The rendering of each is identical, captions may use the location features of WebVTT to indicate a noise came from the left or right of the screen.

Not everyone will be interested in the distinction. But if you have ever watched a movie with closed captions when you wanted subtitles it can be very distracting. An example I watched recently was Deadlands in which the dialog is in Maori, which I don't speak but it included captions which wasn't ideal.

[joeyparrish]

Thanks for the explanation. We'll try to support that distinction better in v2.0.

I've taken another look at the code, and it actually will be difficult to return TextTrack the way it is currently implemented. Let me see if there is something else we can do short-term.

[joeyparrish]

@sarge, Shaka v2 should now distinguish between caption and subtitle tracks. Thanks for the report!

[sarge]

Thanks Joey!
On 22 Mar 2016 06:03, "Joey Parrish" [email protected] wrote:

@sarge https://github.com/sarge, Shaka v2 should now distinguish
between caption and subtitle tracks. Thanks for the report!

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#206 (comment)