Ability to get AudioTracks and their qualities (Representation) and set the audio

[cristian-atehortua]

Have you read the FAQ and checked for duplicate open issues?
Yes, there is nothing related to this.

Is your feature request related to a problem? Please describe.

Currently, our dash manifests can have different AdaptationSet with the same languages and the same roles (or no roles). Those audio tracks are different, but ShakaPlayer is considering that they are the same when calling getAudioLanguagesAndRoles or getAudioLanguages. Therefore, when using those functions, the user is not able to see all options.

Additionally, our UI relies on some properties inside the Representation tag (the qualities of the AudioTrack) to display the name.

We try to achieve this by parsing the values returned by getVariantTracks and change the audio using selectVariantTrack but, as ShakaPlayer considers those audios the same, ABR will change the selected audio at any time.

Describe the solution you'd like

It would be nice if we can call a function, let's say getAudioTracks that returns the information of every AdaptationSet inside the manifest with a structure similar to this:

{
  id: 1 // A generated id to uniquely identify the AudioTrack (AdaptationSet)
  lang: 'es', // The value of the lang attribute of the AdaptationSet
  label: 'A label', // The content of the <Label></Label> tag inside the AdaptationSet
  roles: [], // A list of the <Role></Role> tags the AdaptationSet has
  qualities: [] // A list of objects with the information about the <Representation></Representation> tags inside the AdaptationSet
  // Other attributes extracted from AdaptationSet
}

Each object inside of qualities could have a structure similar to

{
  id: 'id', // The value of the id attribute of the Representation
  codec: 'ec-3', // The value of the codecs attribute of the Representation
  bandwidth: 192000, // The value of the bandwidth attribute of the Representation (The name could be bitrate as well)
  // Other attributes extracted from the Representation
}

It is important to provide a function to change the AudioTrack using the values returned for this hypothetical function. selectAudioTrack

Also, it is important to mention that to work properly, default ABR implementation should change a little bit to not consider those tracks as the same. In fact, with this change, the workaround I mentioned above will work.

[avelad]

This can already be done with the setting manifest.dash.enableAudioGroups = true. So I'm going to close the issue. Thanks!

[cristian-atehortua]

Hi @avelad. Thanks.

However, it's not clear to me how to get and select the audio tracks (functions getAudioTracks and selectAudioTracks as explained in the Feature Request). I don't see any new functions in the player for that.

Can you explain it to me?

[avelad]

getVariantTracks already has all the audio tracks, creating an API that filters getVariantTracks and is called getAudioTracks can be done on the application side, and the same thing happens with selectAudioTrack, we already have the selectVariantTrack function, if there is something they do wrong in selectVariantTrack or getVariantTracks functions should be reported as a bug. The functionality to get all the representations and it can be done. What you cannot do is activate the ABR and only choose a representation, for this to happen AdaptationSetCriteria must be met

[cristian-atehortua]

Well... Of course, everything can be done on the application side. For example, avoiding this behavior from Shaka to consider two languages as the same can be avoided by preprocessing the manifest and changing the language to something unique. In fact, we already have the implementation through the variant tracks and the preprocessing above.

However, this feature request is two things, the one that is already implemented through manifest.dash.enableAudioGroups and the other is these functions. I see the functions are still a good feature to be included (in combination with this flag) because getVariantTracks doesn't only return the audios but a combination of all video tracks with each audio track so it's not practical:

• to get the audio tracks applications still need to:

  • group by audio,
  • keep just one, and
  • remove all non-audio-related properties.

• to select an audio track:

  • check the current variant to get the videoId,
  • search a variant that matches the required audio and the current selected video

The problem gets bigger when an audio AdaptationSet has more than one Representation.

[avelad]

@cristian-atehortua I'll probably work on this in February :)