Single license per content implementation #904

peaBerberian · 2021-02-17T18:01:56Z

Single license per content implementation

resolves #863

The need

What is this?

This PR implements a singleLicensePer option, which allows an application to tell the RxPlayer that the current content only has a single license for the whole content, even when it has multiple keys associated to different tracks or qualities.

The RxPlayer will then be able to perform a single license request and, if the right options are set, to fallback from non-decryptable contents, which will be both:

keys which have the wrong key-status ("output-restricted" if fallbackOn.keyOutputRestricted is set and "internal-error" if fallbackOn.keyInternalError is set).
keys which were not included in that license. Those will be fallbacked from, without needing to set any API.

The idea is also to be able to make evolutions to that API to implement for example a single license per Period in the future.

Why doing that?

The idea here is to perform a single license request, initially for a single quality, which will in response return a license (technically it could be multiple concatenated licenses in the CDM's point of view depending of the key system, but it is still a single request and data structure in the player's point of view) allowing to decrypt all encrypted media from that same content.

The advantages of doing that are multiple. Most of all:

it allows to only perform a single license request instead of multiple ones.
Reducing possible server loads but also reducing network latencies.
We could be able to tell much sooner which key are not supported and thus avoid switching to an undecipherable Representation to only later fallback from it.
some other players not being compatible with contents necessitating multiple license requests, putting all keys in a single license can improve compatibility for when streams are used with multiple players.
it reduces interactions with EME APIs, which we found to be very slow on some devices (we for example found out that generating a challenge can take more than 1 second on some embedded devices).

And surely many others.

Before that commit, the RxPlayer would already play those contents, but would still perform multiple license requests reducing the potential gains.

What it means technically

Technically, the description of this feature is simple:

When singleLicensePer is set to "content":

we should only perform a single license request, event when there's multiple initialization data encountered in the MPD and
initialization segments
we should fallback from Representations whose corresponding key ids are either:
1. Not signaled in the license
2. Signaled in the license but have to be fallbacked from according to the fallbackOn options

Implementation

Re-purposing `handledInitData`

To implement that, the main idea is to re-purpose the handledInitData cache (in the EMEManager), which stores initialization data that has already been encountered.

The only goal of that structure before being to avoid creating multiple MediaKeySessions for the same initialization data, it isn't a stretch to redefine it as a structure avoiding creating multiple MediaKeySessions linked to the same key ids.

I changed the way it is used in two ways:

When singleLicensePer is set to "content" we can simply check when it's empty to only create a session on the first
initialization data encountered (we can imagine also adding several IDs to that structure to easily implement "period" etc.).
As we still need to fallback from any initialization data that has no corresponding key in the license when the latter is loaded, I had to add to that structure an Observable emitting "blacklisted" key ids from the corresponding MediaKeySession and the other key ids, now considered by opposition as "whitelisted".

Simply said, every key ids linked to a MediaKeySession which are not present blacklisted (because of the usual fallbackOn options) will be whitelisted.

By listening to that Observable every time new initialization data is encountered, we can just compare the linked key id with whitelisted key ids that Observable emits - which will be once the license is pushed.
If the license has already been pushed, the Observable will emit immediately.

Adding the key id to `contentProtections$` event

Because we are using key ids here to detect which keys are not in a given license, we have to emit those alongside the initialization data to the EMEManager.
This is done through the already-existing contentProtections$ Observable.

In the hypothesis that no key id is anounced in the Manifest, we may thus have to parse initialization data to extract it from there, so we can support multi-key per license mode with such contents.
This is not done here for the moment, but I'm working on it now.

Does it work?

I tested it with success on:

Widevine L2 on STBs
Widevine L3 on Linux - Chrome
Widevine L3 on Linux - Firefox
Widevine L3 on Mac - Chrome
Widevine L3 on Windows - Edge
PlayReady SL3000 on Windows - Edge

I know it doesn't work yet on some set-top boxes, but this seems more an exception than the norm. We've still scheduled a meeting to learn more about this issue.
Depending on the result, we may want to perform updates on that code.

peaBerberian · 2021-02-17T18:03:27Z

This PR will also simplify the implementation of the following features:

be able to warn when the license fetched does not seem to have the expected key id
be able to switch a Representation from being "undecipherable" to being "decipherable"

src/core/eme/eme_manager.ts

peaBerberian · 2021-02-17T18:06:49Z

src/core/eme/eme_manager.ts

+                  lastKeyUpdate$.next(evt.value);
+                }
+              }),
+              catchError(err => {


From now on, t's just re-indentation, nothing to see here

peaBerberian · 2021-02-17T18:07:59Z

src/core/eme/session_events_listener.ts

@@ -322,8 +323,8 @@ function handleKeyStatusesChangeEvent(
      throw err;
    })
  );
-  return observableConcat(getKeyStatusesEvents(session, keySystemOptions, keySystem),
-                          callback$);
+  return observableMerge(getKeyStatusesEvents(session, keySystemOptions, keySystem),


Technically, there's no difference here as everything from the first Observable is synchronous.

But I felt this should be more of a merge.

src/manifest/representation.ts

src/core/eme/utils/loaded_sessions_store.ts

src/core/eme/eme_manager.ts

…-decipherable before This commit is based on the multiple-keys-per-license commits (in #904) and add on top of it a new behavior. Representations which have their key-id added to one of the `MediaKeyStatusMap` linked to the current content and which are not fallbacked from, will now have their `decipherable` property updated to `true` - even if it was set to `false` before (and thus even if we had fallbacked from it in the past). This development was made a lot easier by the work done to support the `singleLicensePer` option (#904) with now the concept of "whitelisted key ids", in opposition of the "blacklisted key ids" we fallback from. When a key id is found to be whitelisted (technically this means currently that its `MediaKeyStatus` is not either: `"internal-error"`, `"output-restricted"` or `"expired"`, the last one depending on `keySystems options) we will now: - update the related `Representation`'s `decipherable` property to `true` - regardless of its previous state - schedule a `decipherabilityUpdate` event with that update through the API (and through the `Manifest` object with the event of the same name) - In the `AdaptationStream` - only if the updates changed the list of available Representations for that Adaptation, re-construct the list of the Representations the ABR has to choose from --- To note that this has never been tested in real conditions.

grenault73 · 2021-02-23T16:41:15Z

src/core/eme/eme_manager.ts

+      if (options.singleLicensePer === "content" && handledSessions.getLength() > 0) {
+        const keyIds = initializationData.keyIds;
+        if (keyIds === undefined) {
+          log.warn("EME: Initialization data linked to unknown key id, we'll" +


It will log "...we'llnot.."

grenault73 · 2021-02-23T17:03:46Z

src/core/eme/eme_manager.ts

+      const lastKeyUpdate$ = new ReplaySubject<IKeyUpdateValue>(1);
+
+      // First, check that this initialization data is not already handled
+      if (options.singleLicensePer === "content" && handledSessions.getLength() > 0) {


Shouldn't we have two different objects ? handledSessions for singleLicensePer === "init-data" and another one for "content" ?

Wouldn't that just complexify the code?

Also, this code could be adaptable to a singleLicensePer: "period" by adding some metadata to both handledSessions and the initializationData sent to the EMEManager

It was just a little bit strange for me that the way to check if we already handled the init data was to see if there was at least one handled session.

I understand.

The fact is singleLicensePer: "content" also means a single MediaKeySession per content. Maybe I should define a isEmpty methode on handledSessions to make that more readable, and maybe I should find a better name that handledSessions, I probably should add a comment too.

Instead of handledSessions maybe , contentSessions?

This would make:

if (options.singleLicensePer === "content" && !contentSessions.isEmpty()) { // If only a single license is necessary per content, we only need to create // a single MediaKeySession for that content.

Yes, I had thought of a new method, without knowing exactly what it could be!

Thus, will you add a new method ?

Yes! Didn't take the time yet :/

lfaureyt · 2021-03-11T13:50:54Z

Widevine L1 on STBs

More likely Widevine L2 STBs (if you're referring to X1/X2 STBs aka "Cube C").

peaBerberian · 2021-03-11T18:09:55Z

Yes, I was talking about those and found out only after that they actually were L2

…-decipherable before This commit is based on the multiple-keys-per-license commits (in #904) and add on top of it a new behavior. Representations which have their key-id added to one of the `MediaKeyStatusMap` linked to the current content and which are not fallbacked from, will now have their `decipherable` property updated to `true` - even if it was set to `false` before (and thus even if we had fallbacked from it in the past). This development was made a lot easier by the work done to support the `singleLicensePer` option (#904) with now the concept of "whitelisted key ids", in opposition of the "blacklisted key ids" we fallback from. When a key id is found to be whitelisted (technically this means currently that its `MediaKeyStatus` is not either: `"internal-error"`, `"output-restricted"` or `"expired"`, the last one depending on `keySystems options) we will now: - update the related `Representation`'s `decipherable` property to `true` - regardless of its previous state - schedule a `decipherabilityUpdate` event with that update through the API (and through the `Manifest` object with the event of the same name) - In the `AdaptationStream` - only if the updates changed the list of available Representations for that Adaptation, re-construct the list of the Representations the ABR has to choose from --- To note that this has never been tested in real conditions.

## The need ### What is this? This commit implements a `singleLicensePer` option, which allows an application to tell the RxPlayer that the current content only has a single license for the whole content, even when it has multiple keys associated to different tracks or qualities. The RxPlayer will then be able to perform a single license request and, if the right options are set, to fallback from non-decryptable contents, which will be both: 1. keys which have the wrong key-status ("output-restricted" if `fallbackOn.keyOutputRestricted` is set and "internal-error" if `fallbackOn.keyInternalError` is set). 2. keys which were not included in that license. Those will be fallbacked from, without needing to set any API. The idea is also to be able to make evolutions to that API to implement for example a single license per Period in the future. ### Why doing that? The idea here is to perform a single license request, initially for a single quality, which will in response return a license (technically it could be multiple concatenated licenses in the CDM's point of view depending of the key system, but it is still a single request and data structure in the player's point of view) allowing to decrypt all encrypted media from that same content. The advantages of doing that are multiple. Most of all: 1. it allows to only perform a single license request instead of multiple ones. Reducing possible server loads but also reducing network latencies. 2. We could be able to tell much sooner which key are not supported and thus avoid switching to an undecipherable Representation to only later fallback from it. 3. some other players not being compatible with contents necessitating multiple license requests, putting all keys in a single license can improve compatibility for when streams are used with multiple players. 4. it reduces interactions with EME APIs, which we found to be very slow on some devices (we for example found out that generating a challenge can take more than 1 second on some embedded devices). And surely many others. Before that commit, the RxPlayer would already play those contents, but would still perform multiple license requests reducing the potential gains. ### What it means technically Technically, the description of this feature is simple: When `singleLicensePer` is set to `"content"`: - we should only perform a single license request, event when there's multiple initialization data encountered in the MPD and initialization segments - we should fallback from `Representation`s whose corresponding key ids are either: 1. Not signaled in the license 2. Signaled in the license but have to be fallbacked from according to the `fallbackOn` options ## Implementation ### Re-purposing `handledInitData` To implement that, the main idea is to re-purpose the `handledInitData` cache (in the `EMEManager`), which stores initialization data that has already been encountered. The only goal of that structure before being to avoid creating multiple MediaKeySessions for the same initialization data, it isn't a stretch to redefine it as a structure avoiding creating multiple MediaKeySessions linked to the same key ids. I changed the way it is used in two ways: 1. When `singleLicensePer` is set to `"content"` we can simply check when it's empty to only create a session on the first initialization data encountered (we can imagine also adding several IDs to that structure to easily implement `"period"` etc.). 2. As we still need to fallback from any initialization data that has no corresponding key in the license when the latter is loaded, I had to add to that structure an Observable emitting `"blacklisted"` key ids from the corresponding `MediaKeySession` and the other key ids, now considered by opposition as `"whitelisted"`. Simply said, every key ids linked to a MediaKeySession which are not present blacklisted (because of the usual `fallbackOn` options) will be whitelisted. By listening to that Observable everytime new initialization data is encountered, we can just compare the linked key id with whitelisted key ids that Observable emits - which will be once the license is pushed. If the license has already been pushed, the Observable will emit immediately. ### Adding the key id to `contentProtections$` event Because we are using key ids here to detect which keys are not in a given license, we have to emit those alongside the initialization data to the `EMEManager`. This is done through the already-existing `contentProtections$` Observable. In the hypothesis that no key id is anounced in the Manifest, we may thus have to parse initialization data to extract it from there, so we can support multi-key per license mode with such contents. This is not done here for the moment, but I'm working on it now. In the meantime, initialization data for which the corresponding key id is not known will still lead to another license request, which I felt was a good compromise. ## Does it work? I tested it with success on: - Widevine L1 on STBs - Widevine L3 on Linux - Chrome - Widevine L3 on Linux - Firefox - Widevine L3 on Mac - Chrome - Widevine L3 on Windows - Edge - PlayReady SL3000 on Windows - Edge I know it doesn't work yet on some set-top boxes, but this seems more an exception than the norm. We've still scheduled a meeting to learn more about this issue. Depending on the result, we may want to perform updates on that code.

…d-less init data

…e conditions

Before this commit, persistent MediaKeySessions were added to the `PersistentSessionsStore` (for later retrieval) as soon as possible: when their `sessionId` property was known (meaning as soon as the `generateRequest` answered). This worked pretty well, but we found a persistence-related issue that could profit from taking another strategy: After loading a persistent MediaKeySessions (that has previously been added to the `PersistentSessionsStore`), the RxPlayer immediately checks the status of its keys through the `keyStatuses` property. However, on some platforms, the `keyStatuses` property is not directly populated once the `load` call is done, but only after a difficult-to-predict delay (sometimes immediately after, sometimes more than 10 milliseconds after). This imply that we might now have to wait after loading a persistent session just to be sure that all keys information have been added to it. We limited that by only waiting a delay if the `keyStatuses` property is empty, yet this pre-condition was frequent: any time the user loaded a new content before the license request succeeded (but after the `generateRequest` call) we would be legitimately in a case where the `keyStatuses` property is actually empty. To avoid penalizing too much legitimate cases, this commit adds a persistent session to the `PersistentSessionsStore` only once at least one key is added to its `keyStatuses` property. This ensures that having an empty `keyStatuses` property will now be unusual enough. In that case, incurring a delay (let's say up to 100ms) will be much less penalizing.

…update eme: only persist MediaKeySessions once its keys are known

…d by a single session in content mode

Single license per content implementation

…-decipherable before This commit is based on the multiple-keys-per-license commits (in #904) and add on top of it a new behavior. Representations which have their key-id added to one of the `MediaKeyStatusMap` linked to the current content and which are not fallbacked from, will now have their `decipherable` property updated to `true` - even if it was set to `false` before (and thus even if we had fallbacked from it in the past). This development was made a lot easier by the work done to support the `singleLicensePer` option (#904) with now the concept of "whitelisted key ids", in opposition of the "blacklisted key ids" we fallback from. When a key id is found to be whitelisted (technically this means currently that its `MediaKeyStatus` is not either: `"internal-error"`, `"output-restricted"` or `"expired"`, the last one depending on `keySystems options) we will now: - update the related `Representation`'s `decipherable` property to `true` - regardless of its previous state - schedule a `decipherabilityUpdate` event with that update through the API (and through the `Manifest` object with the event of the same name) - In the `AdaptationStream` - only if the updates changed the list of available Representations for that Adaptation, re-construct the list of the Representations the ABR has to choose from --- To note that this has never been tested in real conditions.

Single license per content implementation

peaBerberian added this to the 3.24.0 milestone Feb 17, 2021

peaBerberian commented Feb 17, 2021

View reviewed changes

src/core/eme/eme_manager.ts Outdated Show resolved Hide resolved

peaBerberian commented Feb 17, 2021

View reviewed changes

src/manifest/representation.ts Outdated Show resolved Hide resolved

peaBerberian commented Feb 17, 2021

View reviewed changes

src/core/eme/utils/loaded_sessions_store.ts Outdated Show resolved Hide resolved

peaBerberian added work-in-progress This Pull Request or issue is not finished yet and removed Ready for Review The Pull Request is in its final form and ready to be reviewed by someone labels Feb 17, 2021

peaBerberian commented Feb 17, 2021

View reviewed changes

src/core/eme/eme_manager.ts Outdated Show resolved Hide resolved

peaBerberian added Ready for Review The Pull Request is in its final form and ready to be reviewed by someone and removed work-in-progress This Pull Request or issue is not finished yet labels Feb 18, 2021

peaBerberian mentioned this pull request Feb 18, 2021

Mark decipherable Representations as decipherable even if they were undecipherable before #905

Merged

grenault73 reviewed Feb 23, 2021

View reviewed changes

peaBerberian force-pushed the feat/singleLicensePer branch from 7f1058c to b6be79d Compare March 19, 2021 17:37

peaBerberian mentioned this pull request Mar 19, 2021

eme: only persist MediaKeySessions once its keys are known #926

Merged

peaBerberian force-pushed the feat/singleLicensePer branch from b6be79d to 54062a3 Compare March 23, 2021 13:41

peaBerberian mentioned this pull request Mar 24, 2021

release/v3.24.0 #930

Merged

28 tasks

grenault73 added the Needs discussion This PR / Issue needs to be discussed internally by all RxPlayer maintainers label Mar 26, 2021

peaBerberian force-pushed the feat/singleLicensePer branch from 9395439 to 82a3263 Compare March 29, 2021 15:31

grenault73 approved these changes Mar 30, 2021

View reviewed changes

peaBerberian added 12 commits March 30, 2021 12:27

eme: in singleLicensePerContent mode do not another request for key-i…

f2682c2

…d-less init data

EME: rename LoadedSessionsStore.moveOnTop to LoadedSessionsStore.reuse

d14b82c

eme: update outdated JSDoc in loaded_sessions_store.ts

4709ca9

eme: add forgotten space in log in eme_manager.ts

f49430f

eme: rename variable and add isEmpty method to improve readability

1963153

eme: fix global eme tests by sending key updates after the update

39271b1

tests: raise-up delay for a key update to work-around subtle test rac…

5a227b4

…e conditions

Merge pull request #926 from canalplus/misc/add-persistent-after-key-…

6180d66

…update eme: only persist MediaKeySessions once its keys are known

eme: check that all keyIds for the current encrypted event are handle…

2a2aaae

…d by a single session in content mode

doc/eme: update JSDoc in EME-related code

bfe2826

peaBerberian force-pushed the feat/singleLicensePer branch from 82a3263 to bfe2826 Compare March 30, 2021 11:04

grenault73 approved these changes Mar 30, 2021

View reviewed changes

peaBerberian merged commit c485f5f into next Mar 30, 2021

peaBerberian added a commit that referenced this pull request Mar 30, 2021

Merge pull request #904 from canalplus/feat/singleLicensePer

8e4c054

Single license per content implementation

peaBerberian mentioned this pull request Mar 30, 2021

Multiple keys per license implementation #863

Closed

peaBerberian added a commit that referenced this pull request Mar 30, 2021

Merge pull request #904 from canalplus/feat/singleLicensePer

24aafda

Single license per content implementation

peaBerberian removed the Needs discussion This PR / Issue needs to be discussed internally by all RxPlayer maintainers label May 25, 2021

peaBerberian deleted the feat/singleLicensePer branch June 16, 2021 13:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Single license per content implementation #904

Single license per content implementation #904

peaBerberian commented Feb 17, 2021 •

edited

Loading

peaBerberian commented Feb 17, 2021

peaBerberian Feb 17, 2021

peaBerberian Feb 17, 2021 •

edited

Loading

grenault73 Feb 23, 2021

peaBerberian Feb 24, 2021

grenault73 Feb 23, 2021

peaBerberian Feb 23, 2021

grenault73 Feb 24, 2021

peaBerberian Feb 25, 2021

grenault73 Feb 25, 2021

grenault73 Mar 3, 2021

peaBerberian Mar 3, 2021

peaBerberian Mar 5, 2021

lfaureyt commented Mar 11, 2021

peaBerberian commented Mar 11, 2021

Single license per content implementation #904

Single license per content implementation #904

Conversation

peaBerberian commented Feb 17, 2021 • edited Loading

Single license per content implementation

The need

What is this?

Why doing that?

What it means technically

Implementation

Re-purposing handledInitData

Adding the key id to contentProtections$ event

Does it work?

peaBerberian commented Feb 17, 2021

Choose a reason for hiding this comment

peaBerberian Feb 17, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lfaureyt commented Mar 11, 2021

peaBerberian commented Mar 11, 2021

peaBerberian commented Feb 17, 2021 •

edited

Loading

Re-purposing `handledInitData`

Adding the key id to `contentProtections$` event

peaBerberian Feb 17, 2021 •

edited

Loading