Returning to ABR mode

[BucherTomas]

There seems to be an issue with re-enabling adaptation once user switches to a given quality manually and then attempts to switch back to auto.

1. Start playing any content with multiple qualities.
2. Select some lower quality manually before the player adapts automatically to the highest possible one
3. Re-enable adaptation

Alternatively start with "enable adaptation" checkbox disabled and enable during playback.

Expected behavior would then be that the player enables ABR again and adapts in a short while to the highest possible quality based on network conditions. But this does not happen, playback is stuck on the manually chosen quality.

We attempted to call in our player player.configure({ abr: { enabled: true } }); , hopefully the correct approach, but no difference from the reference Shaka player, ABR does not kick in.

[joeyparrish]

I'm unable to reproduce on the nightly build: http://nightly.shaka-player-demo.appspot.com/demo/

I started with adaptation disabled via the UI. Then I started "Sintel 4k (multicodec)". The resolution was 426x182. I let it play for a bit, then enabled adaptation via the UI. It switched resolutions several times after that in response to bandwidth estimates, hitting both 720p and 1080p.

I also tried the other way. I started with adaptation enabled via the UI. Then I started "Sintel 4k (multicodec)". The resolution adapted up to 1080p. I then disabled adaptation by selecting 256x110 resolution. Then I re-enabled adaptation via the UI. It eventually switched resolution again, hitting both 720p and 1080p.

The problem might be here:

Expected behavior would then be that the player enables ABR again and adapts in a short while

Can you retest with a clip of at least several minutes and wait longer this time? It could be that our default ABR manager is simply slower than you expected in certain conditions. If so, we'll rename this issue and discuss ways we could improve responsiveness.

[BucherTomas]

I now tested on Chrome with nightly with adaptation disabled first and both Sintel 4k (multicodec) and Tears of Steel. Strangely enough, Sintel 4k starts for me in 426x182 (not the lowest resolution). Tears of Steel starts properly in lowest 220x124 resolution. In both cases I've been waiting for 5-7 minutes after enabling adaptation again, but it didn't switch to any higher resolution.

It took about 50-100 ms to get larger video segments for Tears of Steel, while it took about 200 ms to get video segments for Sintel 4k. Difference caused likely due to me being in Europe. When I started throttling my current speed 100 Mbps to 20 Mbps and less, so it took the video segments about 500 ms and more to arrive, adaptation started to work after a while. No exact measurement, but subjectively it felt like the more I throttled the bandwith, the faster it switched to next available resolution from the initial resolution I started with.

How long does it take on average in your case to get the video segments? There is likely a connection.

[joeyparrish]

Interesting. What Shaka version are you using?

[BucherTomas]

These last tests were done with the nightly version as you suggested.
Our player behaving the same way is built from Shaka master from two
days ago.

On 1.7.2016 18:21, Joey Parrish wrote:

Interesting. What Shaka version are you using?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#435 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ARlzQScghvikEYPbGyzrIvgudJ13hdRrks5qRT6JgaJpZM4JDJSH.

[joeyparrish]

It seems likely that there is some issue in shaka.abr.EwmaBandwidthEstimator. Can you try changing some of the constants in that file to see if you get better results?

In particular, minDelayMs_ and minBytes_ affect which samples are used in the estimate. Initial track selection is affected by defaultEstimate_ and minWeight_, which in turn affects the size of segments downloaded and therefore which samples are used. So all four of those are worth tinkering with.

I'm sorry that we don't have the environment necessary to reproduce this issue, but if you can find a change to those parameters that helps in your case, we can work from there to analyze the situation in a theoretical way.

[BucherTomas]

None of the value updates in ewma_bandwidth_estimator.js made any relevant change, unfortunately. When debugging with Chrome dev tools, I could see my custom values being used, but it didn't help. Still stuck on the first selected resolution.

I did find something worth reporting, though, when testing with Tears of Steel from Unified Streaming with adaptation disabled first.
Placing a breakpoint on line 310 in simple_abr_manager.js while video was already running and enabling adaptation shortly after proved by reading value of currentBandwidth that the bandwidth estimation works as it was providing good enough value for the highest quality to be enabled. Value of "chosen" variable on line 311 was still at the lowest bitrate, though. Also when running shakaDemo.player_.getTracks() in browser console, it reported that the lowest bitrate is currently active. So far it was consistent.
However, reading value of streamsByType_['video'] on line 206 for variable oldVideo during the same debugging session provided video object with bitrate of the highest quality.
Could it be that abr manager thinks that the highest quality is already enabled and therefore does not trigger any bitrate switching?

Not sure where bandwidth throttling comes into play and why it helps.

[Ross-cz]

I can confirm same like Tomáš. I saw that a month ago using same debugging session. At some circumstances AbrManager caches incorrect (highest) values, gets stuck with them and don't process switch call - probably some initialization race condition.

My test case:

1. expand Configuration and disable adaptation
2. on shaka test page select "Tears of Steel"
3. click Load
4. cca after 5s enable adaptation
5. even after 2 minutes nothing adapted...

I was able to repro it at work & home using Chrome 51.0.2704.103 m.

Check race condition status using console:

1. abr manager
   shakaDemo.player_.defaultAbrManager_.streamsByType_['video'].height
   shakaDemo.player_.defaultAbrManager_.chooseStreams_().video.height
   => both 944
2. player switch history
   shakaDemo.player_.switchHistory_.filter(function (a) {return a.type == 'video';}).map(function(a) {return shakaDemo.player_.getTracks().filter(function(b) {return b.id == a.id;})[0].height;})
   => [124]
3. abr manager has strange comparisons inside method suggestStreams_ aka related code
   var oldAudio = this.streamsByType_['audio'];
   var oldVideo = this.streamsByType_['video'];
   var chosen = this.chooseStreams_();
   if (chosen['audio'] != oldAudio || chosen['video'] != oldVideo) {
   var currentBandwidthKbps =
   Math.round(this.bandwidthEstimator_.getBandwidthEstimate() / 1000.0);
   shaka.log.debug(
   'Calling switch_()...',
   'bandwidth=' + currentBandwidthKbps + ' kbps');
   this.switch_(chosen);
   }

Why are there cached values inside managers and not reading actual states?

[joeyparrish]

The value of "chosen" at line 311 would be the lowest bitrate stream on the first pass through the loop. The loop should continue, though, and look at all video streams in order of bandwidth from lowest to highest. @BucherTomas, does this happen in your case?

The most recently chosen streams are cached so that the "switch" callback only happens when there are actual changes necessary. segmentDownloaded() calls suggestStreams_. suggestStreams_ calls chooseStreams_ to make a choice, then compares that choice to the cached values before calling switch_. That's the only purpose of those cached values.

One thing that may be confusing the issue is that when we change bitrates, we don't currently clear the buffer at all. So if you've buffered ahead 30 seconds of content at 240p, you won't see 1080p content in the buffer for 30 more seconds. The streams chosen by SimpleAbrManager would be 1080p, streamsByType_ would show the 1080p stream, and the "active" track would be the 1080p track, but the content on screen at that moment would still be 240p.

We've been putting off changes to allow the AbrManager to clear the buffer when switching. Assuming there's no race, that change would make upgrades take effect more quickly.

Of course, there may also be a race somewhere.

To start, I will try to reduce the effect of buffering, because that is the simplest thing we can do. After that, I will see if I can reproduce the race described by @Ross-cz.

[BucherTomas]

Tested again with Tears of Steel on nightly with 6daa7f3 commit included.

1. Starting with Adaptation disabled, value of "Chosen" (due to the update now on line 330) reads 405000 bits/s - the lowest bitrate. Breakpoint is still enabled.
2. I left it running for 20 seconds, so that the bandwidth estimator had enough time to gather information and then enabled adaptation. Breakpoint is reached again and "Chosen" now reads already the highest bitrate 2997000 bits/s.
3. Based on shaka.abr.SimpleAbrManager.SWITCH_INTERVAL_MS, each 8 seconds the breakpoint was reached again and the value was still the highest bitrate. I quickly pressed F8 to continue, and did so continually until playhead was at 3 minutes. Video was still playing at the lowest bitrate in the player up to this point, no bitrate switching took place as opposed to a stream with adaptation enabled from the start where switching to the chosen bitrate occurs within a couple of seconds.

If I repeat the same procedure with adaptation disabled first and bandwidth throttled by using "Good 3G" preset in Chrome dev tools, chosen bitrate in the second step is 814000 - second lowest bitrate and the player switches to it at the 1 min 20 seconds mark. Return to ABR mode works reasonably well in this low bandwidth scenario.

The new feature of clearing the buffer does not tackle the issue I am experiencing, but thank you for it nonetheless. I would have suggested it in the future likely anyway, we were using in our player bufferingGoal of 10 seconds instead of default 30 to shorten the switch time between chosen and displayed track and this should make it redundant.

[joeyparrish]

I've got clear repro on this now. Definitely a bug. The key is enabling adaptation after the manager has enough data to justify a switch.

As soon as the config changes, shaka.Player.prototype.applyConfig_ asks SimplerAbrManager for new streams. But applyConfig_ is filtering that set of streams for language changes and only sending those changes to StreamingEngine.

After that, there's a mismatch between what SimpleAbrManager thinks it asked for and what has actually taken effect. SimpleAbrManager doesn't call switch_() because it already asked for those streams, even though the initial request was filtered out.

Not sure yet if the fix will be to Player or SimpleAbrManager, but we now understand the problem.

[joeyparrish]

Also, thanks to both @BucherTomas and @Ross-cz for the detailed reports and for helping to debug. Much obliged.

[joeyparrish]

@BucherTomas and @Ross-cz, I believe we have this fixed now in master. Can you please retest and confirm?

[BucherTomas]

Yes, from what I can see, I also believe that this is now fixed. Thank you.

[joeyparrish]

Excellent. Thanks!