Keep track of approximate total bytes in SourceBuffer / Prevent quota exceeded with 4K streams #1242

tchakabam · 2017-08-25T11:07:16Z

Description

// With high-bitrate streams (like 4K etc, up to 16-20 Mbit/s)
// we can reach a QuotaExceeded error easily within just 60 seconds
// of buffer. Therefore we need to actively check the exact number
// of bytes occupied in the buffer.
// If we reach the limit, we are triming pre-emptively, enqueuing this action
// to the source-updater before any further loading can happen.
// Even if the back-buffer is alrady trimmed, we should wait before
// we append more data if the buffer is still too occupied.
//
// We monitor the bytes in buffer (see checkBuffer_())
// and drive the loading / triming actively
// to make sure we do not reach this limit.

Specific Changes proposed

add method totalBytesInBuffer to SourceBuffer
modifiy slightly SourceBuffer interface to pass duration
use above method to limit loading in SegmentLoader when limit is exceeded
move significant constants in Config module
trimBackBuffer_ improvements

Requirements Checklist

Feature implemented / Bug fixed
If necessary, more likely in a feature request than a bug fix
- Unit Tests updated or fixed
- Docs/guides updated
- Example created (starter template on JSBin)
Reviewed by Two Core Contributors

…bytes in buffer

…n-bytes

…occupation

…Errors properly

gesinger

Hey @tchakabam , sorry this has gone a while without any review. We definitely need to handle this case, and thank you for submitting the PR!

The main thing I wanted to hear your thoughts on is making use of the segment metadata track (and adding byte info to that) to calculate buffer occupancy on demand instead of maintaining more state in the source updater.

Thanks again!

gesinger · 2018-01-18T21:56:58Z

src/segment-loader.js

+    // we can reach a QuotaExceeded error easily within just 60 seconds
+    // of buffer. Therefore we need to actively check the exact number
+    // of bytes occupied in the buffer.
+    // If we reach the limit, we are triming pre-emptively, enqueuing this action


we are triming pre-emptively, enqueueing => we trim preemptively, and enqueue

src/segment-loader.js

gesinger · 2018-01-18T21:58:15Z

src/segment-loader.js

+    // and drive the loading / triming actively
+    // to make sure we do not reach this limit.
+    if (bufferedBytes >= Config.MAX_SOURCE_BUFFER_OCCUPATION_BYTES) {
+      // If we reached the limit let's trim the buffer already


If we...jamming situation I think we can remove, since the comment above states the intention well

gesinger · 2018-01-18T21:58:39Z

src/segment-loader.js

+      // of jamming situation.
+      // We keep some back-buffer but reduce the allowed size significantly
+      // compared to the initial configuration.
+      this.logger_('SourceBuffer occupation exceeds limit, triming back-buffer and deferring further loading');


gesinger · 2018-01-18T21:59:34Z

src/segment-loader.js

+      // We keep some back-buffer but reduce the allowed size significantly
+      // compared to the initial configuration.
+      this.logger_('SourceBuffer occupation exceeds limit, triming back-buffer and deferring further loading');
+      this.trimBackBuffer_(Config.BACK_BUFFER_LENGTH / 3);


Any specific reason for the / 3? Maybe we can calculate how much buffer we need to trim judging by segment starts/ends and their respective sizes?

gesinger · 2018-01-18T22:00:03Z

src/segment-loader.js

+    // on the source buffer during subsequent appends
+    //
+    // We also monitor the bytes in buffer (see checkBuffer_())
+    // and drive the loading / triming actively


gesinger · 2018-01-18T22:01:13Z

src/segment-loader.js

@@ -859,7 +885,18 @@ export default class SegmentLoader extends videojs.EventTarget {
  loadSegment_(segmentInfo) {
    this.state = 'WAITING';
    this.pendingSegment_ = segmentInfo;
-    this.trimBackBuffer_(segmentInfo);
+
+    // Chrome has a hard limit of 150MB of


Not too big of a deal, but I think this comment still makes sense to be in the trimBackBuffer function, since that is where the logic of how much to clear happens. loadSegment just needs to know to call trimBackBuffer

gesinger · 2018-01-18T22:03:43Z

src/segment-loader.js

    if (typeof segment.start === 'number' && typeof segment.end === 'number') {
-      this.mediaSecondsLoaded += segment.end - segment.start;
+      this.mediaSecondsLoaded += segmentDuration = segment.end - segment.start;


Makes sense, but is a bit hard to read. Maybe the conditionals just set the value of segmentDuration (or a ternary is used), then this.mediaSecondsLoaded += segmentDuration exists after?

gesinger · 2018-01-18T22:07:08Z

src/source-updater.js

    if (mediaSource.readyState === 'closed') {
      mediaSource.addEventListener('sourceopen', createSourceBuffer);
    } else {
      createSourceBuffer();
    }
  }

+  appendToBufferInfoQueue_(timestampOffset, duration, byteLength) {


Instead of putting all of the buffer info in SourceUpdater and keeping track of it, maybe it makes sense for us to add byte info to the segment metadata track https://github.com/videojs/videojs-contrib-hls#segment-metadata and we can make use of that.

For reference, the PR to include the required information in the segment metadata is #1210.

tchakabam · 2018-01-22T23:09:04Z

Hey, thanks for the review!

And yes @squarebracket is right, if we combine both contributions, no need anymore for the rough estimation with the /3.

I hope I can progress on both of these this week! Thanks for the patience ;)

squarebracket · 2018-01-26T06:03:29Z

Hey @tchakabam!

We've been talking in the slack channel about how to handle QuoteExceededErrors in the most robust fashion. There's a consensus that we should catch the exception and dynamically change BACK_BUFFER_LENGTH, MAX_GOAL_BUFFER_LENGTH, and MAX_SOURCE_BUFFER_OCCUPATION_BYTES based on the amount of time and data in the buffer when the exception occurs. This way we could handle various buffer sizes without knowing them in advance, and adjust them on-the-fly if system memory pressures force them to decrease in size.

That solution would require some additional functionality to what you've got here, and a new PR to contrib-media-sources, since there's a SourceBuffer shim that's used when transmuxing from MPEG-TS. (It calls appendBuffer async, so it can't be caught by contrib-hls.)

Would that be something you're up for? I can walk you through the details if you're interested. No pressure if not; I can code up the PRs and credit you with the basis for the solution :)

Thanks again for taking the time to contribute and moving us forwards on a fix.

tchakabam · 2018-01-30T00:38:24Z

@squarebracket Hey, thanks for the update on this.

These days I've got quite some new things on my plate, and unfortunately wont get to work on this one here anytime soon.

So please go on with that proposed solution :) Sounds very good to me 👍

Adapting the buffer limit dynamically is definitely favorable,

and even more favoreable to leverage the contrib-media-sources layer to handle this stuff, as it will be a common concern for any user of the API :)

Feel free to continue or close this here as you see fit!

squarebracket · 2018-01-31T20:28:14Z

Sounds good :) thanks again for the initial PR.

I'll close this one once I've got a new one open.

fbeaudet · 2018-02-09T15:33:59Z

Hi :)
I'm having this issue only in firefox, I think my samples are limit. Looking forward for that robust solution, gonna update as soon as it's out.
Thank you!

genelamarellis · 2018-05-17T13:53:14Z

I know it's been a while, but any progress on this? Running into the exact same issue! I tried implementing the fix but I'm still running into an issue. Running this on some SmartTvs. The error is thrown on some, but not all. Question, if I wanted to reduce the size even more, which values should I change in the code? Thank you!

squarebracket · 2018-05-17T15:27:11Z

Yes, actually! My apologies, I forgot to comment in this ticket.

These features are now being tracked in #1416. It would certainly be a big help to me if everyone interested in these features could go build the project with those changes and test it out with their sources / devices.

Closing this PR in favour of #1416.

tchakabam added 7 commits August 25, 2017 00:42

SourceUpdater: track items appended/removed to allow calculate total …

772ec85

…bytes in buffer

update config with back-buffer-length and max-source-buffer-occupatio…

c7adf60

…n-bytes

SourceUpdater: rm console logs, streamline logic for tracking buffer …

f371f29

…occupation

Segment-loader: use buffer-occupation-tracking to avoid QuotaExceeded…

3a4db81

…Errors properly

improve comments

7265f85

SourceUpdater: dont append init segment data to buffer-items tracking

a2c86de

use debug logger

d8585d3

tchakabam changed the title ~~Feature/prevent quota exceeded~~ Keep track of total bytes in SourceBuffer / Prevent quota exceeded Aug 25, 2017

tchakabam changed the title ~~Keep track of total bytes in SourceBuffer / Prevent quota exceeded~~ Keep track of approximate total bytes in SourceBuffer / Prevent quota exceeded Aug 25, 2017

tchakabam changed the title ~~Keep track of approximate total bytes in SourceBuffer / Prevent quota exceeded~~ Keep track of approximate total bytes in SourceBuffer / Prevent quota exceeded with 4K streams Aug 25, 2017

tchakabam mentioned this pull request Dec 7, 2017

Failed to execute 'appendBuffer' on 'SourceBuffer': The SourceBuffer is full, and cannot free space to append additional buffers. #705

Closed

gesinger reviewed Jan 18, 2018

View reviewed changes

gesinger added the pending label Jan 30, 2018

squarebracket closed this May 17, 2018

bastolino mentioned this pull request Mar 6, 2021

unable to play HLS/DASH uhd(4k) videojs/video.js#6458

Open

GGI1982 mentioned this pull request Oct 24, 2021

QuotaExceededError error flooding log but stream ok AlexxIT/WebRTC#149

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Keep track of approximate total bytes in SourceBuffer / Prevent quota exceeded with 4K streams #1242

Keep track of approximate total bytes in SourceBuffer / Prevent quota exceeded with 4K streams #1242

tchakabam commented Aug 25, 2017

gesinger left a comment

gesinger Jan 18, 2018

gesinger Jan 18, 2018

gesinger Jan 18, 2018

gesinger Jan 18, 2018

gesinger Jan 18, 2018

gesinger Jan 18, 2018

gesinger Jan 18, 2018

gesinger Jan 18, 2018

squarebracket Jan 19, 2018

tchakabam commented Jan 22, 2018

squarebracket commented Jan 26, 2018

tchakabam commented Jan 30, 2018 •

edited

Loading

squarebracket commented Jan 31, 2018

fbeaudet commented Feb 9, 2018

genelamarellis commented May 17, 2018 •

edited

Loading

squarebracket commented May 17, 2018

Keep track of approximate total bytes in SourceBuffer / Prevent quota exceeded with 4K streams #1242

Keep track of approximate total bytes in SourceBuffer / Prevent quota exceeded with 4K streams #1242

Conversation

tchakabam commented Aug 25, 2017

Description

Specific Changes proposed

Requirements Checklist

gesinger left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tchakabam commented Jan 22, 2018

squarebracket commented Jan 26, 2018

tchakabam commented Jan 30, 2018 • edited Loading

squarebracket commented Jan 31, 2018

fbeaudet commented Feb 9, 2018

genelamarellis commented May 17, 2018 • edited Loading

squarebracket commented May 17, 2018

tchakabam commented Jan 30, 2018 •

edited

Loading

genelamarellis commented May 17, 2018 •

edited

Loading