Support Request: sourceBuffer_ detects incorrect timestamps on content.

[sanbornhilland]

I'm working on an HLS Parser plugin for use in safari and I have basic playback working with one major problem. When the content buffers there is an empty gap of ~10 seconds (equivalent to one chunk) at the front of the buffer and this causes playback not to start. If I seek anywhere in that 10 second range the <video> element throws an error. However, if I seek to 10 seconds or after, the video starts playing back from the beginning of the content and I can seek around etc. Essentially the video streams correctly apart from an empty gap at the beginning of the buffer.

As far as I can tell there is no way to set the start time for the data that gets appended to the buffer, it just gets read off the chunk. But if I look at the mp2t chunks they are all formed as described in the playlist. Additionally if I attach the .m3u8 directly to the <video> element it plays back fine so the browser seems to recognize that the first chunks starts at 0. It's only when feeding the chunks into a buffer one at a time that I see this gap.

I'm looking for any initial input/suggestions if you have any. I can also host a more or less working version of the player with an HLS Parser attached if you'd need to take a look, though it's still a WIP.

I'm using this content here: https://commondatastorage.googleapis.com/gtv-videos-bucket/CastVideos/hls/DesigningForGoogleCast.m3u8

[sanbornhilland]

The following is the simplest reproduction of the issue. Starting to suspect this is a Safari bug. Unless I am missing something really obvious:

    var xhr = new XMLHttpRequest();
    xhr.open('GET', 'https://commondatastorage.googleapis.com/gtv-videos-bucket/CastVideos/hls/designing_for_google_cast/fileSequence0.ts');
    xhr.responseType = 'arraybuffer';
    xhr.onreadystatechange = function () {
      if (xhr.readyState !== 4) return;

      var ms = new MediaSource();

      ms.addEventListener('sourceopen', function () {

        var videoBuffer = ms.addSourceBuffer('video/mp2t');

        videoBuffer.addEventListener('updateend', function () {
          console.log(videoBuffer.buffered.start(0));
        });

        videoBuffer.appendBuffer(xhr.response);
      });

      app.video.src = window.URL.createObjectURL(ms);
    }
    xhr.send();

[joeyparrish]

Are you sure that Safari is wrong about the timestamp? I downloaded that TS segment and fed it to ffmpeg, and ffmpeg says it starts at 9.95:

Input #0, mpegts, from 'fileSequence0.ts':
  Duration: 00:00:10.02, start: 9.952111, bitrate: 1286 kb/s

Is there any signalling in the HLS manifest that would let you know to expect this gap?

If your underlying media says "10", but you want it to be treated as "0" in MSE, you can set presentationTimeOffset: 10 in shakaExtern.Stream. But you'd want to be able to deduce that value of 10 from the m3u8.

[sanbornhilland]

It looks like you are correct. I had used mediastreamvalidator and incorrectly assumed that since HLS is assumed to start at 0 and the stream checked out, that the first chunk was starting at 0.

If I set the presentationTimeOffset to 10 then it fills the buffer as expected. The problem is that the playlist does not indicate that the first chunk starts at 10. This also seems to be a problem for a lot of HLS content. All the test content I have used for development has had a start time greater than 0. It seems to me that to support HLS in Safari will require gracefully handling these start times but it's unclear to me how to do that right now as offset time isn't known until after the chunk is downloaded.

[joeyparrish]

Hrm... that's tough. Safari's native HLS implementation, since it's in the browser, probably has easy access to the media timestamps after decode and can apply the offset internally without the user or the content creator having to do anything. We are not so lucky in JavaScript.

You could conceivably parse the first media segment in a response filter, but I think it would be preferable to avoid that.

Alternatively, some part of the player could seek for you to the first buffered timestamp once content was buffered. But it would still leave a dead gap of 10s in the beginning of the presentation timeline. If we don't also update the seek range, the user could seek back to that gap, and playback would hang.

[sanbornhilland]

A related issue is that a single, ostensibly continuous .ts segment gets buffered as multiple, non-adjacent ranges. In Shaka 1.x there was some detection and attempt at handling multiple buffered ranges but I'm not seeing anything in Shaka 2.0 that is similar. Was this intentionally removed? Are multiple ranges acceptable in 2.0? My understanding is that there should only be one range.

To be clear, I think that there is some problem with the segments in question but other HLS players seem to handle it fine so I am trying to determine if this multiple time ranges issue is actually a problem or not.

[joeyparrish]

No, you're mistaken about multiple ranges. In v1, we complained loudly about them because we had made many assumptions that the way we used MSE, they would never exist. In v2, we tried to make fewer assumptions that would require a single range, but we still don't handle them explicitly (such as jumping over gaps).

I wonder why one .ts segment would be buffered as multiple ranges. I can't think of a good reason that would be valid or expected, but I'm not familiar with details of TS.

[sanbornhilland]

I believe this is fine to close. After some additional investigation we tracked it down to a Safari bug and opened an issue with Apple. It seems they've corrected this in the latest Safari Technology Preview Release 10 of Safari 9.1.2.

[joeyparrish]

Thanks! Can you link us to the Safari bug so we can follow along?