Fix captions going missing when switching streams bug #2672
Conversation
|
The missing captions were temporary, right? We would only miss the first caption after changing languages? |
Yes that is correct.
I've updated this diff and added in tests. |
Yes I agree. I was trying to follow the convention of the preceding test, but I think using await here makes more sense as well. |
You would use |
|
All tests passed! |
During video playback, if the user switches the caption stream (e.g. CC1 to CC3 which changes languages), the first caption in the next fragment is missing. In fragmented MP4s, the end time of a caption is determined by the start time of the next caption. Thus for the last caption in a fragment, the end time cannot be determined until the next fragment is parsed. Before this fix: the clearing of the caption stream was being called from a chain of function calls originating from clearBuffer_() on the Media source engine. But clearing a buffer and resetting a caption stream are two independent operations. As a result, the caption parser was being reset (its buffer cleared) during video seeks, and during stream switches. This makes sense for video seeks, because the end time of the last caption in the fragment can't be determined if the entire presentation timestamp changes. However for stream switches, resetting the parser doesn't make sense. Clearing the caption parser during a stream switch would actually get rid of the last caption in that fragment (which wasn't emitted since its end time wasn't determined yet), and we would lose the data, causing the problem. The fix is to reset (and hence clear) the caption parser during seeks, but not during stream switches. Issue #2648
github-actions
bot
added
the
status: archived
This fix also pertains to #2648.
What problem does this change fix?
The problem is (somewhat) described by the comment (that is included in the diff) from
media_source_engine.js. During video playback, if the user switches the caption stream (e.g. CC1 to CC3 which changes languages), the first caption in the next fragment is missing.Why did this problem exist?
In fragmented MP4s, the end time of a caption is determined by the start time of the next caption. Thus for the last caption in a fragment, the end time cannot be determined until the next fragment is parsed.
Before this fix: the clearing of the caption stream was being called from a chain of function calls originating from clearBuffer_() on the Media source engine. But clearing a buffer and resetting a caption stream are two independent operations. As a result, the caption parser was being reset (its buffer cleared) during video seeks, and during stream switches. This makes sense for video seeks, because the end time of the last caption in the fragment can't be determined if the entire presentation timestamp changes. However for stream switches, resetting the parser doesn't make sense. Clearing the caption parser during a stream switch would actually get rid of the last caption in that fragment (which wasn't emitted since its end time wasn't determined yet), and we would lose the data, causing the problem.
What is the fix?
For reasons mentioned previously, the fix is to reset (and hence clear) the caption parser during seeks, but not during stream switches.