Seeking in videos does not always work in SW mode in Chromium extension

[mossroy]

With kiwix-js 2.5.0 or latest code, and with Chromium 75.

It fails with "Video Error" in the console, and the video is stuck with a spinner on it.
I did not notice this issue in older versions of Chromium.
I think we should try to fix this before releasing a version 2.6 (#525)

[mossroy]

This happens often, but not all the time. And it depends on the ZIM file.
For example, on tedxlausannechange-2013_fr_all_2015-03.zim, it happens rarely. But it happens almost every time on ted_en_global_issues_2018-07.zim

The "Video Error" does not give much information.
A few times, the console pointed to this article : https://developers.google.com/web/updates/2017/06/play-request-was-interrupted. I'm not sure it is related (and it should be fixed in the ZIM file itself)

[mossroy]

I forgot to mention that this occurs only inside an extension.
With the same Chromium 75 in SW mode (but not in an extension), seeking always work.
And it is not solved by PR #534
Could it be a bug in Chromium (at least in version 75)?

[mossroy]

I reproduce the same behavior with Chrome 71 (on Windows), so it's unfortunately not a recent bug of the browser.
I'm a bit stuck on that issue.
Maybe we should not block version 2.6 for that?

[Jaifroid]

Seeking seems to work in Edge 44 and, more importantly, in Edge 77 (which is Chromium-based) in SW mode, but I'm not testing this in an extension. If you want me to test Edge 77 with the Chrome extension, please remind me how to do that? But it's fine if we leave this bug-fixing to a later release.

[mossroy]

The issue seems specific to extensions. It would be very interesting to see if you reproduce it on Edge 77.
In Chromium, you have 2 ways to test the extension (which might work the same on Edge) :

• install the latest signed (.crx) extension from https://download.kiwix.org/nightly/, with a drag-and-drop of the file on the extensions tab
• enable developer mode in the extensions tab and load the not-packaged extension from a local copy of the source code (in this case, you'll need to remove the "-WIP" suffix in "version" of manifest.json else it will complain about it)

[Jaifroid]

I've installed the local copy of the extension (removing -WÏP suffix) in Edge 77. Seeking works fine in SW mode in ted_en_global_issues_2018-07.zim and in dirtybiology_fr_all_2018-06.zim. (It also works fine in jQuery mode, but I know we're not testing that specifically.)

[mossroy]

Interesting. Can you reproduce the issue with Chromium/Chrome?

[Jaifroid]

I actually don't have Chromium installed on this machine in order not to conflict with Edge-Chromium. However, I'm seeing if I can test it in Chromium on WSL (Ubuntu on Windows Subsystem for Linux).

[Jaifroid]

Hmm, I can't seem to get Chromium to run at all under WSL, even though Firefox for Ubuntu runs fine with an X-server running. Seems it's not possible yet. I think we'll have to leave it for now, as I'm Ok with just running Edge Chromium on this machine.

[kelson42]

@mossroy @Jaifroid Do we have checked that we still suffer of this bug?

[mossroy]

It seems to me that I've seen it recently

[mossroy]

I reproduced somewhat similar issues on https://download.kiwix.org/zim/videos/voa_learning_english-word-of-the-day_2021-12.zim , where every video stops after a few frames with an error message "The media playback was aborted due to a corruption problem or because the media used features your browser did not support."
It only happens inside a Chromium extension AND in SW mode

I could not reproduce with https://download.kiwix.org/zim/videos/biologie-tout-compris_fr_all_2022-04.zim or https://download.kiwix.org/zim/ted/ted_en_playlist-5-questions-about-climate-change_2021-12.zim (I currently can't download big ones)

[mossroy]

I could reproduce the issue also with https://download.kiwix.org/zim/videos/dirtybiology_fr_all_2022-01.zim
When using the Chromium extension in SW mode, seeking seems to always work when you go forward, but sometimes fails when you go backward. Sometimes with the error message "(CODE:3 MEDIA_ERR_DECODE) The media playback was aborted due to a corruption problem or because the media used features your browser did not support." sent by videojs library

If in jQuery mode, or with Firefox, seeking backward always works properly

[mossroy]

From what I read in https://github.com/search?q=org%3Avideojs+MEDIA_ERR_DECODE&type=Issues and https://bugs.chromium.org/p/chromium/issues/detail?id=987951, it might depend on the OS, platform, and VideoJS version

It would be interesting that someone else tries to reproduce it.

VideoJS 7.8.1 is used by all these ZIM files (the ones where I reproduce the issue... but also the unaffected ones)

[mossroy]

@rgaudin : as I saw you handled a few other video issues on youtube and ted scrapers, could you try to reproduce this issue on your machine?

Maybe I'm the only one to have the problem?

[rgaudin]

@mossroy I just tested VOA's ZIM with 3.4.0 (Firefox) and it works fine 🤷‍♂️

[mossroy]

Thanks, but can you try with a Chromium-based browser (Chrome or Edge), from the browser extension, and in ServiceWorker mode? It's the combination that triggers the issue for me

[rgaudin]

Indeed it's not working with Chrome…

[mossroy]

But it's working fine with the same Chrome when executed outside an extension.
The only difference I can think about for now is the CSP enforced by Chrome in extensions.
There might be some code in VideoJS 7.8.1 that would be blocked by the CSP in some circumstances?
I did not find any obvious mention of that in their changelog https://github.com/videojs/video.js/blob/main/CHANGELOG.md, but I'll try to manually inject a newer version of this library to see if it helps or not

[mossroy]

I tested to force a VideoJS upgrade without modifying the ZIM file (branch https://github.com/kiwix/kiwix-js/tree/upgrade-videojs-quirk : I inject the new version through our ServiceWorker)
Unfortunately, upgrading VideoJS to 7.19.2 or 7.20.1 does not help

[mossroy]

After injecting the non-minified version of VideoJS 7.19.2, I could put breakpoints in it, but it did not give me much more information.
However, I could have some info using chrome-specific URL chrome://media-internals/ :
There's first a warning saying:

Large timestamp gap detected; may cause AV sync to drift. time:0us expected:5381000us delta:-5381000us

And afterwards errors saying:

{"code":1,"data":{},"group":"DecoderStatus","message":"","stack":[{"file":"media/filters/vpx_video_decoder.cc","line":198}]}
"video decode error!"
{"code":3,"data":{},"group":"PipelineStatus","message":"","stack":[{"file":"base/bind_internal.h","line":542}]}

Full JSON export of that:
kiwixjs-media-internals-video-json.log

This log appears ~ 5 seconds after starting the video of "sit (verb)" from https://download.kiwix.org/zim/videos/voa_learning_english-word-of-the-day_2021-12.zim

[Jaifroid]

I'ts curious that it works in JQuery mode but not in SW mode in the extension only. What are the technical differences in how we extract the video? My understanding:

• In jQuery mode, we extract the whole Uint8Array of the video as a blob, so it's in memory before it starts playing. Jumping around within it probably has very low latency;
• In SW mode, are we actually streaming data from the archive in chunks? Or at a low level is the Uint8Array being extracted in one go, and then we dip into it if we skip back and forward?

On the other hand, the fact that it works in Chromium outside of an extension appears to point to the idea that some crucial bit of playback JavaScript is inline (the only thing that is specifically blocked in an extension). This piece of JS must somehow deal with keeping track of the position of the video stream.

Sorry this isn't terribly helpful, it's just an attempt to visualize mentally what is going on.

[Jaifroid]

Would you support a pragmatic solution of detecting that we are in a Chromium extension in SW mode, and so we use the blob: technique of extracting the video? I realize it creates technical debt...

[Jaifroid]

Another data point: I can reproduce this issue in the Kiwix JS Windows Electron app (https://download.kiwix.org/nightly/2022-06-03/). The Electron app runs as a Chromium extension internally. However, the Electron app hacks the extension capabilities and it does run inline JavaScript. The only thing it can't do, like the Chromium extension, is use the Cache API. Again, streaming the video works fine in JQuery mode.

[Jaifroid]

Correction: the Electron app doesn't run as a Chromium extension (that's NWJS). The Electron app instead runs from the file:// protocol. This suggests the issue is possibly a Caching one?

[mossroy]

• In SW mode, are we actually streaming data from the archive in chunks? Or at a low level is the Uint8Array being extracted in one go, and then we dip into it if we skip back and forward?

Currently, our javascript backend does not support reading chunks of the video. So when the browser requests a chunk, we always send the whole video. That might change with upcoming libzim backend
But that works properly outside an extension, and the javascript code does not seem to be inline.
The CSP might block other things, though, like eval calls, but I did not find any of this in their source code.

The "media-internals" log tends to say it would be a low-level video decoding issue (thrown by Chromium itself, in the C code that decodes webm). But I don't see why it only occurs in an extension

Correction: the Electron app doesn't run as a Chromium extension (that's NWJS). The Electron app instead runs from the file:// protocol. This suggests the issue is possibly a Caching one?

You mean that it could come from the fact that we can't use the Cache API in extensions, and might have to re-read the ZIM file many times? That's possible, thanks for the idea. I'll investigate

Would you support a pragmatic solution of detecting that we are in a Chromium extension in SW mode, and so we use the blob: technique of extracting the video? I realize it creates technical debt...

Not for now.

[Jaifroid]

it could come from the fact that we can't use the Cache API in extensions

It's just one thing in common between the Chromium extension of Kiwix JS and the Electron app of Kiwix JS Windows: neither can use Cache API. However, we don't store a video in Cache in any case, only assets, so I'm not sure if it's relevant.

It would be interesting as an experiment to see whether temporarily caching the video file in a memory-based cache solves the issue so it's not called from the ZIM each time a user tries to skip forward or back. I'm talking about monkey-rigging a diagnostic experiment just to see if latency is the issue, not necessarily to offer that as a solution.

[mossroy]

I've added some debug logs in the ServiceWorker, and tested with https://download.kiwix.org/zim/videos/voa_learning_english-word-of-the-day_2021-12.zim
It clearly shows that the error happens right after the browser requests the second chunk of data of the video. I suppose it doesn't like what it receives in the response.
But, for now, I don't see a difference on what the SW does between the extension and a regular http://localhost
Except the requested byte range, that is a bit random. In this test, it requests "bytes=786432-" in the extension, and "bytes=1114112-" with localhost. In both cases, we return all the video data, with the same content-length "5971199", and with the same content-range "bytes 0-5971198/5971199"

[mossroy]

I'm making progress. I think it's related to range/byte-range/content-length headers and maybe the response HTTP code (200 or 206)
I'm modifying the ServiceWorker to send the actual requested range and HTTP code 206 (when a range is requested)

[mossroy]

Indeed it's not working with Chrome…

@rgaudin, a PR has been merged that should fix the issue. Could you test again tomorrow with the nightly build? (or today from source)

[rgaudin]

@mossroy, it works great with 2022-06-07 nightly! 🎉 congrats

[mossroy]

Cool, thanks for your test!