Program stops downloading VOD at 90-99% (the last time at 36%)

[seekomann]

Checklist

• I have checked the issue page for duplicates
• I am running the latest version (download here)

Edition

Windows GUI Application

Describe your issue here

The last ~10 VOD parts (.ts) have 0 KB size in the cache folder.

Previously, the program rarely had problems with a 99% download, but the second download attempt was always successful. Now I can't download VOD at all.

Add any related files or extra information here

I have problem with these VODs:
https://www.twitch.tv/videos/1886532192
https://www.twitch.tv/videos/1887391942

[ScrubN]

If you are able to reproduce again with the current version, please see if 1.53.1 solves the problem.

[seekomann]

I forgot to write that I can download 480p VOD, but 1080p has problem. So i guess it's because of the file size, 4GiB is fine, but 16GiB is a problem.

I successfully downloaded full second VOD on the new version 1.53.2, but the first VOD had the same problem. The last attempt stops at 91% (Download Threads is 1, disk with cache folder has 150 GB of free space), one last 2029.ts part has 0 KB size.

[seekomann]

I tried downloading VOD to the cache folder and finally downloaded second VOD.
Before that, my folder with downloaded VODs was on a separate drive where I didn't have a cache folder. Maybe that was the problem, idk.

[c3Vw]

I believe i've found the reason for downloading issues, it is related to twitch instability.

In general TwitchDownloader doesn't retry downloading partial files that were stuck. If you have many .ts files to download (like 1000 or so) some of them will not download properly because of twitch servers/cdns. Let's say you have 4 download threads setup.
Each interrupted .ts file download locks that download thread. If 4 files fail during downloading, no more partial files get downloaded and the whole process is stuck. Depending on number of files to download, and download threads process will stop at different % progress, it might also stop at 100% because all files were gathered but not downloaded correctly.
Here's example:

• 10 download threads
• 1465 .ts files to download
• 9 files failed (at the end of process only 1 download thread was downloading remaining parts)
  

App needs to check if some of .ts files didn't lock the whole download process

UPDATE:
actually I left the downloader running with status as above and the process finished after 4 hours of staying idle. So probably those threads did timed out and redownloaded the missing parts. Maybe timeout should've happend earlier or download speed should be verified to check if thread is stuck?

9 errors were encountered while downloading: Video part 800.ts failed after 10 retries at MoveNext
The following parts will be redownloaded: 412.ts, 795.ts, 796.ts, 797.ts, 798.ts, 800.ts, 807.ts, 1443.ts, 1446.ts
9 errors were encountered while downloading: Video part 412.ts failed after 10 retries at MoveNext

[ScrubN]

actually I left the downloader running with status as above and the process finished after 4 hours of staying idle. So probably those threads did timed out and redownloaded the missing parts. Maybe timeout should've happend earlier

Thank you for leaving it idle! I added some verification logic in #756 in case downloads were interrupted.

It's annoying that it supposedly took hours though, in theory it should have taken at most 8¼ minutes to timeout all 9 parts (assuming 1 thread). I will reduce the timeout period since we now have redownloading partially implemented.

[ScrubN]

I have found the true source of the 99% stop. When the request to Twitch is closed before the video file is completely written, the content stream is no longer receiving data but still remains open. In theory this would let the cancel button still work and in practice it does.
After a long enough wait I was also able to reproduce the unexpected resume. I'm honestly not really sure why it happens, maybe the httpclient just randomly retries? At any rate, we need a way of detecting when this happens to retry immediately.

[c3Vw]

I am able to reproduce this problem pretty often. How can I help to debug and solve this issue?
I'm not C# dev, but i tried some debugging on my own and the idea I come up with is to check in DownloadVideoPartAsync whether file part size is greater than 0 after download thread was running for 5 seconds. It's dirty solution which was slowing down the whole process, but I was able to identify video parts that were stuck. As far as I see, the generic CopyToAsync doesn't provide any progress to validate actual speed.
Furthemore this issue is not identified by properly by try-catch as the HttpClient doesn't throw timeout exception - it's the whole thread that is stuck. Maybe adding manual validation for task would work (https://stackoverflow.com/questions/4238345/asynchronously-wait-for-taskt-to-complete-with-timeout)

It's just basic idea, after playing with the code for couple hours

[ScrubN]

I am able to reproduce this problem pretty often

Is your internet very unstable? I know others experience it too, but I was only able to reproduce it myself by disconnecting from the internet for 120 seconds.

the idea I come up with is to check in DownloadVideoPartAsync whether file part size is greater than 0 after download thread was running for 5 seconds

The cache files are not actually 0KB, file explorer just has not indexed the file yet because it is still opened by TwitchDownloader. If you open the file properties window you will see it reports the true size on disk. Additionally, "just check after x seconds" means spawning a background thread for each download thread which is a huge waste of resources. If we had just one download thread then I wouldn't really mind, but when users have 20+ threads per download it's a problem.

As far as I see, the generic CopyToAsync doesn't provide any progress to validate actual speed.

I am aware and wrote an extension method to resolve this for the clip downloader.

Furthemore this issue is not identified by properly by try-catch as the HttpClient doesn't throw timeout exception - it's the whole thread that is stuck

The HttpClient will throw a timeout exception when the request is not answered within the timeout period. Once we begin downloading the file to the disk, the HttpClient is not responsible for the response message.

Maybe adding manual validation for task would work (https://stackoverflow.com/questions/4238345/asynchronously-wait-for-taskt-to-complete-with-timeout)

That stackoverflow implementation is disgusting and wastes resources. I have a few better ideas that should work while consuming less resources.

[ScrubN]

I have an implementation that verifiably solves the problem, but it may impact users with very slow internet (<1.3Mbps (160KB/s) for 1080p60 downloads). I have a WIP alternative implementation that monitors the download speed of each thread, but it's causing some extremely weird behavior and I don't understand why.

I will probably just double the timeout such that it will only affect users with extremely slow internet (<600Kbps (75KB/s) for 1080p60 downloads). According to worldpopulationreview.com, this may affect a handful countries if users are 800%+ behind the average broadband speed.