Pixiv grabbing speed is too slow

[nisehime]

Not sure if you saw it, but here: #3083 (comment)

In the end, it took me 8 hours to finish updating all tasks. The possible solution I see is implementing OAuth authorization, like it was done in gallery-dl.

[KurtBestor]

Well, 281 tasks are quite a lot.
I optimized some code, now it's about x4 faster.
Try this: https://github.com/KurtBestor/Hitomi-Downloader/releases/tag/Technical-Preview

[nisehime]

Well, 281 tasks are quite a lot.

When there was login system, restarting this amount would take like 30-40 minutes.

Restarted 74 tasks now - it took me 30 minutes. There were many old ugoiras found, which Hitomi didn't download back then for some reasons, so from these 30 minutes you probably can subtract some time wasted on converting them. Anyway, first impression: better, but still not as good as it used to be.

UPD: wait, it didn't find missed ugoiras, it just redownloaded all them again, so it's a bug actually.

[KurtBestor]

It's not a bug; Bug is an error in a system that causes it to produce an unexpected result. I just didn't implement it yet.
I'll implement these "features" later.

If there is a feature you want, You can contribute to this project directly: https://github.com/KurtBestor/Hitomi-Downloader/blob/master/src/extractor/pixiv_downloader.py

[nisehime]

It's not a bug; Bug is an error in a system that causes it to produce an unexpected result. I just didn't implement it yet.
I'll implement these "features" later.

What do you mean? Every time I restart a pixiv entry which has ugoiras (animated pics), it redownloads all of them again and goes through converting (to gif) process, instead of skipping them since they're already present in the artist's folder. I checked the "date created" in the file properties and it's a new date, so hitomi has indeed overwritten existed file. It never happened before this preview I downloaded yesterday, and it's surely a bug.