New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

exhentai - don't download html files #940

Closed

fkewoafkfdsjfewjio opened this issue Aug 12, 2020 · 3 comments

Labels

bug enhancement

fkewoafkfdsjfewjio commented Aug 12, 2020

for some reason, the image limit calculation for exhentai doesn't work perfectly. maybe the formula is slightly wrong, maybe there's a bug in the code.
as a result, the last file downloaded will sometimes be an html file that just says that your image limit is maxed out. this happens most commonly with super high resolution galleries. example hd gallery. actual html file, renamed to txt so i coud upload it.

this is really bad because it's added to the download-archive, and because the link given to you to resume the download is one page too far. so, if you're using download archive, you have to manually open the link in your web browser, go back one page, manually download the image, and replace the html file.

i do not expect the image limit calculation to ever be perfect.
instead, if possible, please just add a check for if the file is html. if so, exit and print the correct resume link.
if that's not possible with how gallery-dl's code is set up, perhaps a setting could be added to change your max image limit? And I could set this setting to like 4700 instead of 5000 so that this problem is less likely to occur.

Contributor

shinji257 commented Aug 14, 2020

I do not believe (I have not looked at the code myself) that there is anything to track the image limit directly. It is handling the response from the server and reporting accordingly. Mostly because I have a 50,000 image limit on my account and sometimes I'll hit around 10,000 or so.

mikf added a commit that referenced this issue


          [exhentai] adjust image limit costs (#940)

cd9de61

Each original file costs 10 points per 10^6 bytes,
not 10 per 2^20 == 1048576 bytes.

Owner

mikf commented Aug 15, 2020

maybe the formula is slightly wrong

Yep, it was wrong. It assumed every 2^20 bytes costs 10 points, while in actuality it is 10^6 bytes (1048576 vs 1000000). Fixed in cd9de61.

please just add a check for if the file is html

That's not possible, I'm afraid. The code that fetches download links has no way of knowing if the download succeeds or what content it "contains".

perhaps a setting could be added to change your max image limit?

That would be a possible workaround. I'll see if I can implement something like that.

mikf added bug enhancement labels

mikf added a commit that referenced this issue


          [exhentai] add ability to specify custom image limit (#940)

dbbbb21

mikf added a commit that referenced this issue


          [exhentai] fix type check from dbbbb21 (#940)

0f55b8e

'bool' is a subclass of 'int', and therefore
'isinstance(self.limits, int)' also returns True when
'self.limits' has a boolean value

Owner

mikf commented Aug 30, 2020

It is now possible to use the exhentai.limits option to specify a custom image limit.

mikf closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment