Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to new PG server + remove inline JS + remove JS deps from repo #189

Merged
merged 7 commits into from
Aug 17, 2023

Conversation

benoit74
Copy link
Collaborator

@benoit74 benoit74 commented Aug 14, 2023

Fix #187
Fix #145
Fix #116
Fix #162

Changes

  • use aleph.pglaf.org instead of dante.pglaf.org to accomodate PG infra changes
  • move inline JS (body onload + goToAuthor) to tools.js
  • get DataTables (+ Responsive) JS deps at build time instead of storing them in the repo
  • upgrade to DataTables 1.13.6 + Responsive 2.5.0
  • upgrade Docker image to python:3.11.4-bookworm

This change is Reviewable

@benoit74 benoit74 self-assigned this Aug 14, 2023
@benoit74 benoit74 requested a review from rgaudin August 14, 2023 13:43
Copy link
Collaborator Author

@benoit74 benoit74 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+reviewer:@rgaudin

Reviewable status: 0 of 13 files reviewed, all discussions resolved (waiting on @rgaudin)

@benoit74 benoit74 modified the milestone: 2.1.0 Aug 14, 2023
Copy link
Member

@rgaudin rgaudin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Would be good to create a ZIM and ask @Jaifroid to test/confirm before releasing.

Could you maybe also fix the locales warning in the docker image?

/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Reviewed 13 of 13 files at r1, 1 of 11 files at r3, all commit messages.
Reviewable status: 3 of 13 files reviewed, 1 unresolved discussion (waiting on @benoit74)


gutenbergtozim/templates/base.html line 7 at r1 (raw file):

        <title {% if show_books %} data-l10n-id="top-title"{% endif %}>{% block title %}Project Gutenberg Library{% endblock %}</title>
        <meta name="description" content="Project Gutenberg Ebooks." />
        <meta name="show_books" content="{% if show_books %}1{% else %}0{% endif %}" />

I think using a human string (true vs 1) would make more sense

@Jaifroid
Copy link

I'd be happy to test a ZIM against Kiwix JS browser extension.

The locale must be uncommented in /etc/locale.gen so that it can be installed.
@benoit74 benoit74 requested a review from rgaudin August 16, 2023 08:22
Copy link
Collaborator Author

@benoit74 benoit74 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the best/proper way to generate a test ZIM / share it ? Is it ok if I run it on my machine on few sample books (even if it could lead to some quirk due to the "it works on my machine" syndrome) and then upload it to whatever file sharing service?£

That been said, I did not encountered any issue with Kiwix JS on my Firefox (Kiwix JS 3.9.0, Firefox 115.0.2, Mac OS Monterey 12.6.7) and the latest dev ZIM (https://mirror.download.kiwix.org/zim/.hidden/dev/gutenberg_en_all_2023-03.zim), i.e without removing inline JS, so I'm not sure about what has to be tested.

I also realized that with checked the box on kiwix/kiwix-js#865 for openedx + kolibri without a test from @Jaifroid (I ran tests on my side, but I would prefer to have your confirmation as well), I will provide test ZIMs for those two scrapers as well so that we confirm everything is ok before closing the issue.

I fixed the locale issue.

Reviewable status: 3 of 13 files reviewed, 1 unresolved discussion (waiting on @rgaudin)


gutenbergtozim/templates/base.html line 7 at r1 (raw file):

Previously, rgaudin (rgaudin) wrote…

I think using a human string (true vs 1) would make more sense

Done.

Copy link
Collaborator Author

@benoit74 benoit74 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oups, I forgot to push ...

Reviewable status: 2 of 13 files reviewed, 1 unresolved discussion (waiting on @rgaudin)

Copy link
Member

@rgaudin rgaudin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 8 of 11 files at r3, 3 of 3 files at r4, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @benoit74)

@Jaifroid
Copy link

That been said, I did not encountered any issue with Kiwix JS on my Firefox (Kiwix JS 3.9.0, Firefox 115.0.2, Mac OS Monterey 12.6.7) and the latest dev ZIM (https://mirror.download.kiwix.org/zim/.hidden/dev/gutenberg_en_all_2023-03.zim), i.e without removing inline JS, so I'm not sure about what has to be tested.

The issue with inline JS is in the Chrome extension, not the Firefox extension. For security reasons, inline JS is banned in that context. Are you able to provide a small ZIM archive I could test in that context? The Romanian language scrape of Gutenberg is quite small for instance. (a few MB).

@kelson42
Copy link
Contributor

Kudos! Looks good to merge then :)

@benoit74
Copy link
Collaborator Author

@Jaifroid
I will provide you small ZIMs of the three scrappers I've modified tomorrow so that you can test them, I have lots of options to generate small ZIMs on my machine 😄

@benoit74
Copy link
Collaborator Author

I will merge as-is this PR, hence marking the issue as closed, follow-up on tests will be done on kiwix/kiwix-js#865 and we will create a new issue should something bad be found.

@benoit74 benoit74 merged commit 1ea3cc1 into main Aug 17, 2023
@benoit74 benoit74 deleted the issue_145 branch August 17, 2023 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants