Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Populate the real list of languages in ZIM metadata #181

Merged
merged 4 commits into from
Apr 18, 2024
Merged

Conversation

benoit74
Copy link
Collaborator

@benoit74 benoit74 commented Apr 16, 2024

Rationale

Fix #172

Changes

  • moved all lang related functions to languages.py
  • added few tests for existing conversion from input languages to TED code
  • added new function to convert back TED code to ISO639-3 code
  • added code to compute ZIM languages (twice, asap to avoid downloading videos if we know there is a metadata issue and later once the final list of downloaded videos is known)
  • better handle case when opening the video page with the lang queryparameter not set

@benoit74 benoit74 self-assigned this Apr 16, 2024
@benoit74 benoit74 marked this pull request as ready for review April 16, 2024 12:09
@benoit74 benoit74 requested a review from rgaudin April 16, 2024 12:09
Copy link
Member

@rgaudin rgaudin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you ; see inline comments

src/ted2zim/languages.py Outdated Show resolved Hide resolved
src/ted2zim/languages.py Outdated Show resolved Hide resolved
src/ted2zim/scraper.py Outdated Show resolved Hide resolved
src/ted2zim/scraper.py Outdated Show resolved Hide resolved
src/ted2zim/scraper.py Show resolved Hide resolved
src/ted2zim/scraper.py Outdated Show resolved Hide resolved
tests/test_languages.py Show resolved Hide resolved
- moved all lang related functions to languages.py
- added few tests for existing conversion from input languages to TED
  code
- added new function to convert back TED code to ISO639-3 code
- added code to compute ZIM languages (twice, asap to avoid downloading
  videos if we know there is a metadata issue and later once the final
  list of downloaded videos is known)
@benoit74
Copy link
Collaborator Author

I had to rebase the branch so that conflict is gone.

You should do your second review with only last 3 commits, 4th one is not changed, just rebased: https://github.com/openzim/ted/pull/181/files/13de099f8fd4b53753e0150313f02e6a54b63796..HEAD

Note the last commit which is something new, I just experienced a bug in the ZIM UI when displaying a video without and lang query parameter set (don't know exactly why, but it happened to me ... probably when there is only one language in the ZIM)

@benoit74
Copy link
Collaborator Author

don't know exactly why, but it happened to me ... probably when there is only one language in the ZIM

I got it, in fact kiwix-serve removes the lang query parameter in the URL displayed in the browser, so if you just reuse the same link, the lang query parameter is gone and UI was broken (no title, no description displayed at all)

@benoit74 benoit74 requested a review from rgaudin April 18, 2024 09:39
Copy link
Member

@rgaudin rgaudin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rgaudin rgaudin merged commit 47a89e3 into main Apr 18, 2024
5 checks passed
@rgaudin rgaudin deleted the real_languages branch April 18, 2024 09:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Filter and order languages in ZIM Language metadata
2 participants