From 6522c2402a94cd8bc2927fbb2992ce7547d57771 Mon Sep 17 00:00:00 2001 From: Nat Alison Date: Thu, 12 Sep 2019 23:11:42 -0700 Subject: [PATCH 1/6] RFC for Gatsby Docs Localization --- text/0010-gatsby-docs-localization | 104 +++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 text/0010-gatsby-docs-localization diff --git a/text/0010-gatsby-docs-localization b/text/0010-gatsby-docs-localization new file mode 100644 index 0000000..4198432 --- /dev/null +++ b/text/0010-gatsby-docs-localization @@ -0,0 +1,104 @@ +* Start Date: 2019-09-11 +* RFC PR: +* Gatsby Issue: + +## Summary + +Begin the process of localizing the documentation of Gatsby into non-English languages. + +## Motivation + +[You Belong Here.](https://www.gatsbyjs.org/blog/2018-09-07-gatsby-values/#you-belong-here) + +Gatsby believces that everyone belongs in Open Source, and that includes developers that do not speak English. Having access to translated documentation opens up the ability to create websites to people who don't speak English and don't have the privileges to be able to learn. + +## Detailed Design + +There are two main approaches we can take for the translations. Whichever path we choose depends on the prorities of the Gatsby team, hence the Request for Comments. The options are: (1) doing translations on GitHub directly or (2) using a translation SaaS platform such as Crowdin. This document will go over both alternatives and the advantages and drawbacks of each option. + +## Alternative 1: GitHub-based translations + +The first and simplest option is to simply create and host the translations on GitHub itself, along with the rest of the Gatsby documentation. For smaller websites (like [Dan Abramov's blog](https://github.com/gaearon/overreacted.io)) it's enough to put each translation in a markdown file next to the source file and use `gatsby-i18n`. + +For larger websites, especially documentation websites that have lots of PRs and require versioning and up-to-date content, this isn't enough. The docs for [Angular](https://github.com/angular/angular-cn/tree/aio), [Vue](https://github.com/vuejs/jp.vuejs.org), and [React](https://github.com/reactjs/es.reactjs.org) store their translations in copies of the repo. + +### Advantages + +This was the approach that we took for translating the React docs. It appealed to us for a lot of reasons: + +* Open Source Contributors tend to be familiar with Git +* No need to integrate with an external API and software (aside from GitHub) +* People want to be listed as contributors of the React/Gatsby organization on GitHub, and it's an incentive to contribute. +* Can use GitHub features such as Code Owners and pull requests as quality assurance + +### Drawbacks + +The main drawback of this approach is that the Gatsby docs currently live in a monorepo with the rest of the Gatsby source, which means we can't just do a copy of the entire website and use `git merge` like we do with React (and Vue). + +This is also an issue if we want translations to live under a path (e.g. `gatsbyjs.org/ja` rather than `ja.gatsbyjs.org`). + +Other drawbacks we learned from working on the React translation: + +* Need to maintain a custom bot and system to keep the translations up to date. +* It's not a system that can be easily put into a plugin to allow others to use. + +### Mitigation + +I think this is a viable approach if we can: + +1. move the docs to their own repo and +2. are okay with doing subdomains (`ja.gatsbyjs.org`) instead of paths (`gatsbyjs.org/ja`) + +Otherwise we might want to consider the second alternatives: + +## Alternative 2: Translations Platform + +[Crowdin]() is a SaaS that allows projects to source translations using a built-in web based editor. It's popular in the JS community being used for projects such as [Electron](https://crowdin.com/project/electron) and [Yarn](https://crowdin.com/project/yarn). + +### Advantages + +* Lots of features, such as [machine translation](https://support.crowdin.com/pre-translation-via-machine/) and [in-context localization](https://support.crowdin.com/in-context-localization/). +* Has a builtin [GitHub integration](https://support.crowdin.com/github-integration/). +* We can make an official plugin for Crowdin integration that other projects can use. + +### Drawbacks + +We had considered using Crowdin for the localization of the React docs but realized that Crowdin had some [major drawbacks](https://reactjs.org/blog/2019/02/23/is-react-translated-yet.html): + +* It has a steep learning curve and hard for new translators to get accustomed to. +* The GitHub integration isn't customizable: it takes three hours to compile all the languages and just publishes something in a single directory. +* Translation quality issues: + * Doesn't handle web markup well; sites translated with Crowdin sometimes have invalid markup + * (as far as I can tell) no way to verify that a section/page has to pass a quality check before it gets published + +Other drawbacks of Crowdin: + +* No way to prioritize which pages should get translated first +* Hard to attribution of translators +* Paid program ($125/mo for the 'Bronze' Organization Plan) +* Quality of English documentation is mediocre, which is not a good sign for a translation app (see links above). + +### Mitigation + +* We'd need to create our resources to supplement Crowdin's documentation +* Research other translation platforms + +### Other Translation Platforms + +There are more translation platforms other than Crowdin, and using them instead may mitigate some of those issues. + +One I've played around with is [Transifex](http://transifex.com/), which is used to translate [Django](https://www.transifex.com/django/django/). It's pricing is comparable to Crowdin ($139/mo for the "Starter" plan), and it eliminates a lot of the drawbacks Crowdin has: + +* Good, clear documentation on all its features +* A [GitHub integration](https://docs.transifex.com/transifex-github-integrations/github-tx-ui) that is flexible and allows quality checking, allowing you to merge only when each page is completed and reviewed + +My current thoughts are to research Transifex more to find out its strengths and weaknesses. + +## Adoption Strategy / How We Teach This + +// to be filled when we decide on an approach + +## Open Questions + +* What translation platforms do people interested in translating Gatsby use and like? **If you are interested in localizing Gatsby to your language, please let us know!** +* If you've used Crowdin or Transifex, how was the experience? What is good about it and what are the pain points? From 0f9cb6e1fd4e2f1437581953c68aa9e02141fa82 Mon Sep 17 00:00:00 2001 From: Michael <184316+muescha@users.noreply.github.com> Date: Wed, 18 Sep 2019 19:34:52 +0200 Subject: [PATCH 2/6] renamed to Markdown File --- ...-gatsby-docs-localization => 0010-gatsby-docs-localization.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename text/{0010-gatsby-docs-localization => 0010-gatsby-docs-localization.md} (100%) diff --git a/text/0010-gatsby-docs-localization b/text/0010-gatsby-docs-localization.md similarity index 100% rename from text/0010-gatsby-docs-localization rename to text/0010-gatsby-docs-localization.md From b4d2ea72941fc88d422d54fb6ef08ec841fcad6e Mon Sep 17 00:00:00 2001 From: Nat Alison Date: Thu, 19 Sep 2019 14:19:16 -0500 Subject: [PATCH 3/6] Update 0010-gatsby-docs-localization.md --- text/0010-gatsby-docs-localization.md | 212 +++++++++++++++++++------- 1 file changed, 160 insertions(+), 52 deletions(-) diff --git a/text/0010-gatsby-docs-localization.md b/text/0010-gatsby-docs-localization.md index 4198432..0e79ef9 100644 --- a/text/0010-gatsby-docs-localization.md +++ b/text/0010-gatsby-docs-localization.md @@ -2,103 +2,211 @@ * RFC PR: * Gatsby Issue: + ## Summary Begin the process of localizing the documentation of Gatsby into non-English languages. + ## Motivation [You Belong Here.](https://www.gatsbyjs.org/blog/2018-09-07-gatsby-values/#you-belong-here) -Gatsby believces that everyone belongs in Open Source, and that includes developers that do not speak English. Having access to translated documentation opens up the ability to create websites to people who don't speak English and don't have the privileges to be able to learn. +Gatsby believes that everyone belongs in Open Source, and that includes developers that do not speak English. Having access to translated documentation opens up the ability to create websites to people who don't speak English and don't have the privileges to be able to learn. + ## Detailed Design -There are two main approaches we can take for the translations. Whichever path we choose depends on the prorities of the Gatsby team, hence the Request for Comments. The options are: (1) doing translations on GitHub directly or (2) using a translation SaaS platform such as Crowdin. This document will go over both alternatives and the advantages and drawbacks of each option. +The translations will be done in the [same monorepo](https://github.com/gatsbyjs/gatsby) as the rest of the Gatsby documentation and code. Additional tooling, automation, and documentation will be made to ensure that the languages stay up-to-date and to maintain translation quality. -## Alternative 1: GitHub-based translations +### Overview of Changes -The first and simplest option is to simply create and host the translations on GitHub itself, along with the rest of the Gatsby documentation. For smaller websites (like [Dan Abramov's blog](https://github.com/gaearon/overreacted.io)) it's enough to put each translation in a markdown file next to the source file and use `gatsby-i18n`. +The following things need to be created: -For larger websites, especially documentation websites that have lots of PRs and require versioning and up-to-date content, this isn't enough. The docs for [Angular](https://github.com/angular/angular-cn/tree/aio), [Vue](https://github.com/vuejs/jp.vuejs.org), and [React](https://github.com/reactjs/es.reactjs.org) store their translations in copies of the repo. +* [ ] A bot to track changes to the English docs and create pull requests +* [ ] Script to aid in verifying translation accuracy +* [ ] Script to set up a new language +* [ ] Documentation for how to start a new language translation and how to contribute translations +* [ ] A prioritized list of pages to translate -### Advantages +There are several changes to the repo that need to be made in order to support localization: -This was the approach that we took for translating the React docs. It appealed to us for a lot of reasons: +* [ ] All text strings in `/www` need to be moved to `/docs` so they can be translated +* [ ] All content in `/docs` need to be moved to `/docs/en` +* [ ] Support for right-to-left languages +* [ ] Support for language-specific fonts +* [ ] UI to toggle between languages +* [ ] Preserve hashes when switching languages (e.g. `/en/docs/quick-start/#use-the-gatsby-cli` should go to `/es/docs/quick-start/#use-the-gatsby-cli` even though the heading has been translated to Spanish) +* [ ] Make Algolia only search for results within each language -* Open Source Contributors tend to be familiar with Git -* No need to integrate with an external API and software (aside from GitHub) -* People want to be listed as contributors of the React/Gatsby organization on GitHub, and it's an incentive to contribute. -* Can use GitHub features such as Code Owners and pull requests as quality assurance +In addition to these crucial tasks, there are a few nice-to-haves: -### Drawbacks +* [ ] Language auto-detection based on browser language (e.g. redirecting `gatsbyjs.org` to `gatsbyjs.org/fr`). +* [ ] "Not Yet Translated" warning that appears when trying to access a page that doesn't have a translation -The main drawback of this approach is that the Gatsby docs currently live in a monorepo with the rest of the Gatsby source, which means we can't just do a copy of the entire website and use `git merge` like we do with React (and Vue). +### Project Organization -This is also an issue if we want translations to live under a path (e.g. `gatsbyjs.org/ja` rather than `ja.gatsbyjs.org`). +The documentation of each language will be put in a folder under the top-level `/docs` directory in the monorepo: -Other drawbacks we learned from working on the React translation: +``` +|-- /docs + |-- /en + |-- /docs + |-- /blog + |-- /es + |-- /docs + |-- /blog +``` -* Need to maintain a custom bot and system to keep the translations up to date. -* It's not a system that can be easily put into a plugin to allow others to use. +All content currently under `/docs` will need to be moved under an `/en` directory, and we will need to change routing logic so that `gatsbyjs.org/[content]` redirects to `gatsbyjs.org/en/[content]` (or possibly do automatic locale detection). -### Mitigation +The reason for doing this structure is so that all the documentation for each language can be under a single directory instead of being scattered across the `docs/` repository. + +In addition, this allows maintainers of each language to use their language directory to keep additional information such as README's for translators or style guides/glossaries to keep translations consistent. + +Another option that `gatsby-i18n` supports is to annotate each file with its language code (e.g. `accessibility.en.md`). If we do this, some directories would be cluttered with too many files: + +``` +|-- /docs + |-- accessibility.en.md + |-- accessibility.es.md + |-- accessibility.fr.md + |-- accessibility.ja.md + |-- accessibility.zh-Hans.md + |-- accessibility.zh-Hant.md + |-- links.en.md + |-- links.es.md + |-- links.fr.md + |-- links.ja.md + |-- links.zh-Hans.md + |-- links.zh-Hant.md + |-- ... +``` + +To prevent missing routes due to typos, we can also create a check in the build process to make sure that every file in a non-English subdirectory has an equivalent English one. This script could also verify the structure of documents (for example, that all translations have the same sequence of headings). + +### Maintainers + +Each language translation will require at least two *maintainers* who are in charge of coordinating the translation of that language. Responsibilities include: + +* Organizing work around which pages to translate and who translates them +* Reviewing new translations +* Keeping translations up-to-date + +These maintainers will be assigned as the code owners of their language directory. + +### Starting a new language translation + +Contributors who would like to be maintainers of a new language repo should create a new issue with: + +* the English and native name of the language +* the [IETF language code](https://en.wikipedia.org/wiki/IETF_language_tag) of the language +* the GitHub usernames of all proposed maintainers + +in a yaml file like so: + +``` +name: Spanish +localName: Español +code: es +maintainers: + - tesseralis + - KyleAMatthews +``` + +When accepted, a script should: + +* Create a new folder in the repo (e.g. `/docs/es`) with a template README and style guide +* Edit `CODEOWNERS.md` adding the proposed maintainers (either directly or through creating a new team to the Gatsby org) +* Create a new issue for tracking translation progress, and a new label (e.g. `translation-es`). + +The script should *not* copy over any files from the `/docs/en` directory. This should be done as contributors make new translations and to prevent the repo from becoming bloated with copies. -I think this is a viable approach if we can: +### Organizing work -1. move the docs to their own repo and -2. are okay with doing subdomains (`ja.gatsbyjs.org`) instead of paths (`gatsbyjs.org/ja`) +Each supported language should have its own GitHub label (for example `translation-es`) for all changes made to a file in its directory, so that it will be easy to search for issues relating to the language. -Otherwise we might want to consider the second alternatives: +Each language will have a "progress tracking issue" with the prioritized list of pages to translate, much like [React](https://github.com/reactjs/id.reactjs.org/issues/1). Unlike React, the issue should ideally be automated (similar to [Renovate](https://github.com/gatsbyjs/gatsby/issues/16840)). -## Alternative 2: Translations Platform +### Updating content -[Crowdin]() is a SaaS that allows projects to source translations using a built-in web based editor. It's popular in the JS community being used for projects such as [Electron](https://crowdin.com/project/electron) and [Yarn](https://crowdin.com/project/yarn). +When a translation of a page is completed, we still need to ensure that the translation remains updated when the English source is changed. To do that, we use a bot similar to [React](https://github.com/reactjs/de.reactjs.org/pull/88), but adapted for Gatsby's monorepo. -### Advantages +When a change is detected in the `/docs/en` directory, the bot should, for each language: + +* Determine the time a pull request by the bot has occurred +* Run `git diff` on `/docs/en` from that hash +* Apply the patch on `/docs/[lang]` using `git patch` +* Commit all the merge conflicts and create a pull request (or some other way to annotate which lines have changed) +* Assign the issue to the language maintainers + +If an issue already exists for a previous sync, the bot should just edit it with patches from the last sync instead of creating a new issue. + +### Tracking Progress + +We should track the progress of each language translation to keep track of what languages are close to completion, which ones need help, etc. We could either use a site like https://isreacttranslatedyet.com or just use the "Translations" page like [Electron](https://electronjs.org/languages) does. + +Unlike https://isreacttranslatedyet.com, which scrapes GitHub issues to track progress, we can actually just query the website itself to check which pages exist in which languages. + +## Drawbacks + +There is a fair amount of work needed to create the automation necessary to run the translations. Unlike a SaaS platform, we don't have side-by-side editing, automatic translation of strings, or translation memory. Since everything needs to be done in the same repo and website, there is a lot more up-front work and branching logic needed to support things like right-to-left text. + +### Mitigation + +Thankfully, Gatsby has already invested a lot of effort into GitHub-based tooling, and the RFC proposer already has experience implementing a GitHub-based translation management system for React. Translation memory is a nice-to-have, but once the initial work has been done we can think of ways to implement it, for example using a [VSCode extension](https://code.visualstudio.com/docs/editor/extension-gallery). + +## Alternatives + +### SaaS Platform (Crowdin) + +[Crowdin](https://crowdin.com/) is a SaaS that allows projects to source translations using a built-in web based editor. It's popular in the JS community being used for projects such as [Electron](https://crowdin.com/project/electron) and [Yarn](https://crowdin.com/project/yarn). * Lots of features, such as [machine translation](https://support.crowdin.com/pre-translation-via-machine/) and [in-context localization](https://support.crowdin.com/in-context-localization/). * Has a builtin [GitHub integration](https://support.crowdin.com/github-integration/). +* Popular with translators. * We can make an official plugin for Crowdin integration that other projects can use. -### Drawbacks +However, Crowdin has its fair share of disadvantages as well. We had considered using Crowdin for the localization of the React docs but realized that Crowdin had some [major drawbacks](https://reactjs.org/blog/2019/02/23/is-react-translated-yet.html): -We had considered using Crowdin for the localization of the React docs but realized that Crowdin had some [major drawbacks](https://reactjs.org/blog/2019/02/23/is-react-translated-yet.html): +* It has a steep learning curve and hard for new users (especially beginning translators) to get accustomed to. +* Doesn't handle web markup well; sites translated with Crowdin sometimes have invalid markup +* Quality of English documentation is mediocre, which is not a good sign for a translation app (see links above). -* It has a steep learning curve and hard for new translators to get accustomed to. -* The GitHub integration isn't customizable: it takes three hours to compile all the languages and just publishes something in a single directory. -* Translation quality issues: - * Doesn't handle web markup well; sites translated with Crowdin sometimes have invalid markup - * (as far as I can tell) no way to verify that a section/page has to pass a quality check before it gets published - -Other drawbacks of Crowdin: +There are other SaaS platforms such as Transifex that fix some of Crowdin's issues. However, the core issue relies in that they are new systems that most developers are unfamiliar with. We are relying on developers from the Gatsby community rather than professional translators, and those developers tend to be more familiar with GitHub's UI. -* No way to prioritize which pages should get translated first -* Hard to attribution of translators -* Paid program ($125/mo for the 'Bronze' Organization Plan) -* Quality of English documentation is mediocre, which is not a good sign for a translation app (see links above). +### Other GitHub-based arrangements -### Mitigation +The monorepo isn't the only way to organize our code. We could do something like React and have each language be in a separate repo copy. This makes it easier to copy content and check for changes, since we can just use `git merge`. + +But this has its own drawbacks as well. Each language is its own website and we can't easily switch between languages. The rest of Gatsby's code is in a monorepo and has tooling to make it work, so it makes sense to continue with that paradigm. We'd also like to dogfood Gatsby's localization utilities such as [`gatsby-plugin-i18n`](https://www.gatsbyjs.org/docs/localization-i18n/#gatsby-plugin-i18n). + +## Adoption Strategy + +To test-drive the process, we can start with three languages or so, based on user interest in this RFC. We will have to manually set up those translations while we create the automation, but they should provide a test bed to make sure our checks work. + +The only thing we need to do before those languages are set up is move content over to `/docs/en` and handle the routing appropriately. Once that's done, the translations can start and we can divide up the other changes needed for the website. -* We'd need to create our resources to supplement Crowdin's documentation -* Research other translation platforms +Once the automation work for setting up new translations is done, we can open up pull requests to create new language to more people. -### Other Translation Platforms +When the first languages have a "core" set of pages translated (for example, the Home Page, Tutorial, and API Reference), that translation can be marked as "Complete". -There are more translation platforms other than Crowdin, and using them instead may mitigate some of those issues. -One I've played around with is [Transifex](http://transifex.com/), which is used to translate [Django](https://www.transifex.com/django/django/). It's pricing is comparable to Crowdin ($139/mo for the "Starter" plan), and it eliminates a lot of the drawbacks Crowdin has: +## How will we teach this -* Good, clear documentation on all its features -* A [GitHub integration](https://docs.transifex.com/transifex-github-integrations/github-tx-ui) that is flexible and allows quality checking, allowing you to merge only when each page is completed and reviewed +We'll create additional documentation in both the monorepo and the site itself on the process of contributing translations. There should be documentation for: -My current thoughts are to research Transifex more to find out its strengths and weaknesses. +* How to contribute a new translation (with a link to the progress issue) +* How the automation works and what to expect +* The responsibilities of a translation maintainer +* Tips for translators -## Adoption Strategy / How We Teach This +We should also do outreach to Gatsby users and communities around the world to inform them that the translations are in progress and (once completed) spread the word that the localizations exist. -// to be filled when we decide on an approach -## Open Questions +## Unresolved Questions -* What translation platforms do people interested in translating Gatsby use and like? **If you are interested in localizing Gatsby to your language, please let us know!** -* If you've used Crowdin or Transifex, how was the experience? What is good about it and what are the pain points? +* If we intend to publish different versions of the Gatsby docs, how will versioning work across the different languages? +* How do we limit the Algolia search to only return results for each language? +* How do we manage SEO? +* What is the process for determining who gets to maintain a language? For React, we pretty much let the first two people to create a pull request start their own language translation. From 7bfab05c59f852db81b1e2c338e9ea9a11422c94 Mon Sep 17 00:00:00 2001 From: Nat Alison Date: Fri, 20 Sep 2019 00:46:08 -0500 Subject: [PATCH 4/6] `/docs/es` --> `/translations/es` --- text/0010-gatsby-docs-localization.md | 30 +++++++++++++-------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/text/0010-gatsby-docs-localization.md b/text/0010-gatsby-docs-localization.md index 0e79ef9..027c581 100644 --- a/text/0010-gatsby-docs-localization.md +++ b/text/0010-gatsby-docs-localization.md @@ -31,8 +31,10 @@ The following things need to be created: There are several changes to the repo that need to be made in order to support localization: -* [ ] All text strings in `/www` need to be moved to `/docs` so they can be translated -* [ ] All content in `/docs` need to be moved to `/docs/en` +* [ ] Move all text strings in `/www` to `/docs` so they can be translated +* [ ] Create a `translations/` directory +* [ ] Make routes such as `/docs` redirect to `/en/docs` +* [ ] Support routes for translations (such as `/es/docs`) * [ ] Support for right-to-left languages * [ ] Support for language-specific fonts * [ ] UI to toggle between languages @@ -46,20 +48,18 @@ In addition to these crucial tasks, there are a few nice-to-haves: ### Project Organization -The documentation of each language will be put in a folder under the top-level `/docs` directory in the monorepo: +The documentation of each language will be put in a folder under the top-level `/translations` directory in the monorepo: ``` |-- /docs - |-- /en - |-- /docs - |-- /blog + |-- /docs + |-- /blog +|-- /translations |-- /es |-- /docs |-- /blog ``` -All content currently under `/docs` will need to be moved under an `/en` directory, and we will need to change routing logic so that `gatsbyjs.org/[content]` redirects to `gatsbyjs.org/en/[content]` (or possibly do automatic locale detection). - The reason for doing this structure is so that all the documentation for each language can be under a single directory instead of being scattered across the `docs/` repository. In addition, this allows maintainers of each language to use their language directory to keep additional information such as README's for translators or style guides/glossaries to keep translations consistent. @@ -83,7 +83,7 @@ Another option that `gatsby-i18n` supports is to annotate each file with its lan |-- ... ``` -To prevent missing routes due to typos, we can also create a check in the build process to make sure that every file in a non-English subdirectory has an equivalent English one. This script could also verify the structure of documents (for example, that all translations have the same sequence of headings). +To prevent missing routes due to typos, we can also create a check in the build process to make sure that every file in a translated subdirectory has an equivalent English one. This script could also verify the structure of documents (for example, that all translations have the same sequence of headings). ### Maintainers @@ -116,11 +116,11 @@ maintainers: When accepted, a script should: -* Create a new folder in the repo (e.g. `/docs/es`) with a template README and style guide +* Create a new folder in the repo (e.g. `/translations/es`) with a template README and style guide * Edit `CODEOWNERS.md` adding the proposed maintainers (either directly or through creating a new team to the Gatsby org) * Create a new issue for tracking translation progress, and a new label (e.g. `translation-es`). -The script should *not* copy over any files from the `/docs/en` directory. This should be done as contributors make new translations and to prevent the repo from becoming bloated with copies. +The script should *not* copy over any files from the `/docs` directory. This should be done as contributors make new translations and to prevent the repo from becoming bloated with copies. ### Organizing work @@ -132,11 +132,11 @@ Each language will have a "progress tracking issue" with the prioritized list of When a translation of a page is completed, we still need to ensure that the translation remains updated when the English source is changed. To do that, we use a bot similar to [React](https://github.com/reactjs/de.reactjs.org/pull/88), but adapted for Gatsby's monorepo. -When a change is detected in the `/docs/en` directory, the bot should, for each language: +When a change is detected in the `/docs` directory, the bot should, for each language: * Determine the time a pull request by the bot has occurred -* Run `git diff` on `/docs/en` from that hash -* Apply the patch on `/docs/[lang]` using `git patch` +* Run `git diff` on `/docs` from that hash +* Apply the patch on `/translations/[lang]` using `git apply` * Commit all the merge conflicts and create a pull request (or some other way to annotate which lines have changed) * Assign the issue to the language maintainers @@ -185,8 +185,6 @@ But this has its own drawbacks as well. Each language is its own website and we To test-drive the process, we can start with three languages or so, based on user interest in this RFC. We will have to manually set up those translations while we create the automation, but they should provide a test bed to make sure our checks work. -The only thing we need to do before those languages are set up is move content over to `/docs/en` and handle the routing appropriately. Once that's done, the translations can start and we can divide up the other changes needed for the website. - Once the automation work for setting up new translations is done, we can open up pull requests to create new language to more people. When the first languages have a "core" set of pages translated (for example, the Home Page, Tutorial, and API Reference), that translation can be marked as "Complete". From 571b7b1022536ec5b1dd0962e2b478bbbd453059 Mon Sep 17 00:00:00 2001 From: Nat Alison Date: Fri, 20 Sep 2019 01:00:51 -0500 Subject: [PATCH 5/6] Update 0010-gatsby-docs-localization.md --- text/0010-gatsby-docs-localization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0010-gatsby-docs-localization.md b/text/0010-gatsby-docs-localization.md index 027c581..c2ae5cf 100644 --- a/text/0010-gatsby-docs-localization.md +++ b/text/0010-gatsby-docs-localization.md @@ -31,7 +31,7 @@ The following things need to be created: There are several changes to the repo that need to be made in order to support localization: -* [ ] Move all text strings in `/www` to `/docs` so they can be translated +* [ ] Move all text strings in `/www` to `/docs` so they can be translated: https://github.com/gatsbyjs/gatsby/issues/17758 * [ ] Create a `translations/` directory * [ ] Make routes such as `/docs` redirect to `/en/docs` * [ ] Support routes for translations (such as `/es/docs`) From 13acd0917d59c03f9704a8855819f05e08cbadcc Mon Sep 17 00:00:00 2001 From: Nat Alison Date: Mon, 30 Sep 2019 16:31:29 +0100 Subject: [PATCH 6/6] Update 0010-gatsby-docs-localization.md --- text/0010-gatsby-docs-localization.md | 74 ++++++++------------------- 1 file changed, 21 insertions(+), 53 deletions(-) diff --git a/text/0010-gatsby-docs-localization.md b/text/0010-gatsby-docs-localization.md index c2ae5cf..35a16eb 100644 --- a/text/0010-gatsby-docs-localization.md +++ b/text/0010-gatsby-docs-localization.md @@ -17,23 +17,23 @@ Gatsby believes that everyone belongs in Open Source, and that includes develope ## Detailed Design -The translations will be done in the [same monorepo](https://github.com/gatsbyjs/gatsby) as the rest of the Gatsby documentation and code. Additional tooling, automation, and documentation will be made to ensure that the languages stay up-to-date and to maintain translation quality. +The translations will be done on GitHub using Markdown files, in separate repos per translation, similar to [React](https://github.com/reactjs/es.reactjs.org) and [Vue](https://github.com/vuejs/jp.vuejs.org). Additional tooling, automation, and documentation will be made to ensure that the languages stay up-to-date and to maintain translation quality. ### Overview of Changes The following things need to be created: * [ ] A bot to track changes to the English docs and create pull requests -* [ ] Script to aid in verifying translation accuracy +* [ ] Script/linters to aid in verifying translation accuracy * [ ] Script to set up a new language * [ ] Documentation for how to start a new language translation and how to contribute translations * [ ] A prioritized list of pages to translate There are several changes to the repo that need to be made in order to support localization: -* [ ] Move all text strings in `/www` to `/docs` so they can be translated: https://github.com/gatsbyjs/gatsby/issues/17758 -* [ ] Create a `translations/` directory +* [ ] Move all text strings in `/www` to YAML files so they can be translated: https://github.com/gatsbyjs/gatsby/issues/17758 * [ ] Make routes such as `/docs` redirect to `/en/docs` +* [ ] Plugin to pull in translated content from different repos * [ ] Support routes for translations (such as `/es/docs`) * [ ] Support for right-to-left languages * [ ] Support for language-specific fonts @@ -48,42 +48,7 @@ In addition to these crucial tasks, there are a few nice-to-haves: ### Project Organization -The documentation of each language will be put in a folder under the top-level `/translations` directory in the monorepo: - -``` -|-- /docs - |-- /docs - |-- /blog -|-- /translations - |-- /es - |-- /docs - |-- /blog -``` - -The reason for doing this structure is so that all the documentation for each language can be under a single directory instead of being scattered across the `docs/` repository. - -In addition, this allows maintainers of each language to use their language directory to keep additional information such as README's for translators or style guides/glossaries to keep translations consistent. - -Another option that `gatsby-i18n` supports is to annotate each file with its language code (e.g. `accessibility.en.md`). If we do this, some directories would be cluttered with too many files: - -``` -|-- /docs - |-- accessibility.en.md - |-- accessibility.es.md - |-- accessibility.fr.md - |-- accessibility.ja.md - |-- accessibility.zh-Hans.md - |-- accessibility.zh-Hant.md - |-- links.en.md - |-- links.es.md - |-- links.fr.md - |-- links.ja.md - |-- links.zh-Hans.md - |-- links.zh-Hant.md - |-- ... -``` - -To prevent missing routes due to typos, we can also create a check in the build process to make sure that every file in a translated subdirectory has an equivalent English one. This script could also verify the structure of documents (for example, that all translations have the same sequence of headings). +Each language translation will be held in a separate repository copied over from the monorepo that contains only the `docs/` directory for markdown files and `data/` for YAML files, and any additional tooling such as linters. ### Maintainers @@ -103,7 +68,7 @@ Contributors who would like to be maintainers of a new language repo should crea * the [IETF language code](https://en.wikipedia.org/wiki/IETF_language_tag) of the language * the GitHub usernames of all proposed maintainers -in a yaml file like so: +in YAML like so: ``` name: Spanish @@ -116,27 +81,23 @@ maintainers: When accepted, a script should: -* Create a new folder in the repo (e.g. `/translations/es`) with a template README and style guide -* Edit `CODEOWNERS.md` adding the proposed maintainers (either directly or through creating a new team to the Gatsby org) -* Create a new issue for tracking translation progress, and a new label (e.g. `translation-es`). - -The script should *not* copy over any files from the `/docs` directory. This should be done as contributors make new translations and to prevent the repo from becoming bloated with copies. +* Create a new repository `gatsby-[code]` +* Create `CODEOWNERS` adding the proposed maintainers (either directly or through creating a new team to the Gatsby org) +* Create a new issue for tracking translation progress ### Organizing work -Each supported language should have its own GitHub label (for example `translation-es`) for all changes made to a file in its directory, so that it will be easy to search for issues relating to the language. - Each language will have a "progress tracking issue" with the prioritized list of pages to translate, much like [React](https://github.com/reactjs/id.reactjs.org/issues/1). Unlike React, the issue should ideally be automated (similar to [Renovate](https://github.com/gatsbyjs/gatsby/issues/16840)). ### Updating content When a translation of a page is completed, we still need to ensure that the translation remains updated when the English source is changed. To do that, we use a bot similar to [React](https://github.com/reactjs/de.reactjs.org/pull/88), but adapted for Gatsby's monorepo. -When a change is detected in the `/docs` directory, the bot should, for each language: +When a change is detected in the `/docs` directory of the monorepo, the bot should, for each language: * Determine the time a pull request by the bot has occurred * Run `git diff` on `/docs` from that hash -* Apply the patch on `/translations/[lang]` using `git apply` +* Apply the patch on the `gatsby-[lang]` repo using `git apply` * Commit all the merge conflicts and create a pull request (or some other way to annotate which lines have changed) * Assign the issue to the language maintainers @@ -175,11 +136,18 @@ However, Crowdin has its fair share of disadvantages as well. We had considered There are other SaaS platforms such as Transifex that fix some of Crowdin's issues. However, the core issue relies in that they are new systems that most developers are unfamiliar with. We are relying on developers from the Gatsby community rather than professional translators, and those developers tend to be more familiar with GitHub's UI. -### Other GitHub-based arrangements +### Separate Websites + +We could do what reactjs.org did and have each language be its own separate website (e.g. es.gatsbyjs.org). The advantage of this for React is that it was easier to get languages started quickly and make changes like RTL themselves. + +This doesn't quite work for Gatsby. Since we have a monorepo, it takes a lot longer to copy the entire website and all its requirements. We'd also like to natively support localized versions in a single site using the Gatsby APIs to take full advantage of Gatsby's features. + +### Everything in the Monorepo + +Another GitHub-based alternative would be to put everything inside the monorepo in separate directories. The problem with this is that it would add a lot of clutter to Gatsby issues and pull requests. It's also a lot harder to create automation to sync the website because of the way that Git works. -The monorepo isn't the only way to organize our code. We could do something like React and have each language be in a separate repo copy. This makes it easier to copy content and check for changes, since we can just use `git merge`. +By moving everything to different repos, translation maintainers can self-organize and implement whatever tools they need to get translations completed. -But this has its own drawbacks as well. Each language is its own website and we can't easily switch between languages. The rest of Gatsby's code is in a monorepo and has tooling to make it work, so it makes sense to continue with that paradigm. We'd also like to dogfood Gatsby's localization utilities such as [`gatsby-plugin-i18n`](https://www.gatsbyjs.org/docs/localization-i18n/#gatsby-plugin-i18n). ## Adoption Strategy