Sort out API docs story

[Tyriar]

The .d.ts file is the API source of truth and the website falls behind and becomes stale easily. We should sort out some processes of syncing the two, maybe a script that translates the .d.ts to markdown in https://github.com/xtermjs/xtermjs.org/blob/master/_docs/api/Terminal.md?

It may also be worth calling out the .d.ts/website in the README

See #1454

[ghost]

I wanted to discuss this a bit more before I send a PR. Here's the WIP and preview.

The preview is the result of typedoc + typedoc-plugin-markdown + typedoc-mdmerge.
Adds three new dev-dependencies: gulp-typedoc, typedoc, and typedoc-plugin-markdown
I created typedoc-mdmerge to generate a single markdown document which avoids this file tree.

Examples can be embed in typings/xterm.d.ts using markdown syntax.

Do we want to merge the output? If so, where should typedoc-mdmerge.js live?
Should the addons documentation being generated in a similar way?
Do you have any other suggestions/improvements in mind for this?
What else should go in the PR? i.e. pre-commit npm-script, link in README.MD that you mentioned, etc.

[Tyriar]

@pro-src thanks for looking at this 😃

Examples can be embed in typings/xterm.d.ts using markdown syntax.

Great idea! Anything that encourages improving the .d.ts is good 🙂

Do we want to merge the output? If so, where should typedoc-mdmerge.js live?

It may actually be a good idea to split out the different interfaces into different pages. Currently we only have https://xtermjs.org/docs/api/terminal/, but maybe we could redirect that to /docs/api/classes/terminal and have the interfaces as well. Personally I find it hard to navigate API does that don't separate like this. Thoughts @xtermjs/core?

We would need to come up with a ToC for a better experience for this (could be deferred).

Should the addons documentation being generated in a similar way?

This would be great if we could do this, as you can see the docs have drifted far from the addons in the project. We may want to use a different doc generation strategy for addons, like generating the page by pulling in the README and then generate the types after. It might be a good idea to defer this as a follow up task to simplify the initial PR.

Do you have any other suggestions/improvements in mind for this?

I think we should move all the doc generation over to https://github.com/xtermjs/xtermjs.org. The script could download the .d.ts files for the version or just clone the whole repo before generating.

Can we tweak some typedoc options to change some of the way things are outputted? I find this particularly verbose and hard to read: https://github.com/pro-src/xterm.js/blob/documentation/docs/Terminal.md#interface-itheme. It's like 5-6 pages just for an interface containing a bunch of strings.

[ghost]

I haven't fully addressed all of the stuff you mentioned as of yet but I plan to very soon. For now I've created xtermjs.org/docs-generator.

Lemme know what you think about that please 😄

[ghost]

I'll give an overview of that tomorrow to provide a bit more insight without digging.

[Tyriar]

I notice there's a script to trigger a build, how does this work exactly? We only want to do this once per version. Also we might be moving away from Travis in the future so would prefer not to invest in tight integrations.

[ghost]

I don't think this would be a tight integration but is very useful if you decide to make use of it.

So what I've done is to separate the nodejs based project from the jekyll files by using a separate branch with it's own git history. Github uses this same technique to separate gh-pages from a project's master branch.

I then created an update script that first init/updates (xterm.js repo) the git-submodule in that branch then switches branches, updates _docs/, commits, and pushes. The update script also checks to see if it's being called on CI (appveyor, etc) and if the CI isn't Travis then terminates. Otherwise if on Travis Ci, it accounts for being in a detached head state and does the same as before. It requires a Github token to achieve this and will work on any CI worth mentioning, albeit with different environment variable checks.

The small script to trigger a build of xtermjs/xtermjs.org/docs-generatorsimply uses curl to call Travis CI's v3 API when a tagged commit is pushed to the master branch of xtermjs/xterm.js/master. This script will only run for commits by maintainers aka non-PR commits. The script should be added to the after_success script of the CI being used to test xterm.js however it currently checks the environment variables of Travis CI as that is the CI being used. The trigger-build script, if desired to move away from CI in the future, can be used on other CIs just by changing/adding environment variable checks. It requires a Travis token to be added to xtermjs/xtermjs.org.

This is still a WIP, just wanting to get more feedback and the trigger-build either needs to be called in the deploy stage or an additional small check to prevent being called for each build env matrix permutation.

[ghost]

Also I should mention that the git-submodule is automatically initialized/updated whenever travis-ci clones the repo. That's a very nice feature but it's not really a strict dependency as we can do that ourselves with/without a submodule.

Edit: Also worth mentioning is that the triggered build of docs-generator is of the latest non-PR commit to that branch.

[ghost]

@Tyriar if any of the above isn't desired behavior for the initial PR let me know and I will defer or permanently stash it for reference.

Should there be a redirect from the old API docs if so are you familiar with jekyll-redirect-from? I've briefly experimented unsuccessfully with it.

I'm going to focus my efforts on the typedoc output after it's known how to proceed with auto? syncing and where the project files should live.

[ghost]

@Tyriar I've considered all of the typedoc and plugin options. If we want to change the resulting markdown, I think the best way is to fork typedoc-plugin-markdown and add our own options.

gulp.task('typedoc', function() {
  return gulp.src('./xterm.js/typings/xterm.d.ts')
  .pipe(typedoc({
    // TypeScript options (see typescript docs)
    module: 'commonjs',
    target: 'es5',

    // Output options (see typedoc docs)
    // The mode currently doesn't affect the output
    mode: 'file',
    out: './docs~temp/api/terminal/',
    // Zero output without this option being true
    includeDeclarations: true,
    // This option causes a werid bug when set to false
    excludeExternals: true,
    // Excludes private class members
    excludePrivate: true,
    // The branch or revision that will be used to create src links
    gitRevision: 'master',

    // TypeDoc options (see typedoc docs)
    // It's the (*maybe* after some processing) header
    // of ./docs~temp/api/terminal/README.md which
    // is the top level markdown document of the out dir.
    readme: './toc.md',
    // Required to output markdown
    theme: 'markdown',
    ignoreCompilerErrors: false
  }));
});

[ghost]

Alternatively we can experiment with jsdoc's JSON output and maybe generate markdown from that.

[Tyriar]

So what I've done is to separate the nodejs based project from the jekyll files by using a separate branch with it's own git history.

It might be better to stay away from this approach as the branch is difficult to find. If the site is jekyll we can put files/folders in the _config.yml to exclude it on the deployed site. Building to a sibling folder should be easier than another branch too.

Should there be a redirect from the old API docs

I'm actually not that concerned about a redirect, I doubt there are many links out there to the docs. You should be able to do it without a plugin though by adding a html file to the location like this:

<meta http-equiv="refresh" content="0; url=<url>/">
<link rel="canonical" href="<url>" />

That's how I do it on my website.

typedoc-plugin-markdown

Definitely don't want to fork anything. Then we have to maintain it, rather than just update the dep.

---

Feedback on branch:

• I don't think you need this https://github.com/pro-src/xtermjs.org/blob/f43f8f4e4759f0ae069a890cd4bebfe4b035cd13/package.json#L15
• On triggering the build, we only want this to happen when a new build comes out, not on every Travis CI build. What if we checked for a newer version using the tags? New versions will always be tagged with x.y.z

[ghost]

Definitely don't want to fork anything. Then we have to maintain it, rather than just update the dep.

I decided to just patch the typedoc output

I don't think you need this https://github.com/pro-src/xtermjs.org/blob/f43f8f4e4759f0ae069a890cd4bebfe4b035cd13/package.json#L15

Nice catch, I removed the unnecessary dependencies and added fs-extra.

On triggering the build, we only want this to happen when a new build comes out, not on every Travis CI build. What if we checked for a newer version using the tags? New versions will always be tagged with x.y.z

Presently, the build is only triggered for tagged commits and will only update for unique tags.

I've yet to do anything about the verbosity of the interface docs. I could just patch that too. Also note that the github src links can be disabled via an option in the typedoc-plugin-markdown.

There is something wrong with the deployment. /docs results in a 404. I'm sure it's something simple and will try to diagnose a bit later.