Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(learn): add article for publishing a typescript package #7279

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
b20dcbf
feat(learn): add article for publishing a typescript package
JakobJingleheimer Nov 23, 2024
2d5d359
WIP: initial content for article
JakobJingleheimer Dec 29, 2024
86394b6
WIP: polish sample code, & dir overviews
JakobJingleheimer Dec 29, 2024
d583e03
rename article to be more specific
JakobJingleheimer Dec 29, 2024
48e2ad9
fix unsuported lang
AugustinMauroy Dec 29, 2024
081b7f2
fix navigation.json
JakobJingleheimer Dec 29, 2024
2a20202
fix links
JakobJingleheimer Dec 30, 2024
52d3a13
extract note from codeblock into article
JakobJingleheimer Dec 30, 2024
2228ab3
tidy codeblocks
JakobJingleheimer Dec 30, 2024
5034918
wordsmith
JakobJingleheimer Dec 30, 2024
ddb1cf1
fixup!: remove controversial "optionalDependencies"
JakobJingleheimer Jan 6, 2025
9258389
fixup!: wordsmith & align code samples
JakobJingleheimer Jan 8, 2025
4906609
fixup!: tsconfig
JakobJingleheimer Jan 8, 2025
07095c6
fixup!: switch sequence of repo vs package
JakobJingleheimer Jan 8, 2025
33e744a
fixup!: note types and unit tests are complementary
JakobJingleheimer Jan 8, 2025
f1703ad
fixup!: `IDE` → `editor`
JakobJingleheimer Jan 8, 2025
3fd8076
fixup!: note file extensions in package.json fields (js vs ts)
JakobJingleheimer Jan 9, 2025
64fd531
fixup!: add alternative samples & configs
JakobJingleheimer Jan 13, 2025
7305f8e
fixup!: remove version from npm links
JakobJingleheimer Jan 14, 2025
13b8e1c
fixup!: shorter code sample display names
JakobJingleheimer Jan 14, 2025
0f7f993
fixup!: add note about `NPM_TOKEN`
JakobJingleheimer Jan 15, 2025
53793f1
fixup!: switch node version matrix to LTS matrix action
JakobJingleheimer Jan 15, 2025
4b06b6c
fixup!: shorten displayNames (they were breaking page layout)
JakobJingleheimer Jan 15, 2025
d250a86
fixup!: update references to samples
JakobJingleheimer Jan 15, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions apps/site/navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,10 @@
"runNatively": {
"link": "/learn/typescript/run-natively",
"label": "components.navigation.learn.typescript.links.runNatively"
},
"publishingTSProject": {
"link": "/learn/typescript/publishing-a-ts-project",
"label": "components.navigation.learn.typescript.links.publishingTSProject"
}
}
},
Expand Down
288 changes: 288 additions & 0 deletions apps/site/pages/en/learn/typescript/publishing-a-ts-project.md
himself65 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,288 @@
---
title: Publishing a TypeScript project
layout: learn
authors: JakobJingleheimer
---

# Publishing a TypeScript project

This article augments TypeScript's own [Publishing guide](https://www.typescriptlang.org/docs/handbook/declaration-files/publishing.html) with specifics for native node support.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sort of questioning linking to this page; reading it, it's pretty out of date and is part of the declaration file section, so sort of misses out on other important stuff.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to change it to a different one—is there one you had in mind?


Some important things to note:

- Everything from [Publishing a package](../modules/publishing-a-package) applies here.

- Fields like `main` operate on _published_ content, so when TypeScript source-code is transpiled to JavaScript, JavaScript is the published content and `main` would point to a JavaScript file with a JavaScript file extension (ex `main.ts` → `"main": "main.js"`).

- Fields like `scripts.test` operate on source-code, so they would use the file extensions of the source code (ex `"test": "node --test './src/**/*.test.ts'`).

- Node runs TypeScript code via a process called "[type stripping](https://nodejs.org/api/typescript.html#type-stripping)", wherein node (via [Amaro](https://github.com/nodejs/amaro)) removes TypeScript-specific syntax, leaving behind vanilla JavaScript (which node already understands). This behaviour is enabled by default as of node version 23.6.0.

- Node does **not** strip types in `node_modules` because it can cause significant performance issues for the official TypeScript compiler (`tsc`) and parts of VS Code, so the TypeScript maintainers would like to discourage people publishing raw TypeScript, at least for now.
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved

- Consuming TypeScript-specific features like `enum` in node still requires a flag ([`--experimental-transform-types`](https://nodejs.org/api/typescript.html#typescript-features)). There are often better alternatives for these anyway.

- Use [dependabot](https://docs.github.com/en/code-security/dependabot) to keep your dependencies current, including those in github actions. It's a very easy set-and-forget configuration.

- `.nvmrc` comes from [NVM](https://github.com/nvm-sh/nvm), a multi-version manager for node. It allows you to specify the version of node the project should generally use.

A repository would look something like:

```text displayName="Source of the example TypeScript package (directory overview)"
example-ts-pkg/
├ .github/
├ workflows/
├ ci.yml
└ publish.yml
└ dependabot.yml
├ src/
├ foo.fixture.js
├ main.ts
├ main.test.ts
├ some-util.ts
└ some-util.test.ts
├ LICENSE
├ package.json
├ README.md
└ tsconfig.json
```

```text displayName="(alt 1) Source of the example TypeScript package (directory overview)"
example-ts-pkg/
├ .github/
├ workflows/
├ ci.yml
└ publish.yml
└ dependabot.yml
├ src/
├ __test__
├ foo.fixture.js
├ main.test.ts
├ main.ts
└ some-util.ts
├ __test__
└ some-util.test.ts
└ some-util.ts
├ LICENSE
├ package.json
├ README.md
└ tsconfig.json
```

```text displayName="(alt 2) Source of the example TypeScript package (directory overview)"
example-ts-pkg/
├ .github/
├ workflows/
├ ci.yml
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
└ publish.yml
└ dependabot.yml
├ src/
├ main.ts
├ some-util.ts
├ test/
├ foo.fixture.js
├ main.ts
└ some-util.ts
├ LICENSE
├ package.json
├ README.md
└ tsconfig.json
```

And its published package would look something like:

```text displayName="Published example TypeScript package (directory overview)"
example-ts-pkg/
├ LICENSE
├ main.d.ts
├ main.d.ts.map
├ main.js
├ package.json
├ README.md
├ some-util.d.ts
├ some-util.d.ts.map
└ some-util.js
```

```text displayName="(alt) Published example TypeScript package (directory overview)"
example-ts-pkg/
├ dist/
├ main.d.ts
├ main.d.ts.map
├ main.js
├ some-util.d.ts
├ some-util.d.ts.map
└ some-util.js
├ LICENSE
├ package.json
└ README.md
```

A note about directory organisation: There are a few common practices for placing tests. Principle of least knowledge says to co-locate them (put them adjacent to implementation). Sometimes, that's in the same directory, or within a drawer like a `__test__` (also adjacent to the implementation, "alt 1"). Alternatively, some opt to create a `test/` sibling to `src/` ("alt 2"), either with a mirrored structure or a "junk drawer".

## What to do with your types

### Treat types like a test
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved

The purpose of types is to warn an implementation will not work:

```ts
const foo = 'a';
const bar: number = 1 + foo;
// ^^^ Type 'string' is not assignable to type 'number'.
```

TypeScript has warned that the above code will not behave as intended, just like a unit test warns that code does not behave as intended. They are complementary and verify different things—you should have both.

Your editor (ex VS Code) likely has built-in support for TypeScript, displaying errors as you work. If not, and/or you missed those, CI will have your back.

The following [GitHub Action](https://github.com/features/actions) sets up a CI task to automatically check (and require) types pass inspection for a PR into the `main` branch.

```yaml displayName=".github/workflows/ci.yml"
name: Tests

on:
pull_request:
branches: ['main']

jobs:
check-types:
# Separate these from tests because
# they are platform and node-version independent
# and need be run only once.

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: '.nvmrc'
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
cache: 'npm'
- name: npm clean install
run: npm ci
# You may want to run a lint check here too
- run: node --run types:check

test:
runs-on: ubuntu-latest

strategy:
matrix:
node:
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
- version: 23.x
- version: 22.x
fail-fast: false # Prevent a failure in one version cancelling other runs

steps:
- uses: actions/checkout@v4
- name: Use node ${{ matrix.node.version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node.version }}
cache: 'npm'
- name: npm clean install
run: npm ci
- run: node --run test
```

```json displayName="package.json"
{
"name": "example-ts-pkg",
"scripts": {
"test": "node --test './src/**/*.test.ts'",
"types:check": "tsc --noEmit"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noted this in the other thread, but I would be cautious about this as a default; I really only see people setting noEmit when they're doing a quick check, or are using a bundler or something.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the type "test" command. Why would you want to emit a compilation?

Maybe the names could be better? When I have unit and end-to-end tests with different setups, I split those into different commands like:

  • test:unit
  • test:e2e

So in that scenario, it could make sense to name types:checktest:types.

But in the sample, there's no differentiation between units and e2e, so then what do I call what is currently test?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, I guess it's fine, I am just wary of cases where tsc and tsc --noEmit output different errors because the former is doing more. Maybe you'd hit it on prepack and that's okay, but it's a little unfortunate to only hit an error when you go to release...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that likely? I've been doing this for years and never encountered that—am I just very lucky?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how to gauge "likely", probably unlikely, but they're cases like "tsc failed to write the files", along with potentially some declaration transform errors. (The latter shouldn't actually end up mattering by my reading of the code, though.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The specific issues that come up that I can think of are when a declaration file can't reliably be generated because doing so might require referencing entities that are private or non-exported. Trying to figure out why this error is happening can be pretty frustrating, especially if you've been relying a specific pattern over time. Having a divergence between publish/CI probably just makes this even more confusing since most people outside of the person who set up the build won't be aware of any differences.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tsc failed to write the files

That does sound like an issue that you legitimately wouldn't catch with a dry-run, but also seems a very unlikely issue (and one that could still occur at publishing even if you were using non-dry-run for the test step). It seems an edge-case worth noting but not worth taking a hit every time to avoid something that likely will never happen (if it does, there are a very limited number of causes—two? permissions and storage availability).

The specific issues that come up that I can think of are when a declaration file can't reliably be generated because doing so might require referencing entities that are private or non-exported.

That sounds very detectable for --noEmit; if it doesn't do that, that sounds like a defect in tsc? Why would you need to writing to disk in order to discover it?


But maybe let's take a step back for a second: The reason I wrote the setup this way is because of performance—but perhaps my information is outdated. Last I heard, tsc --noEmit was significantly faster than with emit.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds very detectable for --noEmit; if it doesn't do that, that sounds like a defect in tsc? Why would you need to writing to disk in order to discover it?

I think as of recent releases (since --isolatedDeclarations), we've actually always checked declaration errors without emit, so I think the only errors one can see differently are just the errors that happen while writing, which may not really be important except in the case where you've somehow accidentally marked output paths as readonly in the FS or something.

But maybe let's take a step back for a second: The reason I wrote the setup this way is because of performance—but perhaps my information is outdated. Last I heard, tsc --noEmit was significantly faster than with emit.

It can be, though I think at the scale of this demo, it's definitely not a big difference.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the take-away then that what I have is fine? There is a potentially (and probably likely) significant perf savings, and there're basically no type-related errors this won't catch?

},
"devDependencies": {
"typescript": "^5.7.2"
}
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
}
```

```json displayName="tsconfig.json"
{
"compilerOptions": {
"allowArbitraryExtensions": true,
"declaration": true,
"declarationMap": true,
"lib": ["ESNext"],
"module": "NodeNext",
"outDir": "./",
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
"resolveJsonModule": true,
"rewriteRelativeImportExtensions": true
},
// These may be different for your repo:
DanielRosenwasser marked this conversation as resolved.
Show resolved Hide resolved
"include": ["./src"],
"exclude": ["**/*/*.test.*", "**/*.fixture.*"]
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
}
```

Note that test files may well have a different `tsconfig.json` applied (hence why they are excluded in the above sample).

### Generate type declarations

Type declarations (`.d.ts` and friends) provide type information as a sidecar file, allowing the execution code to be vanilla JavaScript whilst still having types.

Since these are generated based on source code, they can be built as part of your publication process and do not need to be checked into your repository.

Take the following example, where the type declarations are generated just before publishing to the NPM registry.

```yaml displayName=".github/workflows/publish.yml"
name: Publish to NPM
on:
push:
tags:
- '**@*'
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved

jobs:
build:
runs-on: ubuntu-latest

permissions:
contents: read
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: '.nvmrc'
registry-url: 'https://registry.npmjs.org'
- run: npm ci

# You can probably ignore the boilerplate config above

- name: Publish with provenance
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is NPM_TOKEN automatically provided somehow? I think we should mention how to get this or avoid automated publishing accordingly. having a separate guide on automated publishing is worthwhile.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

t's not provide automatically. But this GA example from https://docs.npmjs.com/generating-provenance-statements#example-github-actions-workflow

Copy link
Member Author

@JakobJingleheimer JakobJingleheimer Jan 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would we have enough content for a dedicated guide on automated publishing?

For now, I think let's add a note explaining what the token is and where it comes from: 0f7f993

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for note

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a ton of problems with automated publishing (discussed above) and I think a separate guide that can fully explain the pros and cons is definitely warranted.

Copy link
Member Author

@JakobJingleheimer JakobJingleheimer Jan 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, sure, happy to do subsequently (subsequently because otherwise we end up with a web of ½ finished, inter-dependent guides, and this doesn't seem like a show-stopper—unless someone strongly thinks that it needs to happen first).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made the case above that it would be better to give no publishing guidance prior to such a properly nuanced guide, then to provide instructions absent nuance - iow, I suggest removing this section entirely and hand-wave over publishing for the time being.

I don't think we can avoid interdependent guides; the reality is too complex to allow for a single "does everything" document imo.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not just redirect to npm docs ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh for sure they'll be inter-dependent. I just want to avoid a bunch of unfinished inter-dependent guides that are stuck in a spiderman meme.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So let's remove provenance and add note or let the security team write another article about it

run: npm publish --access public --provenance
```

```diff displayName="package.json"
{
"name": "example-ts-pkg",
"scripts": {
+ "prepack": "tsc",
"types:check": "tsc --noEmit"
}
}
```

```text displayName=".npmignore"
*.ts
!*.d.ts
*.fixture.*
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
```

```text displayName="(alt: dist/) .npmignore"
src
test
```

`npm publish` will automatically run [`prepack` beforehand](https://docs.npmjs.com/cli/v11/using-npm/scripts#npm-publish). `npm` will also run `prepack` automatically before `npm pack --dry-run` (so you can easily see what your published package will be without actually publishing it). **Beware**, [`node --run` does _not_ do that](../command-line/run-nodejs-scripts-from-the-command-line.md#using-the---run-flag). You can't use `node --run` for this step, so that caveat does not apply here, but it can for other steps.
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved

#### Breaking this down

Generating type declarations is deterministic: you'll get the same output from the same input, every time. So there is no need to commit these to git.

[`npm publish`](https://docs.npmjs.com/cli/v11/commands/npm-publish) grabs everything applicable and available at the moment the command is run; so generating type declarations immediately before means those are available and will get picked up.

By default, `npm publish` grabs (almost) everything (see [Files included in package](https://docs.npmjs.com/cli/v11/commands/npm-publish#files-included-in-package)). In order to keep your published package minimal (see the "Heaviest Objects in the Universe" meme about `node_modules`), you want to exclude certain files (like tests and test fixtures) from from packaging. Add these to the opt-out list specified in [`.npmignore`](https://docs.npmjs.com/cli/v11/using-npm/developers#keeping-files-out-of-your-package); ensure the `!*.d.ts` exception is listed, or the generated type declartions will not be published! Alternatively, you can use [package.json "files"](https://docs.npmjs.com/cli/v11/configuring-npm/package-json#files) to create an opt-in list.
Copy link
Member

@DanielRosenwasser DanielRosenwasser Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default, npm publish grabs (almost) everything (see Files included in package).

Similar to what @andrewbranch and @jakebailey said above, if you specify an --outDir, then you can use the package.json "files" array to avoid other hazards.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or .npmignore, which avoids hazards endemic to files :-)

2 changes: 2 additions & 0 deletions apps/site/shiki.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import powershellLanguage from 'shiki/langs/powershell.mjs';
import shellScriptLanguage from 'shiki/langs/shellscript.mjs';
import shellSessionLanguage from 'shiki/langs/shellsession.mjs';
import typeScriptLanguage from 'shiki/langs/typescript.mjs';
import yamlLanguage from 'shiki/langs/yaml.mjs';
import shikiNordTheme from 'shiki/themes/nord.mjs';

/**
Expand All @@ -29,6 +30,7 @@ export const LANGUAGES = [
...shellSessionLanguage,
...dockerLanguage,
...diffLanguage,
...yamlLanguage,
];

// This is the default theme we use for our Shiki Syntax Highlighter
Expand Down
3 changes: 2 additions & 1 deletion packages/i18n/locales/en.json
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@
"introduction": "Introduction to TypeScript",
"transpile": "Running TypeScript code using transpilation",
"run": "Running TypeScript with a runner",
"runNatively": "Running TypeScript Natively"
"runNatively": "Running TypeScript Natively",
"publishingTSProject": "Publishing a TypeScript project"
}
},
"asynchronousWork": {
Expand Down
Loading