Skip to content

Commit

Permalink
Core: Add optional telemetry and crash reporting
Browse files Browse the repository at this point in the history
  • Loading branch information
shilman committed Apr 28, 2022
1 parent 4acf6c0 commit 5f9f6f3
Show file tree
Hide file tree
Showing 92 changed files with 2,800 additions and 176 deletions.
2 changes: 2 additions & 0 deletions __mocks__/fs-extra.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ function __setMockFiles(newMockFiles) {
const readFile = async (filePath) => mockFiles[filePath];
const readFileSync = (filePath = '') => mockFiles[filePath];
const existsSync = (filePath) => !!mockFiles[filePath];
const readJsonSync = (filePath = '') => JSON.parse(mockFiles[filePath]);
const lstatSync = (filePath) => ({
isFile: () => !!mockFiles[filePath],
});
Expand All @@ -23,6 +24,7 @@ const lstatSync = (filePath) => ({
fs.__setMockFiles = __setMockFiles;
fs.readFile = readFile;
fs.readFileSync = readFileSync;
fs.readJsonSync = readJsonSync;
fs.existsSync = existsSync;
fs.lstatSync = lstatSync;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ export const buildStandaloneErrorHandler = (error: any): any => {
logger.line();
return error.close
? dedent`
FATAL broken build!, will close the process,
Fix the error below and restart storybook.
`
FATAL broken build!, will close the process,
Fix the error below and restart storybook.
`
: dedent`
Broken build, fix the error above.
You may need to refresh the browser.
`;
Broken build, fix the error above.
You may need to refresh the browser.
`;
};
16 changes: 9 additions & 7 deletions docs/api/cli-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@ Usage: start-storybook [options]
| `-V`, `--version` | Output the version number <br/>`start-storybook -V` |
| `-p`, `--port [number]` | Port to run Storybook <br/>`start-storybook -p 9009` |
| `-h`, `--host [string]` | Host to run Storybook <br/>`start-storybook -h my-host.com` |
| `-s`, `--static-dir` | **Deprecated** [see note](#static-dir-deprecation). Directory where to load static files from, comma-separated list <br/>`start-storybook -s public` |
| `-s`, `--static-dir` | **Deprecated** [see note](#static-dir-deprecation). Directory where to load static files from, comma-separated list<br/>`start-storybook -s public` |
| `-c`, `--config-dir [dir-name]` | Directory where to load Storybook configurations from <br/>`start-storybook -c .storybook` |
| `--https` | Serve Storybook over HTTPS. Note: You must provide your own certificate information. <br/>`start-storybook --https` |
| `--ssl-ca` | Provide an SSL certificate authority. (Optional with --https, required if using a self-signed certificate) <br/>`start-storybook --ssl-ca my-certificate` |
| `--https` | Serve Storybook over HTTPS. Note: You must provide your own certificate information<br/>`start-storybook --https` |
| `--ssl-ca` | Provide an SSL certificate authority. (Optional with --https, required if using a self-signed certificate)<br/>`start-storybook --ssl-ca my-certificate` |
| `--ssl-cert` | Provide an SSL certificate. (Required with --https)<br/>`start-storybook --ssl-cert my-ssl-certificate` |
| `--ssl-key` | Provide an SSL key. (Required with --https)<br/>`start-storybook --ssl-key my-ssl-key` |
| `--smoke-test` | Exit after successful start<br/>`start-storybook --smoke-test` |
Expand All @@ -32,7 +32,8 @@ Usage: start-storybook [options]
| `--debug-webpack` | Display final webpack configurations for debugging purposes<br/>`start-storybook --debug-webpack` |
| `--webpack-stats-json` | Write Webpack Stats JSON to disk<br/>`start-storybook --webpack-stats-json /tmp/webpack-stats` |
| `--docs` | Starts Storybook in documentation mode. Learn more about it in [here](../writing-docs/build-documentation.md#preview-storybooks-documentation)<br/>`start-storybook --docs` |
| `--no-manager-cache` | Disables Storybook's manager caching mechanism. See note below.<br/>`start-storybook --no-manager-cache` |
| `--no-manager-cache` | Disables Storybook's manager caching mechanism. See note below<br/>`start-storybook --no-manager-cache` |
| `--disableTelemetry` | Disables Storybook's telemetry. Learn more about it [here](../configure/telemetry.md)<br/>`start-storybook --disableTelemetry` |

<div class="aside">
💡 The flag <code>--no-manager-cache</code> disables the internal caching of Storybook and can severely impact your Storybook loading time, so only use it when you need to refresh Storybook's UI, such as when editing themes.
Expand All @@ -54,16 +55,17 @@ Usage: build-storybook [options]
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-h`, `--help` | Output usage information<br/>`build-storybook --help` |
| `-V`, `--version` | Output the version number<br/>`build-storybook -V` |
| `-s`, `--static-dir` | **Deprecated** [see note](#static-dir-deprecation). Directory where to load static files from, comma-separated list<br/>`build-storybook -s public` |
| `-s`, `--static-dir` | **Deprecated** [see note](#static-dir-deprecation).<br/> Directory where to load static files from, comma-separated list<br/>`build-storybook -s public` |
| `-o`, `--output-dir [dir-name]` | Directory where to store built files<br/>`build-storybook -o /my-deployed-storybook` |
| `-c`, `--config-dir [dir-name]` | Directory where to load Storybook configurations from<br/>`build-storybook -c .storybook` |
| `--loglevel [level]` | Controls level of logging during build. Can be one of: [silly, verbose, info (default), warn, error, silent]<br/>`build-storybook --loglevel warn` |
| `--loglevel [level]` | Controls level of logging during build.<br/> Available options: `silly`, `verbose`, `info` (default), `warn`, `error`, `silent`<br/>`build-storybook --loglevel warn` |
| `--quiet` | Suppress verbose build output<br/>`build-storybook --quiet` |
| `--no-dll` | Do not use dll reference (no-op)<br/>`build-storybook --no-dll` |
| `--debug-webpack` | Display final webpack configurations for debugging purposes<br/>`build-storybook --debug-webpack` |
| `--webpack-stats-json` | Write Webpack Stats JSON to disk<br/>`build-storybook --webpack-stats-json /my-storybook/webpack-stats` |
| `--docs` | Builds Storybook in documentation mode. Learn more about it in [here](../writing-docs/build-documentation.md#publish-storybooks-documentation)<br/>`build-storybook --docs` |
| `--disableTelemetry` | Disables Storybook's telemetry. Learn more about it [here](../configure/telemetry.md).<br/>`build-storybook --disableTelemetry` |

<div class="aside">
💡 If you're using npm instead of yarn to publish Storybook, the commands work slightly different. For example, <code>npm run build-storybook -- -o ./path/to/build</code>.
</div>
</div>
174 changes: 174 additions & 0 deletions docs/configure/telemetry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
---
title: 'Telemetry'
---

Storybook collects completely anonymous data to help us improve user experience. Participation in this anonymous program is optional, and you may opt-out if you'd not like to share any information.

## Why is telemetry collected?

Hundreds of thousands of developers use Storybook daily to build, test, and document components. Storybook is framework agnostic and integrates with the front-end ecosystem:

- **JavaScript frameworks** such as [React](https://reactjs.org/), [Vue](https://vuejs.org/), and [Svelte](https://svelte.dev/)
- **Libraries** such as [Styled-Components](https://styled-components.com/), [Tailwind](https://tailwindcss.com/), [Redux](https://redux.js.org/)
- **Design tools** such as [Figma](https://figma.com/), [Sketch](https://www.sketch.com/), [Zeplin](https://zeplin.io/) and [InVision](https://www.invisionapp.com/)
- **Workflow tools** such as [Notion](https://www.notion.so/product), [Confluence](https://www.atlassian.com/software/confluence), and [Jira](https://www.atlassian.com/software/jira)

In the past, our improvement process relied on manually gathering feedback. But with a growing userbase and the need to support a wide variety of integrations, we need a more accurate method for gauging Storybook usage and pain points.

These telemetry data help us (the maintainers) to prioritize the highest impact projects. That allows us to keep up with trends in the front-end ecosystem and verify that our community's hard work achieves the intended result.

## What is being collected?

We collect general usage details, including command invocation, Storybook version, addons, and the view layer.

Specifically, we track the following information in our telemetry events:

- Timestamp of the occurrence.
- Command invoked (e.g., `init`, `upgrade`, `start-storybook`).
- Storybook unique identifier: One-way hash generated during Storybook installation process.
- One way hash of the IP address where the event occurred.
- Story count.
- Storybook version.
- Storybook metadata:
- Language (e.g., TypeScript, JavaScript).
- Supported view layers (e.g., React, Vue, Angular, Svelte).
- Builder (e.g., Webpack4, Webpack5, Vite).
- Meta framework (e.g., [Next](https://nextjs.org/), [Gatsby](https://www.gatsbyjs.com/), [CRA](https://create-react-app.dev/)).
- [Addons](/addons) (e.g., [Essentials](../essentials/introduction), [Accessibility](https://storybook.js.org/addons/@storybook/addon-a11y/)).
- [Feature flags](./overview.md#feature-flags) (e.g., `buildStoriesJson`).
- Whether the command was invoked on CI or not.

The access to the raw data is highly controlled, and we cannot identify individual users from the dataset. It is anonymized and untraceable back to the user.

## What about sensitive information?

We take your privacy and our security very seriously. We perform additional steps to ensure that secure data (e.g., environment variables or other forms of sensitive data) **do not** make their way into our analytics. You can view all the information we collect by setting the `STORYBOOK_TELEMETRY_DEBUG` to `1` to print out the information gathered. For example:

<!-- prettier-ignore-start -->

<CodeSnippets
paths={[
'common/storybook-telemetry-preview-event.yarn.js.mdx',
'common/storybook-telemetry-preview-event.npm.js.mdx',
]}
/>

<!-- prettier-ignore-end -->

Will generate the following output:

```json
{
"anonymousId": "8bcfdfd5f9616a1923dd92adf89714331b2d18693c722e05152a47f8093392bb",
"eventType": "start",
"payload": {
"storyIndex": {
"storyCount": 4,
"version": 3
}
},
"inCI": false,
"metadata": {
"generatedAt": 1648233198722,
"builder": {
"name": "webpack4"
},
"hasCustomBabel": false,
"hasCustomWebpack": true,
"hasStaticDirs": true,
"hasStorybookEslint": false,
"refCount": 0,
"metaFramework": {
"name": "CRA",
"packageName": "react-scripts",
"version": "4.0.3"
},
"features": {
"buildStoriesJson": true
},
"storybookVersion": "6.5.0",
"language": "typescript",
"storybookPackages": {
"@storybook/addons": {
"version": "6.5.0"
},
"@storybook/builder-webpack4": {
"version": "6.5.0"
},
"@storybook/react": {
"version": "6.5.0"
}
},
"framework": {
"name": "react"
},
"addons": {
"@storybook/preset-create-react-app": {
"version": "3.2.0"
},
"@storybook/addon-ie11": {
"version": "0.0.7--canary.5e87b64.0"
},
"@storybook/addon-essentials": {
"options": {
"viewport": false
},
"version": "6.5.0"
}
}
}
}
```

## Will this data be shared?

The data we collect is anonymous, not traceable to the source, and only meaningful in aggregate form. No data we collect is personally identifiable.
In the future, we plan to share relevant data with the community through public dashboards (or similar data representation formats).

## How to opt-out

You may opt-out of the telemetry by setting Storybook's configuration element `disableTelemetry` to `true`, using the `--disableTelemetry` flag, or setting the environment variable`STORYBOOK_DISABLE_TELEMETRY` to `1`. For example:

<!-- prettier-ignore-start -->

<CodeSnippets
paths={[
'common/storybook-main-disable-telemetry.main-js.js.mdx',
'common/storybook-main-disable-telemetry.main-ts.ts.mdx',
'common/storybook-disable-telemetry-flag.yarn.js.mdx',
'common/storybook-disable-telemetry-flag.npm.js.mdx',
'common/storybook-disable-telemetry-env.env-var.js.mdx',
]}
/>

<!-- prettier-ignore-end -->

## Crash reports (disabled by default)

In addition to general usage telemetry, you may also choose to share crash reports. Storybook will then sanitize the error object (removing all user paths) and append it to the telemetry event. To enable crash reporting, you can set the `enableCrashReports` configuration element to `true`, using the `--enable-crash-reports` flag, or set the `STORYBOOK_ENABLE_CRASH_REPORTS` environment variable to `1`. For example:

<!-- prettier-ignore-start -->

<CodeSnippets
paths={[
'common/storybook-telemetry-main-enable-crash-reports.main-js.js.mdx',
'common/storybook-telemetry-main-enable-crash-reports.main-ts.ts.mdx',
'common/storybook-telemetry-storybook-enable-crash-reports.yarn.js.mdx',
'common/storybook-telemetry-storybook-enable-crash-reports.npm.js.mdx',
'common/storybook-telemetry-storybook-enable-crash-reports.env-var.js.mdx',
]}
/>

<!-- prettier-ignore-end -->

Generates the following item in the telemetry event:

<!-- prettier-ignore-start -->

<CodeSnippets
paths={[
'common/storybook-telemetry-crash-report-event.js.mdx',
]}
/>

<!-- prettier-ignore-end -->
3 changes: 2 additions & 1 deletion docs/get-started/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ The command above will make the following changes to your local environment:
- 🛠 Setup the necessary scripts to run and build Storybook.
- 🛠 Add the default Storybook configuration.
- 📝 Add some boilerplate stories to get you started.
- 📡 Set up telemetry to help us improve Storybook. Read more about it [here](../configure/telemetry.md).

Depending on your framework, first, build your app and then check that everything worked by running:

Expand Down Expand Up @@ -92,4 +93,4 @@ If all else fails, try asking for [help](https://storybook.js.org/support)

</details>

Now that you installed Storybook successfully, let’s take a look at a story that was written for us.
Now that you installed Storybook successfully, let’s take a look at a story that was written for us.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```shell
STORYBOOK_DISABLE_TELEMETRY=1 yarn storybook
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```shell
npm run storybook -- --disableTelemetry
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```shell
yarn storybook --disableTelemetry
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
```js
// .storybook/main.js

module.exports = {
stories: ['../src/**/*.stories.mdx', '../src/**/*.stories.@(js|jsx|ts|tsx)'],
addons: [
'@storybook/addon-links',
'@storybook/addon-essentials',
'@storybook/addon-interactions',
'@storybook/preset-create-react-app',
],
framework: '@storybook/react',
core: {
builder: 'webpack5',
disableTelemetry: true, // 👈 Disables telemetry
},
};
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
```ts
// .storybook/main.ts

// Imports Storybook's configuration API
import type { StorybookConfig } from '@storybook/core-common';

const config: StorybookConfig = {
stories: ['../src/**/*.stories.mdx', '../src/**/*.stories.@(js|jsx|ts|tsx)'],
addons: [
'@storybook/addon-links',
'@storybook/addon-essentials',
'@storybook/addon-interactions',
'@storybook/preset-create-react-app',
],
framework: '@storybook/react',
core: {
builder: 'webpack5',
disableTelemetry: true, // 👈 Disables telemetry
},
};

module.exports = config;
```
13 changes: 13 additions & 0 deletions docs/snippets/common/storybook-telemetry-crash-report-event.js.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
```js
{
stack: 'Error: Your button is not working\n' +
' at Object.<anonymous> ($SNIP/test.js:39:27)\n' +
' at Module._compile (node:internal/modules/cjs/loader:1103:14)\n' +
' at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)\n' +
' at Module.load (node:internal/modules/cjs/loader:981:32)\n' +
' at Function.Module._load (node:internal/modules/cjs/loader:822:12)\n' +
' at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)\n' +
' at node:internal/main/run_main_module:17:47',
message: 'Your button is not working'
}
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
```js
// .storybook/main.js

module.exports = {
stories: ['../src/**/*.stories.mdx', '../src/**/*.stories.@(js|jsx|ts|tsx)'],
addons: [
'@storybook/addon-links',
'@storybook/addon-essentials',
'@storybook/addon-interactions',
'@storybook/preset-create-react-app',
],
framework: '@storybook/react',
core: {
builder: 'webpack5',
enableCrashReports: true, // 👈 Appends the crash reports to the telemetry events
},
};
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
```ts
// .storybook/main.ts

// Imports Storybook's configuration API
import type { StorybookConfig } from '@storybook/core-common';

const config: StorybookConfig = {
stories: ['../src/**/*.stories.mdx', '../src/**/*.stories.@(js|jsx|ts|tsx)'],
addons: [
'@storybook/addon-links',
'@storybook/addon-essentials',
'@storybook/addon-interactions',
'@storybook/preset-create-react-app',
],
framework: '@storybook/react',
core: {
builder: 'webpack5',
enableCrashReports: true, // 👈 Appends the crash reports to the telemetry events
},
};

module.exports = config;
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```shell
STORYBOOK_TELEMETRY_DEBUG=1 npm run storybook
```
Loading

0 comments on commit 5f9f6f3

Please sign in to comment.