Use Monorepo tools #148

adamziel · 2023-03-10T15:36:24Z

Description

Setting up the build and dev mode of separate packages in a monorepo is really challenging. The current amalgamation of vite, esbuild, and rollup doesn't really cut it. Every package has a separate build config with its own resolve.alias, external, and viteStaticCopy setup.

Vite doesn't seem to support merging multiple configurations so the monorepo dependencies must be either:

Built before the dev script can be started, ideally in a watch mode (which is slow in Vite)
Aliased inside the Playground's vite.config.ts and configured again like in their original vite.config.js

Proposed solution

Let's explore these tools created specifically for managing monorepos and use one of them here.

Requirements:

Build all packages in the correct order
Fast dev mode without prior build, edit any file in the repo and see the outcome right away.
Dev mode setup should reflect all parts of the build setup like copied static assets. If it works in the dev mode, it should work in the production mode with near-100% certainty.
Dev mode should use the bundled source and types and also support source maps

The text was updated successfully, but these errors were encountered:

eliot-akira · 2023-03-12T11:35:43Z

Gutenberg uses Lerna for publishing multiple packages from a single repo. Maybe you've already considered it, or it might not suit all requirements of the Playground - but I've seen it used in a number of projects, so it seems to be a fairly standard tool for this purpose.

https://lerna.js.org/

https://github.com/lerna/lerna

## Description Sets up the correct build pipeline for all parts of Playground and PHP.wasm. This enables a public release of the [Playground API](#149) npm package! I've been [struggling](#146) with [this](#70) for [a while](#150) and couldn't understand what's so hard. NX made it apparent – look at this dependency graph! <img width="1291" alt="CleanShot 2023-03-16 at 23 16 26@2x" src="https://user-images.githubusercontent.com/205419/225764795-7fa8e972-09f8-41ef-aac2-1c96bd100ea0.png"> No wonder it's been almost impossible to set everything up by hand! ## Usage Start with `yarn install` ### Shortcuts To start a copy of `wasm.wordpress.net` locally, run `yarn run dev`. To build it, run `yarn run build`. ### Fully qualified commands In reality, these `yarn run` commands are just triggering the underlying project's nx `dev` and `build` commands: ```bash nx dev playground-website nx build playground-website ``` Here's a few more interesting commands: ```bash # Build and run PHP.wasm CLI nx start php-wasm-cli # Build latest WordPress releases nx recompile-wordpress:all playground-remote # Recompile PHP 5.6 - 8.2 releases to .wasm for web nx recompile-php:all php-wasm-web # Recompile PHP 5.6 - 8.2 releases to .wasm for node nx recompile-php:all php-wasm-node # Builds markdown files for readthedocs site nx build docs-site # Builds the Playground Client npm package nx build playground-client ``` ## NX is the tool Playground needed from the outset It's ridiculous how many problems this solves: * The build pipeline is explicitly defined and easy to modify * Tasks only run once their dependencies are ready * The dev mode works and is fast * The build works and is fast * We get CI checks to confirm the entire build process still works (which solves #150) * Cross-package TypeScript just works * There are linters and formatters (which solves #14) * Documentation is correctly generated from the latest built artifacts * There are nice generators for bootstraping new packages and moving the existing ones around * There are checks to ensure the private `php-wasm-common` package is not imported by anything else than `php-wasm-web` and `php-wasm-node` ## Next steps * Add Lerna to harness package publishing * Additional developer documentation for the nx-based flow Related to #148 and #147

adamziel · 2023-04-01T12:52:58Z

I just tried using Lerna and found an issue. lerna publish bumps a package version, but it does so in the source directory and not the dist directory. The packages are build before calling lerna publish. I have no solution yet.

Edit: Found a solution: lerna publish from-package. Now Lerna is unhappy about uncommitted changes: Working tree has uncommitted changes

Edit 2: I found a solution to that as well – the build needs to happen during the "prepublishOnly" npm hook.

## Description It's hard to publish all the npm packages living in this repo without a tool. After considering [changesets](https://github.com/changesets/changesets) and [Lerna](https://lerna.js.org/) this commit moves forward the latter since it's maintained by the authors of nx – another tool used in this project. Solves #148 This commit also bundles wp-playground/client dependencies This is useful in itself and makes the client package self-contained. In this instance we need it to avoid producing an invalid list of dependencies in the final package.json. This package technically depends on php-wasm/web, but takes great care not to import anything from it other than TypeScript types that are then released in a rollup file.

adamziel · 2023-04-01T15:02:01Z

Soled in #151 and #184. This project now uses nx and Lerna.

## Description Sets up the correct build pipeline for all parts of Playground and PHP.wasm. This enables a public release of the [Playground API](WordPress/wordpress-playground#149) npm package! I've been [struggling](WordPress/wordpress-playground#146) with [this](WordPress/wordpress-playground#70) for [a while](WordPress/wordpress-playground#150) and couldn't understand what's so hard. NX made it apparent – look at this dependency graph! <img width="1291" alt="CleanShot 2023-03-16 at 23 16 26@2x" src="https://user-images.githubusercontent.com/205419/225764795-7fa8e972-09f8-41ef-aac2-1c96bd100ea0.png"> No wonder it's been almost impossible to set everything up by hand! ## Usage Start with `yarn install` ### Shortcuts To start a copy of `wasm.wordpress.net` locally, run `yarn run dev`. To build it, run `yarn run build`. ### Fully qualified commands In reality, these `yarn run` commands are just triggering the underlying project's nx `dev` and `build` commands: ```bash nx dev playground-website nx build playground-website ``` Here's a few more interesting commands: ```bash # Build and run PHP.wasm CLI nx start php-wasm-cli # Build latest WordPress releases nx recompile-wordpress:all playground-remote # Recompile PHP 5.6 - 8.2 releases to .wasm for web nx recompile-php:all php-wasm-web # Recompile PHP 5.6 - 8.2 releases to .wasm for node nx recompile-php:all php-wasm-node # Builds markdown files for readthedocs site nx build docs-site # Builds the Playground Client npm package nx build playground-client ``` ## NX is the tool Playground needed from the outset It's ridiculous how many problems this solves: * The build pipeline is explicitly defined and easy to modify * Tasks only run once their dependencies are ready * The dev mode works and is fast * The build works and is fast * We get CI checks to confirm the entire build process still works (which solves #150) * Cross-package TypeScript just works * There are linters and formatters (which solves #14) * Documentation is correctly generated from the latest built artifacts * There are nice generators for bootstraping new packages and moving the existing ones around * There are checks to ensure the private `php-wasm-common` package is not imported by anything else than `php-wasm-web` and `php-wasm-node` ## Next steps * Add Lerna to harness package publishing * Additional developer documentation for the nx-based flow Related to WordPress/wordpress-playground#148 and WordPress/wordpress-playground#147

## Description It's hard to publish all the npm packages living in this repo without a tool. After considering [changesets](https://github.com/changesets/changesets) and [Lerna](https://lerna.js.org/) this commit moves forward the latter since it's maintained by the authors of nx – another tool used in this project. Solves WordPress/wordpress-playground#148 This commit also bundles wp-playground/client dependencies This is useful in itself and makes the client package self-contained. In this instance we need it to avoid producing an invalid list of dependencies in the final package.json. This package technically depends on php-wasm/web, but takes great care not to import anything from it other than TypeScript types that are then released in a rollup file.

adamziel mentioned this issue Mar 10, 2023

Migrate to Yarn workspaces #146

Closed

adamziel mentioned this issue Mar 16, 2023

Migrate to NX-based build #151

Merged

adamziel mentioned this issue Apr 1, 2023

Add Lerna #184

Merged

adamziel closed this as completed Apr 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use Monorepo tools #148

Use Monorepo tools #148

adamziel commented Mar 10, 2023

eliot-akira commented Mar 12, 2023 •

edited

Loading

adamziel commented Apr 1, 2023 •

edited

Loading

adamziel commented Apr 1, 2023 •

edited

Loading

Use Monorepo tools #148

Use Monorepo tools #148

Comments

adamziel commented Mar 10, 2023

Description

Proposed solution

Requirements:

eliot-akira commented Mar 12, 2023 • edited Loading

adamziel commented Apr 1, 2023 • edited Loading

adamziel commented Apr 1, 2023 • edited Loading

eliot-akira commented Mar 12, 2023 •

edited

Loading

adamziel commented Apr 1, 2023 •

edited

Loading

adamziel commented Apr 1, 2023 •

edited

Loading