-
Notifications
You must be signed in to change notification settings - Fork 275
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Blueprints: Directory Resources (#1793)
## Description
Adds a Directory resource type to enable loading file trees from git
repositories, local hard drive, deeply nested zip archives etc:
```json
{
"steps": [
{
"step": "installPlugin",
"pluginData": {
"resource": "git:directory",
"repositoryUrl": "https://github.com/WordPress/wordpress-playground.git",
"ref": "HEAD",
"path": "packages/docs"
}
},
{
"step": "installPlugin",
"pluginData": {
"resource": "literal:directory",
"name": "hello-world",
"files": {
"README.md": "Hello, World!",
"index.php": "<?php\n/**\n* Plugin Name: Hello World\n* Description: A simple plugin that says hello world.\n*/",
}
}
}
]
}
```
## Motivation
This PR opens the door to:
* Seamless Git integration. Import path mapping from git to Playground
is now just a few steps referencing specific git directories. No more
custom logic required!
* Blueprint-based site imports and exports without any Playground
webapp-specific logic.
* Runtime-specific "resource overrides", e.g.
`--resource-override=GUTENBERG:./gutenberg.zip` in CLI to test a
Blueprint with a my local version of Gutenberg. The same logic would be
used by the Blueprints builder to use files selected via `<input
type="file">` controls.
### Schema
Every step can declare which kinds of resources it accepts (file-based
resources vs directory-based resources). Using a single `pluginData`
property in the `installPlugin` step means less choices for the
developer. It also makes local resource overrides easy, e.g. we could
tell Playground CLI to load a local Gutenberg directory instead of a
remote Gutenberg zip. This wouldn't be as easy had we used separate
options for passing ZIP-based and directory-based resources.
On one hand, `pluginData` is less informative than `pluginZipFile`. On
the other, the name accommodates for non-zip resources such as
directories.
## Developer notes about specific API changes introduced in this PR
This PR changes introduces a new `literal:directory` resource that can
be used in Blueprints as follows:
```json
{
"steps": [
{
"step": "installPlugin",
"pluginData": {
"resource": "literal:directory",
"name": "hello-world",
"files": {
"README.md": "Hello, World!",
"index.php": "<?php\n/**\n* Plugin Name: Hello World\n* Description: A simple plugin that says hello world.\n*/",
}
}
}
]
}
```
Or via the JS API:
```ts
await installTheme(php, {
themeData: {
name: 'test-theme',
files: {
'index.php': `/**\n * Theme Name: Test Theme`,
},
},
ifAlreadyInstalled: 'overwrite',
options: {
activate: false,
},
});
```
It also introduces a new `writeFiles` step:
```ts
{
"steps": [
{
"step": "writeFiles",
"writeToPath": "/wordpress/wp-content/plugins/my-plugin",
"filesTree": {
"name": "my-plugin",
"files": {
"index.php": "<?php echo '<a>Hello World!</a>'; ?>",
"public": {
"style.css": "a { color: red; }"
}
}
}
}
]
}
```
Specific changes:
* Adds a `Resource<Directory>` resource type that provides a `name:
string` and `files: FileTree`.
* Renames `pluginZipFile` to `pluginData` in the `installPlugin` step
* Renames `themeZipFile` to `themeData` in the `installPlugin` step
* Adds a new `writeFiles` step for writing entire directory trees
* Adds a new `literal:directory` resource type where an entire file tree
can be specified inline
* Adds a new `git:directory` resource type that throws an error for now,
but will load arbitrary directories from git repositories once
#1764 lands
## Remaining work
- [x] Discuss the scope and the ideas
- [x] Add unit tests
- [x] Update the documentation
- [x] Adjust the `installPlugin` and `installTheme` step for
compatibility with it's former signature. Ensure the existing packages
consuming those functions from the `@wp-playground/blueprints` package
will continue to work.
- [x] Confirm we can safely omit streaming from the system design at
this point without setting ourselves up for a grand refactor a few
months down the road.
* I think we can! Streaming support could be an addition to the system,
not a change in how the system works. For example, there could be a new
`DirectoryStream` resource type producing an `AsyncDirectoryIterator`
with streamable `File` or `Blob` objects as its leafs. It would work
nicely with remote APIs or the ZIP streaming plumbing in
`@php-wasm/stream-compression`. Any existing code expecting a
`DirectoryResource` should be relatively easily adaptable to use these
async iterators instead.
## Follow-up work
- [ ] Include actual git support once the [Git sparse checkout
PR](#1764) lands
- [ ] Ship a Playground CORS proxy to enable using git checkout in the
webapp
- [ ] Once we have a use-able `git:directory` resource, expand the
developer notes from this PR and other related PRs and write a post on
https://make.wp.org/playground
## Tangent – Streaming and a shorthand URL notation
Without streaming, the entire directory must be loaded into memory. Our
git sparse checkout implementation buffers everything anyway, but we
will want to stream-read directory resources in the future. For example:
```js
{
"steps": [
// Stream plugin files directly from a doubly zipped Git artifact
{
"step": "installPlugin",
"pluginData": {
"resource": "zip:github-artifact",
"zipFile": {
"resource": "url",
"url": "https://github.com/WordPress/guteneberg/pr/54713/artifacts/build.zip"
}
}
}
}
}
```
That's extremely verbose, I'd love to explore a shorthand notation. One
idea would be to make it a valid
[URI](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier) shaped
after the [data URL
syntax](https://developer.mozilla.org/en-US/docs/Web/URI/Schemes/data):
```js
const dataUri = `data:text/html;base64,%3Cscript%3Ealert%28%27hi%27%29%3B%3C%2Fscript%3E`;
const githubArtifactUri = `zip-github-artifact+url:https://github.com/WordPress/gutenberg/pr/54713/artifacts/build.zip`;
const gitResourceUri = `git:branch=HEAD;path=src,https://github.com/WordPress/hello-dolly.git`;
```
It wouldn't allow easy composition of the resources, e.g. a directory
inside a zip sourced from a GitHub repo. Maybe that's for the best,
though, since such a string would be extremely dense and difficult for
humans to read. The object-based syntax might still be the most
convenient way of to declare those.- Loading branch information
Showing
35 changed files
with
3,501 additions
and
1,746 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.