[RFC] User Extensibility - Lifecycles and Plugins

[thescientist13]

Type of Change

• New Feature Request
• Documentation / Website
• Improvement / Suggestion
• Bug
• Other (please clarify below)

Summary

Allowing users to extend functionality of the tool will be the best way to gain adoption and support. Things like lifecycles and plugins can provide consistent APIs and hooks for users to tap into the Greenwood build process and extend the functionality to support their project's needs.

Details

Some of these may be maintained by Greenwood itself, but would want to flesh out

• Lifecycles: How and when are operations run (phases), and what's the inputs and outputs for each phase
• Plugins: Extend the build phase by providing your own webpack plugins?

Full details mapped out here
#17 (comment)

[thescientist13]

In thinking about #115 and how its use cases are getting defined more clearly, it seems this RFC should also get some clearly defined uses cases, like #86? What exactly do we want to "open up" with this kind of feature? 🤔

[thescientist13]

Watched this video and they used rollup instead of webpack, but it definitely highlighted a little bit of a more simple way to write plugins.
https://www.youtube.com/watch?v=TsTt7Tja30Q

Or we could just people pass in webpack plugins?

[thescientist13]

So had an idea for plugins, that they can be used (at least for one case) to inject HTML into index.html. A couple use cases being:

• Google Analytics
• Poylfills

[thescientist13]

Overview

What I'm thinking is that plugins will be the top level "primitive" with a few different types and APIs:

1. hook - Prefined places in index.html that you can be used to inject custom HTML, like with spaIndexFallback. Anticipated hooks:
   • analytics - Google Analytics, Snowplow. Will make an issue for creating one for Greenwood.
   • meta - Useful for adding static build time <meta>, <title>, favicons, etc
   • polyfills- Good for supporting what Greenwood will provide as a plugin for polyfills
   • For more complex use cases, user can just as easily override index.html themselves.
2. build - Alter the build directly by passing in webpack plugins directly! (YMMV). Will have options for common, develop, prod depending on where the plugin should get used
3. source - Tapping into the graph, see [RFC] Build Time Data Injection #115 / [RFC] External Data (Graph) Sources #21 like when consuming from external sources like a headless CMS
4. lifecycles - TBD and should be defined in another issue.
5. transform - Different input types like .yaml, html, or JavaScript instead of just markdown, like in custom elements as pages (aka one off page templates) #170. Will need its own issue

Plugins would get access to a read only version of the Greenwood compilation object that would provide plugins information like contexts (inpout / output directories and paths), configuration (e.g. greenwood.config.js), and the overall build state. Plugins would be instantiated in greenwood.config.js as an Array called plugins.

example:

module.exports = {
  title: 'My Project',

  ...

  plugins: [{
    type: 'hook|webpack|data|lifecycle|transform',  // @string required, no default
    provider: (compilation) => { /* do stuff here */ } // @function required (@return depends on type), no default
  }]
}

A plugin can define multiple types depending on what the task at hand. An example of a plugin having multiple types is that of a Polyfill plugin that installs dependencies locally via npm. In that case not only would we need a hook for tapping into the analytics hook, but would also need to be able to includes those polyfills from node_modules as part of the Greenwood build using something like CopyWebpackPlugin so that the files will be available in public/ to match the path set in the hook in index.html.

Greenwood should try and encourage development / contributions to the monorepo itself, a la Gatsby
https://github.com/gatsbyjs/gatsby/tree/master/packages

For the sake of this issue, I'll just cover hook and build plugin types but it should provide enough of a blue print for more follow up tasks. (covered at the end)

Hook

In particular index.html provides the most oppourtunity right now as things like polyfills, analytics, and PWA features like manifest.json and ServiceWorkers need to be installed in a global scope and /or unbundled. These would essentially be pre-defined find / replace hooks in index.html.

Usage

A great example use case is injecting Google Analytics into index.html.

In Greenwood's index.html we would add this marker

  ...

  <%= htmlWebpackPlugin.options.hookGreenwoodAnalytics %>

Then in Greenwood config, you tap into it like this:

module.exports = {
  title: 'My App'

  ...

  plugins: [{
    type: 'hook',
    provider: (compilation) => {
      // you can access things like config, context if you need from compilation
      return {
        hookGreenwoodAnalytics: `
          <!-- Google Analytics -->
          <script>
            window.ga=window.ga||function(){(ga.q=ga.q||[]).push(arguments)};ga.l=+new Date;
            ga('create', 'UA-XXXXXXX', 'auto');
            ga('send', 'pageview');
          </script>
          <script async src='https://www.google-analytics.com/analytics.js'></script>
          <!-- End Google Analytics -->
        `
      ]
    }
  }]
}

The greenwood plugin would essentially just wrap all of this for a user and just ask for analyticsId.

It should be noted that if these specific hook types are too limiting (we should support a basic set of common hooks), users can just make their own index.html and configure all of these Greenwood hooks themselves (if they wanted) or just create their own.

Build

Basically you can pass a webpack plugin! Mileage may vary, but we could provide a list of known plugins that work in this way. (like ones that we use in our own plugins)

Usage

For a consumer, usage would look something like this in greenwood.config.js where the provider function in this case returns an instance of a webpack plugin.

example:

const { version } = require('package.json');
const webpack = require('webpack');

module.exports = {
  title: 'My App'
  
  ...
  
  plugins: [{
    type: 'build'
    provider: (compilation) => {
      const { config } = compilation;
      
      return new webpack.BannerPlugin({
        banner: `${config.title} v${version}: built on ${new Date().getTime()}`
      });
    }
  ]
}

Roadmap / Next Steps

1. Create new issues:
   • RFC: Lifecycles (backlog)
   • RFC: Transforms (backlog) - maybe convert custom elements as pages (aka one off page templates) #170?
   • Plugin: Google Analytics (will add to current project)
   • Plugin: Polyfills (will add to current project)
2. Create a release branch, to make a success of PRs and opened against master
3. Setup Lerna + Yarn workspaces
4. Create Google Analytics Plugin
5. Create Polyfills Plugin
6. Install Polyfills from npm
7. Differential Loading
8. Alpha testing
9. Merge to master + release
10. Further considerations
    • more hooks: manifest, meta, title, favicon hooks
    • PWA plugin: manifest, favicon, service workers, etc

[hutchgrant]

Those examples are only related to index.html, I'm concerned with build time hooks related to components, templates, content. I was thinking more of a base class, or series of base classes which contain all the functions needed for specific types of plugins. We would create a lifecycle that specifically loops through all the installed plugins, creates a new object from each plugin class, each of which extends and utilizes a library of plugin functions. The implementation of that is my issue, because each plugin will require unique configurations, and certain plugins as you pointed out need to be implemented at different times and places. While I do want webpack plugins accessible to the default webpack configuration, I'm thinking about different forms of extensibility like hooks within templates(not just index.html files) that require pre-build caching etc (e.g. Meta plugin, headless CMS).

[thescientist13]

Hmm... From your PR it seems like we are more or less on the same page but your comment would make it seem like we are not, so that still has me scratching my head a little bit. Let me cover some of your points and see if that helps further align us.

I'm thinking about different forms of extensibility like hooks within templates(not just index.html files)

I did lead with five examples of this in the opening of my comment. I've since numbered them so hopefully this makes things a bit clearer?

Hooks and direct webpack plugins are but 2 of these 5. The rest are all better covered in RFC issues which I linked out to.

I'm concerned with build time hooks related to components, templates, content

Like I said, i've listed all the examples I can think of including content. I want to avoid "components" in the build / CLI though because that should probably be left to userland / runtime code responsibilities. This is what we've been talking about already in the Build Time Data Injection RFC so that's where I would like to see that go, generally.

Perhaps you could be more specific though in identifying what the role of these use cases would be and how they differentiate from what I've provided so that I can better understand what gaps you are seeing?

Those examples are only related to index.html,

Correct, and that was intentional. I did say this after all...

"For the sake of this issue, I'll just cover hook and build plugin types but it should provide enough of a blue print for more follow up tasks. (covered at the end)"

But the build example with BannerPlugin has nothing to do with an index.html file, so to be fair, only one of my examples has to do with index.html.

I was thinking more of a base class, or series of base classes which contain all the functions needed for specific types of plugins.

Yeah, a base class is another way to establish the contract plugins so that does raise a good point and might be a good way to establish some consistency.

Though some compelling advantages towards functions are that they are first class citizens in JavaScript, are super easy to pass around, don't have to worry about new, and are going to be a lot more performant (I would imagine) then spinning up a bunch of classes that just get used and tossed away.

---

I encourage you to go back over and read through my comments again as I think it covers a broad set of use cases pulling from existing RFCs we've discussed and aims to pull them under one common API (plugins), of which there will be a few different options within that.

I just happen to be starting with the hook / webpack options because:

• they are smaller chunks to bite off right now
• have pretty clearly defined use cases already
• given their "smallness" make for a good first step into setting up our monorepo

I think we are more or less in line so I just want to make sure I'm not missing any use cases but I would like to start following my action items at least and making / tracking all the relevant work.

[thescientist13]

OK, made the following issues and will defer out to those for the rest of this RFC work

• [RFC] Plugins: Index Hooks #182
• [RFC] Plugins: Build / Pipeline (webpack) #183
• [RFC] Plugins: Lifecycles #184
• [RFC] Plugins: Transforms #185
• Google Analytics Plugin #186
• Polyfills Plugin #187

Will merge #181 and start tracking the order of these tasks there.

[hutchgrant]

• graph
• scaffold
• context
• config

those 4 lifecycles need to utilize plugins. Some plugins will utilize more than one. Some plugins may require modifying different parts of the lifecycle. This is not possible from a single function. You would require multiple plugins under the design you've suggested.

An example would be a headless Wordpress plugin. I would need a plugin just to pull from the API and cache then create a graph from it, and then I'd need a different scaffold plugin to scaffold out different parts of the page template.

While your idea considers the use cases of webpack and modifying the index.html, it does not take into account scenarios that modify lifecycles or different sub-lifecycles of lifecycles. I'd prefer a solution that does, as that's what I need for graphQL and other plugins.

[thescientist13]

I'll be honest, I don't think this conversation is going to progress much further over GitHub unfortunately and would prefer to talk about it in person instead.

Only thing I'll say at this point is nothing is stopping you from proposing anything you want. I never said "No" to anything. I just took a bigger problem, broke it down to smaller independent problems, then picked some of the smallest ones and said I would start with that. I was happy to talk to you about it in person since I continue to struggle with why you express your views as if there is something in conflict here. If it's just that you want to use a class then sure, that's fine let's talk about that instead.

So, before I close this I'll just say there are more RFCs for plugins that you can contribute to including Build Time Data Injection, Lifecycles, and External Data Sources that have no overlap with the current tasks I am picking up, and no one is stopping you from working on any of them and bringing forth an API. It would be much appreciated in fact since between code and docs, this will take a lot of work, so the more the merrier to help out on other elements of this. 🙌