Extract docs engine into reusable package

[rogermparent]

We currently use our custom docs engine on dvc.org and cml.dev, with plans to add it to all our product sites. The docs engine is currently "shared" between the two sites by simply copy-pasting 90+% of the code between the dvc.org and cml.dev repos. This already poses a problem when it comes to updating the engine, and will only get worse as we introduce new sites.

Due to these requirements and issues, we're undertaking an effort to move the docs engine to a reusable package, leaning on the Gatsby Plugin and Theme API as well as more general Node package interfaces like module exports. The long-term plan is to package the docs engine as an NPM package.

To ease the transition, we'll break development into multiple phases

Phase 1: dvc.org local plugin

We'll start by moving all dvc.org docs logic into a local Gatsby plugin, a pseudo-npm package in the plugins folder. This will allow for much easier rapid incremental development with minimal worry about maintaining updates for a half-working package. This phase ends after we move all of dvc.org's docs engine to this local package.

• Move docs node generation to a local plugin #3113
• Move docs sidebar logic to local plugin (Move Documentation components to theme #3170)
• Move generalized React components to local plugin (either as exports or shadowable components)(Move Documentation components to theme #3170)

Phase 2: Internal package sharing

Once we're confident we've moved all the docs code we want to share, we can start turning the local plugin into a real package and use it among all our internal sites. This phase specifically puts less of an emphasis on how external users would use the plugin, instead putting the most consideration on how we'll use the plugin for our product sites.

• Create a new repo and NPM package with the contents of the local plugin folder(https://github.com/iterative/gatsby-theme-iterative/)
• Remove the local plugin from dvc.org and ensure all imports are changed to the npm package name (Implement Reusable Gatsby Theme #3396)
• Remove the copy-paste docs engine from cml.dev and use the reusable package(Updated Docs engine cml.dev#233)

Phase 3: Generalizing for outside audiences

After we're comfortably using this package for all our internal websites, we can consider the possibility of sharing it with the public as an option for other Gatsby sites to use as a docs engine. This is easily the most variable step- it could range from a massive refactor to being scrapped altogether if we decide to only ever consider internal use.

[jorgeorpinel]

Will #3396 close this? (If so please link them.)

[yathomasi]

Will #3396 close this? (If so please link them.)

Yes, that would complete phase 2.