cuelang.org: preprocessor: make examples' files available in a repo

[jpluscplusm]

Some pages contain named files inside code or upload blocks, and we could make those files available as stand alone, plain text files in a repo.

This would either need to be implemented inside the preprocessor, or authors will need to manually sync content and files.

Purpose

Making these files available would be a low friction way to allow a reader to play with an example's CUE/JSON/YAML files by simply cloning the repo and navigating to the relevant page's directory.

Considerations

Which repo?

If we were to commit the files into cue-lang/cuelang.org, this would have several advantages:

• files rendered and committed as part of the standard document author process, without any need to set up cross-repo commits to /another/ repo (e.g. in CI)
• files would be automatically checked by the check-if-working-tree-is-clean CI test

The only disadvantages I can see are that the cue-lang/cuelang.org repo's size would grow a little faster on each (example-altering) change; and that the repo that the inquisitive reader would need to download would be a bit larger than strictly needed, as they'd necessarily pull down the entire site contents and not just the plain text example files. However, both of these size concerns are pretty trivial.

Suggestion: include the files in cue-lang/cuelang.org in the same commit that introduces/modifies the markdown source of the files.

Where, in the cue-lang/cuelang.org repo?

Whilst we could place the example files alongside the content they're sourced from, we'd probably want to place them in a sub-directory of the content directory so that the user's cue commands wouldn't interact with the content directory's site.cue site package, and so that the user could simply copy cue commands from the the on-page examples and have them work without modification. How this putative content/docs/howto/foo-a-bar/examples/ directory would pass through the Hugo processing, however, is not yet known. Would Hugo try to render a page under that path? Or complain about there not being a renderable source file?

However, instead of using the content directory as the root for the example files, we could place the example files inside a dedicated directory hierarchy, with a structure mirroring the URL paths that readers see on the front end.

Furthermore, if we're clever about the example files' location, then we can achieve both this issue's goal and give ourselves a mechanism for (later) linking to the files via HTTP.

If we use Hugo's static files union-fs-like system, we can place the files in scope for the Hugo build whilst also leaving them isolated on disk for cue users. We can even instantiate a top-level symlink directory in the repo (examples) for cue users to experiment with, as is demonstrated in this CL, simultaneously making this file's URL available for later on-page linkage if we so choose. (The symlink has to be this way round, examples->hugo/examples, as Hugo forbids the other way round)

Suggestion: serialise into hugo/examples and symlink examples into there.

code: overlapping filenames

code blocks are evaluated by the preprocessor in isolation from one another, meaning that multiple code blocks can contain the same filename. How should such a situation be serialised?

script: mutable file content over the course of the page

script blocks build on the context of preceding script and upload blocks. There can be multiple upload blocks for the same (updated) file, and a script block can create, update and delete files. How should a multi-step script's fake filesystem get serialised? Once for each step? After the last step?

Rendering

Nothing on-page, yet. Perhaps a single page should explain that these files exist, but nothing needs changing on the preprocessor's pages emitted into the hugo directory.

[myitcv]

I think the main suggestion discussed on the call was making these files available in a GitHub repo. So let's discuss how that might work first.

[jpluscplusm]

I think the main suggestion discussed on the call was making these files available in a GitHub repo. So let's discuss how that might work first.

Now reflected in the initial comment.