CUE_EXPERIMENT and possibly CUE_DEBUG should not fail on unknown keys

[mvdan]

We currently fail if CUE_EXPERIMENT or CUE_DEBUG contain keys which are unknown:

$ CUE_EXPERIMENT=toposort cue-v0.10.0 export foo.cue
cannot parse CUE_EXPERIMENT: unknown toposort
$ CUE_EXPERIMENT=toposort cue export foo.cue
[...]

As @mpvl points out, this is potentially a problem, because being able to enable or disable a number of experiments across CUE versions - even if some very old or very new versions do not support them - is useful to compare behavior, such as when investigating or bisecting bugs.

I see a spectrum of possible approaches, from most to least strict:

1. Unknown keys and invalid key values are forbidden. A key for an experiment or debug flag is removed as soon as it's no longer useful. This is the current status quo, as we mark completed experiments as "deprecated" and they can no longer be set.
2. Unknown keys and invalid key values are forbidden, but any keys for experiments or debug flags remain forever, fixed to the value of the final behavior. For example, this would mean that in CUE v0.11.0 I could set CUE_EXPERIMENT=modules, but not CUE_EXPERIMENT=modules=0, as the pre-modules code is now deleted.
3. A variation of option 2, but without forcing values for completed experiments. For example, this would mean that I could set CUE_EXPERIMENT=modules=0 in CUE v0.11.0 and it would be a no-op.
4. Invalid key values are forbidden, but unknown keys are silently ignored. This would have allowed using CUE_EXPERIMENT=modules all the way back in CUE v0.4.0 for example, even if it did nothing at all. It would also make typos such as CUE_EXPERIMENT=evlav3 silently do nothing, which is unfortunate.

I believe Marcel was arguing for option 4 above. My personal take is that we should do more of a middle ground for the sake of preventing human error such as typos, or users being misled into thinking that an env var is doing something when it's doing nothing at all. So my inclination is for option 2 above.

[mvdan]

Here is an example of a new experiment which an old CUE does not yet know about:

$ CUE_EXPERIMENT=toposort cue-v0.10.0 export foo.cue
cannot parse CUE_EXPERIMENT: unknown toposort
$ CUE_EXPERIMENT=toposort cue export foo.cue
[...]

Resolving this would require a very lax approach, such as option 4 above.

Here is an example of a completed experiment which we removed entirely, following option 1 above:

$ CUE_EXPERIMENT=yamlv3decoder=1 cue-v0.11.0 export
[...]
$ CUE_EXPERIMENT=yamlv3decoder=1 cue export
cannot parse CUE_EXPERIMENT: unknown yamlv3decoder=1

And here is an example of a completed experiment which we left behind following option 2 above, as @rogpeppe correctly pointed out that we don't need to break users who set the env var to the right value:

$ CUE_EXPERIMENT=modules=1 cue-v0.10.0 export
[...]
$ CUE_EXPERIMENT=modules=1 cue-v0.11.0 export
[...]
$ CUE_EXPERIMENT=modules=0 cue-v0.11.0 export
cannot parse CUE_EXPERIMENT: cannot change default value of deprecated flag "modules"

So we currently follow a mix of options 1 and 2 above, being a bit inconsistent.

[mvdan]

Note that this is most important with CUE_EXPERIMENT, as we use it exclusively for changes in behavior which are likely to affect users and require careful transitions. However, CUE_DEBUG may also benefit from some relaxation, given that e.g. CUE_DEBUG=sortfields or CUE_DEBUG=openinline also affect the behavior of CUE and may be useful when comparing versions of CUE - even past the point where those flags were implemented.

Alternatively, if we only relax CUE_EXPERIMENT, then I would argue that openinline needs to move from CUE_DEBUG to CUE_EXPERIMENT, given that it's also a transition of user behavior. I would personally do this move anyway for that reason.