cmd/cue: root keys with dashes in their name cannot be extracted with cue eval -e

[cueckoo]

Originally opened by @capelio in cuelang/cue#358

What version of CUE are you using (cue version)?

I performed a fresh go get -u cuelang.org/go/cmd/cue as of the time and date this issue was opened.

$ cue version
cue version devel linux/amd64

Does this issue reproduce with the latest release?

I tested with the latest devel code as of the date and time this issue was created. I have not tested with any prior official releases.

What did you do?

I attempted to extract a root key from a CUE file using cue eval -e. The root key has a dash in its name, which appears to be causing problems. Here is the repo case:

Create a new data.cue file with the following contents:

"foo": { one: 1 }
"foo-bar": { two: 2 }

Run cue eval data.cue -e foo, which returns one: 1 as expected.

However, the following attempts to get the foo-bar key fail.

$ cue eval data.cue -e foo-bar
reference "bar" not found:
    --expression:1:5

$ cue eval data.cue -e 'foo-bar'
reference "bar" not found:
    --expression:1:5

$ cue eval data.cue -e '["foo-bar"]'
["foo-bar"]

What did you expect to see?

I expected to see two: 2 returned.

What did you see instead?

See above errors.

[cueckoo]

Original reply by @verdverm in cuelang/cue#358 (comment)

This is some tracing I have done to follow the code path:

• https://github.com/cuelang/cue/blob/master/cmd/cue/cmd/eval.go#L90 (effective entrypoint to the eval command, but the cue and expr flags have already been processed and determined at this point)
• https://github.com/cuelang/cue/blob/master/cmd/cue/cmd/common.go#L361 (func parseArgs)
• https://github.com/cuelang/cue/blob/master/cmd/cue/cmd/common.go#L471 (func parseFlags, loop over flags adding to buildPlan)

This is where we go to the parser with the flag, and likely return the wrong kind of expression. This is where to put a break point and check input/output with Delve.

The following may be related:

• https://github.com/cuelang/cue/blob/master/cmd/cue/cmd/common.go#L376 -> 109
• https://github.com/cuelang/cue/blob/master/cmd/cue/cmd/common.go#L341
• https://github.com/cuelang/cue/blob/master/cmd/cue/cmd/common.go#L346
• https://github.com/cuelang/cue/blob/master/cmd/cue/cmd/eval.go#L136

These may be a bit out of order, but the main points are there. My best guess is that something in parsing the expressions is broken down this path. I have also tried the input -e '"foo-bar"' to avoid bash and added them inside the code when a hyphen is present, in which case it returns the string as is, rather than treating it as an IDENT and looking for it in the cue.Value like non-hyphened expression flags.

Really need to get into Delve more by default...

[cueckoo]

Original reply by @verdverm in cuelang/cue#358 (comment)

TL;DR
I'm going to go ahead an assume this is the correct output for the parser, and that the first version should be handled differently down the line, where the AST tokens are handled in the broken code flow. At this point, all the parser can do is interpret this as a string and let the evaluator handle the contextual interpretation. So the correct way for a user to get at a dashed key from the CLI is the first of the following examples.

---

At https://github.com/cuelang/cue/blob/master/cmd/cue/cmd/common.go#L472

cue eval -e '"dashed-key"' foo.cue returns

cuelang.org/go/cue/ast.Expr(*cuelang.org/go/cue/ast.BasicLit) *{
	ValuePos: cuelang.org/go/cue/token.Pos {
		file: *(*"cuelang.org/go/cue/token.File")(0xc0002aa380),
		offset: 18,},
	Kind: STRING (8),
	Value: "\"dashed-key\"",
	comments: cuelang.org/go/cue/ast.comments {
		groups: *[]*cuelang.org/go/cue/ast.CommentGroup nil,},
	expr: cuelang.org/go/cue/ast.expr {
		decl: cuelang.org/go/cue/ast.decl {},},
	label: cuelang.org/go/cue/ast.label {},}

while cue eval -e dashed foo.cue returns

cuelang.org/go/cue/ast.Expr(*cuelang.org/go/cue/ast.Ident) *{
	NamePos: cuelang.org/go/cue/token.Pos {
		file: *(*"cuelang.org/go/cue/token.File")(0xc000348d90),
		offset: 18,},
	Name: "dashed",
	Scope: cuelang.org/go/cue/ast.Node nil,
	Node: cuelang.org/go/cue/ast.Node nil,
	comments: cuelang.org/go/cue/ast.comments {
		groups: *[]*cuelang.org/go/cue/ast.CommentGroup nil,},
	label: cuelang.org/go/cue/ast.label {},
	expr: cuelang.org/go/cue/ast.expr {
		decl: cuelang.org/go/cue/ast.decl {},},}

and cue eval -e "dashed-key" foo.cue returns

cuelang.org/go/cue/ast.Expr(*cuelang.org/go/cue/ast.BinaryExpr) *{
	X: cuelang.org/go/cue/ast.Expr(*cuelang.org/go/cue/ast.Ident) *{
		NamePos: (*"cuelang.org/go/cue/token.Pos")(0xc0002991d0),
		Name: "dashed",
		Scope: cuelang.org/go/cue/ast.Node nil,
		Node: cuelang.org/go/cue/ast.Node nil,
		comments: (*"cuelang.org/go/cue/ast.comments")(0xc000299210),
		label: cuelang.org/go/cue/ast.label {},
		expr: (*"cuelang.org/go/cue/ast.expr")(0xc000299218),},
	OpPos: cuelang.org/go/cue/token.Pos {
		file: *(*"cuelang.org/go/cue/token.File")(0xc000306d20),
		offset: 114,},
	Op: SUB (14),
	Y: cuelang.org/go/cue/ast.Expr(*cuelang.org/go/cue/ast.Ident) *{
		NamePos: (*"cuelang.org/go/cue/token.Pos")(0xc0002993b0),
		Name: "key",
		Scope: cuelang.org/go/cue/ast.Node nil,
		Node: cuelang.org/go/cue/ast.Node nil,
		comments: (*"cuelang.org/go/cue/ast.comments")(0xc0002993f0),
		label: cuelang.org/go/cue/ast.label {},
		expr: (*"cuelang.org/go/cue/ast.expr")(0xc0002993f8),},
	comments: cuelang.org/go/cue/ast.comments {
		groups: *[]*cuelang.org/go/cue/ast.CommentGroup nil,},
	expr: cuelang.org/go/cue/ast.expr {
		decl: cuelang.org/go/cue/ast.decl {},},}

[cueckoo]

Original reply by @verdverm in cuelang/cue#358 (comment)

Here might be the "root" question... "How does one access top-level keys which require indexing due to the format of the identifier?"

https://cuelang.org/docs/tutorials/tour/references/selectors/

In this link, the indexing happens on a struct field. Is there a notation to specify the root / file level equivalent to a in this example?

[cueckoo]

Original reply by @mpvl in cuelang/cue#358 (comment)

This is not a bug: the dashed key is not a valid identifier, and the -e option expects a valid CUE expression.

Within CUE, the workaround is to use a field alias

X="foo-bar": ...

As aliases are not visible outside the scope in which they are defined, this is not a solution for -e.

There are two possible solutions:

1. support access to the top-level
2. have free-form identifiers, e.g. using back quotes.

The latter was actually supported at some point, but it seems to be too confusing. It may be a possibility, though.

Option 1. has deliberately not been supported, as it is too easy to misuse this feature: it's use will almost always be wrong and it is hardly ever necessary. That said, we could consider supporting a symbol for root that is only available from the command line, such as $ or @. There is something like that with the --with-context feature. This currently only works for non-CUE file, as this is a transformation feature, but one could imagine this as an alternative.

The use of @ would interact well with the Query proposal: #165.

[cueckoo]

Original reply by @capelio in cuelang/cue#358 (comment)

@mpvl From my end-user perspective, the dashed cue is "valid" in so much as we can import the cue file and get to the underlying values with paths like package["foo-bar"].two. I think my assumption about what constitutes a valid identifier is off and missing details. Why would an identifier that can be worked with via imports, etc, not be considered a valid identifier when using cue eval -e?

[jpluscplusm]

For folks searching for relevant issues in the future: this also affects field names containing periods.

[jpluscplusm]

This may be sufficiently different from the headline issue that it's not helpful, but here's how I worked around this issue when trying to use cue export (not eval) to pick out a field from a JSON file (not CUE):

cue export -l 'foo:' file.json -e 'foo["name.with.period"]'