Different results from different yq versions.

[trainormg]

Describe the bug
The same eval-all expression returns different values in yq 4.9.8 and yq 4.25.1.

Command:

yq eval-all '[[.default, .boot][] | .terraform_states_bucket | select(. != null)][-1]' ../../configs/common-config.yaml ../../configs/00
1-bootstrap/app-config.yaml

Result using yq 4.9.8:

my-tf-states-bucket-ew1

Result using yq 4.25.1 or 4.24.5:

my-tf-states-bucket-ew1
my-tf-states-bucket-ew1

Same expression without array index to extract a single value:

yq  eval-all '[[.default, .boot][] | .terraform_states_bucket | select(. != null)]' ../../configs/common-config.yaml ../../configs/001-bo
otstrap/app-config.yaml

Result (the same with all the versions of yq already mentioned):

- demos-v4-states-bucket
- my-tf-states-bucket-ew1

The expected result is to have the same return values for all the yq versions.

[trainormg]

Checking the behavior of different versions I found that the problem appeared in release 4.16.1.
Release 4.15.1 behaves correctly, or at list has the same behavior of releases 4.9.8, 4.13.3, 4.14.2.

[mikefarah]

Hi @trainormg - thanks for raising the issue. Can you please provide me with sample yaml so that I can try reproducing and diagnosing the issue?

[trainormg]

Hi.
I slightly modified the command to have it work with a couple of sample files:

yq eval-all '[[.default, .boot][] | .val | select(. != null)][-1]' file1.yaml file2.yaml

The results change, in the same way I described in the first comment, from yq <=4.15.1 and yq>=4.16.1.

The sample files:

• file1.yaml

default:
  val: default_value

• file2.yaml

default:
  val: file2_default
boot:
  val: boot_value

[mikefarah]

Digging into this, I can replicate it.

Until I work out what's going on and fix it, you can workaround the issue by adding a pipe:

yq eval-all '[[.default, .boot][] | .val | select(. != null)] | .[-1]' file1.yaml file2.yaml

[mikefarah]

Ok I understand what's happening now. When the expression is like this:

[[.default, .boot][] | .val | select(. != null)][-1]

the [-1] portion is evaluated against the current context, which is two files. So it produces a result for each file.

Whereas the expression:

[[.default, .boot][] | .val | select(. != null)] | [-1]

Because of the pipe operator, the [-1] expression is evaluated against the result of [[.default, .boot][] | .val | select(. != null)] which is a single array - and so produces a single result.

The change in 4.16.1 was a fix to the collect operator - which as a side effect now does this. It does seem odd at first, but digging into it I think it's actually being consistent.

Still I'm not satisfied as it's not intuitive - but if I revert the fix I did that will just cause a bunch of other issues. I'll sit on this for a bit and see if I (or anyone else) thinks of something clever to do...