Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

yq crashes when piping a yaml file with another yaml file as input #1200

Closed
hickersonj opened this issue Apr 27, 2022 · 3 comments
Closed

yq crashes when piping a yaml file with another yaml file as input #1200

hickersonj opened this issue Apr 27, 2022 · 3 comments
Labels
bug v4

Comments

@hickersonj
Copy link

Describe the bug
When piping a file to yq as one input with another file as the other input there is a runtime error. However, providing the two files as arguments works.

Version of yq: 4.24.5
Operating system: Linux (Fedora 33)
Installed via: binary release

Input Yaml
Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less)
namespace.yml

---
apiVersion: v1
kind: Namespace
metadata:
  name: namespace

app.yml

---
apiVersion: v1
kind: Deployment
metadata:
  name: deploy-app
  labels:
    app: deploy-label
---
apiVersion: v1
kind: Daemonset
metadata:
  name: daemonset-app
  labels:
    app: daemonset-label

Command

cat namespace.yml | yq eval-all 'with(select(fi == 0); .metadata.name = env(Namespace)) | with(select(fi != 0); .metadata += {"namespace": env(Namespace)})' app.yml -

Actual behavior

panic: runtime error: index out of range [2] with length 2

goroutine 1 [running]:
github.com/mikefarah/yq/v4/pkg/yqlib.addMaps(0xc0003eb9d0, 0xc0003feba0?, 0xc0003e1a80?)
	/home/runner/work/yq/yq/pkg/yqlib/operator_add.go:193 +0x3af
github.com/mikefarah/yq/v4/pkg/yqlib.add(0xc0000a1800?, {0xc0004020c0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc0003eb500, 0xc0003eb960)
	/home/runner/work/yq/yq/pkg/yqlib/operator_add.go:63 +0x539
github.com/mikefarah/yq/v4/pkg/yqlib.resultsForRHS(0x19?, {0xc0004020c0, 0xc000397860, 0x1, {0x0, 0x0}}, 0x203000?, {0x0, 0x0, 0x7d7740}, ...)
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:88 +0x503
github.com/mikefarah/yq/v4/pkg/yqlib.doCrossFunc(0x22?, {0xc0004020c0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc00037e7c8, {0x0, 0x0, 0x7d7740})
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:125 +0x413
github.com/mikefarah/yq/v4/pkg/yqlib.crossFunctionWithPrefs(0x1?, {0xc0004020c0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc000397f50?, {0x0, 0x0, 0x7d7740})
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:152 +0x212
github.com/mikefarah/yq/v4/pkg/yqlib.crossFunction(0xc0000a1800?, {0xc0004020c0, 0xc000397860, 0x1, {0x0, 0x0}}, 0x0?, 0xaeb540?, 0x0?)
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:136 +0x9f
github.com/mikefarah/yq/v4/pkg/yqlib.addOperator(0xc0000a1800?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0x0?)
	/home/runner/work/yq/yq/pkg/yqlib/operator_add.go:46 +0x1c5
github.com/mikefarah/yq/v4/pkg/yqlib.(*dataTreeNavigator).GetMatchingNodes(0xc000397860?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc00037e7c8)
	/home/runner/work/yq/yq/pkg/yqlib/data_tree_navigator.go:37 +0x25c
github.com/mikefarah/yq/v4/pkg/yqlib.resultsForRHS(0x19?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0x203000?, {0x0, 0x0, 0xc0003e1980}, ...)
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:69 +0x1fc
github.com/mikefarah/yq/v4/pkg/yqlib.doCrossFunc(0x22?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc00037e7e0, {0x0, 0x0, 0xc0003e1980})
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:125 +0x413
github.com/mikefarah/yq/v4/pkg/yqlib.crossFunctionWithPrefs(0x1?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0x7f82cdaf95b8?, {0x0, 0x0, 0xc0003e1980})
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:152 +0x212
github.com/mikefarah/yq/v4/pkg/yqlib.crossFunction(0xc0003d2df8?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc00037e7b0?, 0xc0003d2e60?, 0xf?)
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:136 +0x9f
github.com/mikefarah/yq/v4/pkg/yqlib.assignUpdateOperator(0xc0000a1800?, {0xc0003977d0, 0xc000397860, 0x0, {0x0, 0x0}}, 0xc00037e7e0)
	/home/runner/work/yq/yq/pkg/yqlib/operator_assign.go:31 +0x305
github.com/mikefarah/yq/v4/pkg/yqlib.(*dataTreeNavigator).GetMatchingNodes(0xc00037e798?, {0xc0003977d0, 0xc000397860, 0x0, {0x0, 0x0}}, 0xc00037e7e0)
	/home/runner/work/yq/yq/pkg/yqlib/data_tree_navigator.go:37 +0x25c
github.com/mikefarah/yq/v4/pkg/yqlib.compoundAssignFunction(0x4cf825?, {0xc0003977d0, 0xc000397860, 0x0, {0x0, 0x0}}, 0xc00037e5d0, 0x7d7820)
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:35 +0x3ba
github.com/mikefarah/yq/v4/pkg/yqlib.addAssignOperator(0xc0000a1800?, {0xc0003977d0, 0xc000397860, 0x0, {0x0, 0x0}}, 0x0?)
	/home/runner/work/yq/yq/pkg/yqlib/operator_add.go:19 +0x94
github.com/mikefarah/yq/v4/pkg/yqlib.(*dataTreeNavigator).GetMatchingNodes(0xc0000a1800?, {0xc0003977d0, 0xc000397860, 0x0, {0x0, 0x0}}, 0xc00037e5d0)
	/home/runner/work/yq/yq/pkg/yqlib/data_tree_navigator.go:37 +0x25c
github.com/mikefarah/yq/v4/pkg/yqlib.withOperator(0xc0000a1800?, {0xc0002f7080, 0x0, 0x0, {0x0, 0x0}}, 0xc00037e600)
	/home/runner/work/yq/yq/pkg/yqlib/operator_with.go:25 +0x34c
github.com/mikefarah/yq/v4/pkg/yqlib.(*dataTreeNavigator).GetMatchingNodes(0x46af9e?, {0xc0002f7080, 0x0, 0x0, {0x0, 0x0}}, 0xc00037e600)
	/home/runner/work/yq/yq/pkg/yqlib/data_tree_navigator.go:37 +0x25c
github.com/mikefarah/yq/v4/pkg/yqlib.pipeOperator(0xc0000a1800?, {0xc0002f7080, 0x0, 0x0, {0x0, 0x0}}, 0xc00037e618)
	/home/runner/work/yq/yq/pkg/yqlib/operator_pipe.go:13 +0x147
github.com/mikefarah/yq/v4/pkg/yqlib.(*dataTreeNavigator).GetMatchingNodes(0xc0003902a0?, {0xc0002f7080, 0x0, 0x0, {0x0, 0x0}}, 0xc00037e618)
	/home/runner/work/yq/yq/pkg/yqlib/data_tree_navigator.go:37 +0x25c
github.com/mikefarah/yq/v4/pkg/yqlib.(*allAtOnceEvaluator).EvaluateCandidateNodes(0xc0003d3ca8, {0x7fffa10a18cf?, 0x7fffa10a1952?}, 0xc0002f7080)
	/home/runner/work/yq/yq/pkg/yqlib/all_at_once_evaluator.go:42 +0xba
github.com/mikefarah/yq/v4/pkg/yqlib.(*allAtOnceEvaluator).EvaluateFiles(0x7b1cdb?, {0x7fffa10a18cf, 0x7a}, {0xc0000a1cc0, 0x2, 0xc0000c4f28?}, {0x838cb0, 0xc00038e180}, 0x1, {0x838220, ...})
	/home/runner/work/yq/yq/pkg/yqlib/all_at_once_evaluator.go:85 +0x35d
github.com/mikefarah/yq/v4/cmd.evaluateAll(0xc0000cb680?, {0xc0000a1cb0?, 0x3, 0x3})
	/home/runner/work/yq/yq/cmd/evaluate_all_command.go:144 +0x137e
github.com/spf13/cobra.(*Command).execute(0xc0000cb680, {0xc0000a1c80, 0x3, 0x3})
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:856 +0x67c
github.com/spf13/cobra.(*Command).ExecuteC(0xc0000cb180)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3b4
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:902
main.main()
	/home/runner/work/yq/yq/yq.go:22 +0x1f1

Expected behavior

---
apiVersion: v1
kind: Namespace
metadata:
  name: app-test
---
apiVersion: v1
kind: Deployment
metadata:
  name: deploy-app
  labels:
    app: deploy-label
  namespace: app-test
---
apiVersion: v1
kind: Daemonset
metadata:
  name: daemonset-app
  labels:
    app: daemonset-label
  namespace: app-test

Additional context
The goal is to append the Namespace resource to the app.yml without needing to have seperate files, i.e. the namespace.yml would be piped into yq inside of a bash script.
@mikefarah
Copy link
Owner

Ok I found the bug - it's with appending to a map when the new key entry matches a value in an existing map (in this case namespace). I'll have a fix in the next release, till then, as a workaround, instead of .metadata += {"namespace": env(Namespace)} just do .metadata.namespace = env(Namespace)

mikefarah added a commit that referenced this issue Apr 28, 2022
@mikefarah
Fixes append map bug when key matches value in existing map #1200
67c79b7
@hickersonj
Copy link
Author

Glad you found the bug quickly! Also I didn't know that syntax .metadata.namespace = env(Namespace) would add a key value pair, glad to have learned that.

It is now working with the suggestions thanks.

@mikefarah
Copy link
Owner

Fixed in 4.25.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug v4
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mikefarah @hickersonj