resource template output parameter valueFrom.jqFilter results in error in v3.4.0

[hype-eater]

Pre-requisites

• I have double-checked my configuration
• I can confirm the issues exists when I tested with :latest
• I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?

What happened: Defining an output parameter for a resource template using valueFrom.jqFilter results failed Workflow with message Error (exit code 1): exec: "sh": executable file not found in $PATH. The node's main container (argoexec) is the source of the error: "exec: \"sh\": executable file not found in $PATH".

What I expect: No error

The code appears to specify a container command that will not work because the (freshly slimmed down) argoexec image no longer has sh:

argo-workflows/workflow/executor/resource.go

Lines 309 to 310 in e556fe3

	cmdStr := fmt.Sprintf("kubectl get %s -o json | jq -rc '%s'", strings.Join(resArgs, " "), param.ValueFrom.JQFilter)
	cmd = exec.Command("sh", "-c", cmdStr)

Version

v3.4.0

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: resource-jq-
spec:
  serviceAccountName: argo
  entrypoint: main
  templates:
  - name: main
    resource:
      action: get
      manifest: |
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: workflow-controller-configmap
    outputs:
      parameters:
      - name: data
        valueFrom:
          jqFilter: '.'

Logs from the workflow controller

$ kubectl logs -n argo deploy/workflow-controller | grep ${workflow}
time="2022-09-21T22:19:20.722Z" level=info msg="Processing workflow" namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:20.725Z" level=info msg="Updated phase  -> Running" namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:20.725Z" level=info msg="Pod node resource-jq-7dbdd initialized Pending" namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:20.738Z" level=info msg="Created pod: resource-jq-7dbdd (resource-jq-7dbdd)" namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:20.738Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:20.738Z" level=info msg=reconcileAgentPod namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:20.747Z" level=info msg="Workflow update successful" namespace=argo phase=Running resourceVersion=1226 workflow=resource-jq-7dbdd
time="2022-09-21T22:19:30.723Z" level=info msg="Processing workflow" namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:30.723Z" level=info msg="Task-result reconciliation" namespace=argo numObjs=0 workflow=resource-jq-7dbdd
time="2022-09-21T22:19:30.724Z" level=info msg="node changed" namespace=argo new.message="Error (exit code 1): exec: \"sh\": executable file not found in $PATH" new.phase=Failed new.progress=0/1 nodeID=resource-jq-7dbdd old.message= old.phase=Pending old.progress=0/1 workflow=resource-jq-7dbdd
time="2022-09-21T22:19:30.724Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:30.724Z" level=info msg=reconcileAgentPod namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:30.725Z" level=info msg="Updated phase Running -> Failed" namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:30.725Z" level=info msg="Updated message  -> Error (exit code 1): exec: \"sh\": executable file not found in $PATH" namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:30.725Z" level=info msg="Marking workflow completed" namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:30.725Z" level=info msg="Checking daemoned children of " namespace=argo workflow=resource-jq-7dbdd
time="2022-09-21T22:19:30.732Z" level=info msg="cleaning up pod" action=deletePod key=argo/resource-jq-7dbdd-1340600742-agent/deletePod
time="2022-09-21T22:19:30.737Z" level=info msg="Workflow update successful" namespace=argo phase=Failed resourceVersion=1255 workflow=resource-jq-7dbdd
time="2022-09-21T22:19:30.749Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo/resource-jq-7dbdd/labelPodCompleted

Logs from in your workflow's wait container

main container logs (not the wait container logs) as this is a resource type template

$ kubectl logs -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded -n argo
time="2022-09-21T22:19:22.145Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=argo podName=resource-jq-7dbdd template="{\"name\":\"main\",\"inputs\":{},\"outputs\":{\"parameters\":[{\"name\":\"data\",\"valueFrom\":{\"jqFilter\":\".\"}}]},\"metadata\":{},\"resource\":{\"action\":\"get\",\"manifest\":\"apiVersion: v1\\nkind: ConfigMap\\nmetadata:\\n  name: workflow-controller-configmap\\n\"},\"archiveLocation\":{\"archiveLogs\":true,\"s3\":{\"endpoint\":\"minio:9000\",\"bucket\":\"my-bucket\",\"insecure\":true,\"accessKeySecret\":{\"name\":\"my-minio-cred\",\"key\":\"accesskey\"},\"secretKeySecret\":{\"name\":\"my-minio-cred\",\"key\":\"secretkey\"},\"key\":\"resource-jq-7dbdd/resource-jq-7dbdd\"}}}" version="&Version{Version:v3.4.0,BuildDate:2022-09-19T03:47:58Z,GitCommit:047952afd539d06cae2fd6ba0b608b19c1194bba,GitTag:v3.4.0,GitTreeState:clean,GoVersion:go1.18.6,Compiler:gc,Platform:linux/amd64,}"
time="2022-09-21T22:19:22.145Z" level=info msg="Loading manifest to /tmp/manifest.yaml"
time="2022-09-21T22:19:22.145Z" level=info msg="kubectl get -f /tmp/manifest.yaml -o json"
time="2022-09-21T22:19:22.208Z" level=info msg="Resource: argo/configmap./workflow-controller-configmap. SelfLink: api/v1/namespaces/argo/configmaps/workflow-controller-configmap"
time="2022-09-21T22:19:22.208Z" level=info msg="Saving resource output parameters"
time="2022-09-21T22:19:22.208Z" level=info msg="[sh -c kubectl get configmap./workflow-controller-configmap -n argo -o json | jq -rc '.']"
time="2022-09-21T22:19:22.208Z" level=error msg="executor error: exec: \"sh\": executable file not found in $PATH"
time="2022-09-21T22:19:22.208Z" level=fatal msg="exec: \"sh\": executable file not found in $PATH"
time="2022-09-21T22:19:23.103Z" level=info msg="sub-process exited" argo=true error="<nil>"
Error: exit status 1

[sarabala1979]

@hype-eater Do you like to submit the PR for this issue?