Job failed with just error and without log output #13469

Halytskyi · 2023-01-25T01:02:49Z

Please confirm the following

I agree to follow this project's code of conduct.
I have checked the current issues for duplicates.
I understand that AWX is open source software provided for free and that I might not receive a timely response.

Bug Summary

Clean deployment and I use "Demo Job Template". Job failing and no output except "Finished":

In "awx-ee" container logs I see:

DEBUG 2023/01/24 23:27:28 Client connected to control service @
DEBUG 2023/01/24 23:27:28 Control service closed
DEBUG 2023/01/24 23:27:28 Client disconnected from control service @
DEBUG 2023/01/24 23:27:28 Client connected to control service @
DEBUG 2023/01/24 23:27:29 Kubernetes version v1.23.15 is at least v1.23.14, using reconnect support
DEBUG 2023/01/24 23:27:29 [4Pmqfss9] streaming stdout with reconnect support
DEBUG 2023/01/24 23:27:29 [4Pmqfss9] Detected EOF for pod my-ns/automation-job-6-njfz8. Will retry 5 more times.
DEBUG 2023/01/24 23:27:29 [4Pmqfss9] Detected EOF for pod my-ns/automation-job-6-njfz8. Will retry 4 more times.
DEBUG 2023/01/24 23:27:29 [4Pmqfss9] Detected EOF for pod my-ns/automation-job-6-njfz8. Will retry 3 more times.
DEBUG 2023/01/24 23:27:29 [4Pmqfss9] Detected EOF for pod my-ns/automation-job-6-njfz8. Will retry 2 more times.
DEBUG 2023/01/24 23:27:29 [4Pmqfss9] Detected EOF for pod my-ns/automation-job-6-njfz8. Will retry 1 more times.
DEBUG 2023/01/24 23:27:30 Sending service advertisement: &{awx-69c867d6c6-vjqc4 control 2023-01-24 23:27:30.066549844 +0000 UTC m=+6185.772568701 1 map[type:Control Service] [{local false} {kubernetes-runtime-auth false} {kubernetes-incluster-auth false}]}
DEBUG 2023/01/24 23:27:30 Stdout complete - closing channel for: 4Pmqfss9
WARNING 2023/01/24 23:27:30 Could not read in control service: read unix /var/run/receptor/receptor.sock->@: use of closed network connection
DEBUG 2023/01/24 23:27:30 Client disconnected from control service @
WARNING 2023/01/24 23:27:30 Could not close connection: close unix /var/run/receptor/receptor.sock->@: use of closed network connection
DEBUG 2023/01/24 23:27:30 Client connected to control service @
DEBUG 2023/01/24 23:27:30 Control service closed
DEBUG 2023/01/24 23:27:30 Client disconnected from control service @
DEBUG 2023/01/24 23:27:36 Client connected to control service @
DEBUG 2023/01/24 23:27:36 Control service closed
DEBUG 2023/01/24 23:27:36 Client disconnected from control service @
DEBUG 2023/01/24 23:27:49 Client connected to control service @
DEBUG 2023/01/24 23:27:49 Control service closed
DEBUG 2023/01/24 23:27:49 Client disconnected from control service @

"awx-task" log:

2023-01-24 23:27:28,072 INFO     [5d337ea185d3432397012a6188cd4d12] awx.analytics.job_lifecycle job-6 waiting
2023-01-24 23:27:28,248 INFO     [5d337ea185d3432397012a6188cd4d12] awx.analytics.job_lifecycle job-6 pre run
2023-01-24 23:27:28,320 INFO     [5d337ea185d3432397012a6188cd4d12] awx.analytics.job_lifecycle job-6 preparing playbook
2023-01-24 23:27:28,421 INFO     [5d337ea185d3432397012a6188cd4d12] awx.analytics.job_lifecycle job-6 running playbook
2023-01-24 23:27:28,442 INFO     [5d337ea185d3432397012a6188cd4d12] awx.analytics.job_lifecycle job-6 work unit id received
2023-01-24 23:27:28,482 INFO     [5d337ea185d3432397012a6188cd4d12] awx.analytics.job_lifecycle job-6 work unit id assigned
2023-01-24 23:27:30,238 INFO     [5d337ea185d3432397012a6188cd4d12] awx.main.commands.run_callback_receiver Starting EOF event processing for Job 6
2023-01-24 23:27:30,238 INFO     [5d337ea185d3432397012a6188cd4d12] awx.analytics.job_lifecycle job-6 post run
2023-01-24 23:27:30,386 INFO     [5d337ea185d3432397012a6188cd4d12] awx.analytics.job_lifecycle job-6 finalize run
2023-01-24 23:27:30,391 WARNING  [5d337ea185d3432397012a6188cd4d12] awx.main.dispatch job 6 (error) encountered an error (rc=None), please see task stdout for details.

"/var/lib/awx/venv/awx/bin/receptorctl --socket /var/run/receptor/receptor.sock work list" output:

Warning: receptorctl and receptor are different versions, they may not be compatible
{
    "4Pmqfss9": {
        "Detail": "Finished",
        "ExtraData": {
            "Command": "",
            "Image": "",
            "KubeConfig": "",
            "KubeNamespace": "",
            "KubePod": "",
            "Params": "",
            "PodName": "automation-job-6-njfz8"
        },
        "State": 2,
        "StateName": "Succeeded",
        "StdoutSize": 0,
        "WorkType": "kubernetes-incluster-auth"
    }
}

"/var/lib/awx/venv/awx/bin/receptorctl --socket /var/run/receptor/receptor.sock work results 4Pmqfss9" (empty output):

Warning: receptorctl and receptor are different versions, they may not be compatible

"/var/lib/awx/venv/awx/bin/receptorctl --socket /var/run/receptor/receptor.sock status":

Warning: receptorctl and receptor are different versions, they may not be compatible
Node ID: awx-69c867d6c6-vjqc4
Version: 1.3.0+g8f8481c
System CPU Count: 36
System Memory MiB: 70231

Node         Service   Type       Last Seen             Tags
awx-69c867d6c6-vjqc4 control   Stream     2023-01-25 00:47:30   {'type': 'Control Service'}

Node         Work Types
awx-69c867d6c6-vjqc4 local, kubernetes-runtime-auth, kubernetes-incluster-auth

Part of logs from "automation-job-6-njfz8":

{"status": "starting", "runner_ident": "6", "command": ["ansible-playbook", "-u", "admin", "-i", "/runner/inventory/hosts", "-e", "@/runner/env/extravars", "hello_world.yml"],
...
{"uuid": "b6a38639-6dd2-4016-b9bf-8760e3feaf5b", "counter": 9, "stdout": "\r\nPLAY RECAP *********************************************************************\r\n\u001b[0;32mlocalhost\u001b[0m                  : \u001b[0;32mok=2   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   ", "start_line": 10, "end_line": 14, "runner_ident": "6", "event": "playbook_on_stats", "job_id": 6, "pid": 19, "created": "2023-01-24T23:27:31.692476", "parent_uuid": "6cd6a954-f087-44e1-ac2e-1629dc2d8d99", "event_data": {"playbook": "hello_world.yml", "playbook_uuid": "6cd6a954-f087-44e1-ac2e-1629dc2d8d99", "changed": {}, "dark": {}, "failures": {}, "ignored": {}, "ok": {"localhost": 2}, "processed": {"localhost": 1}, "rescued": {}, "skipped": {}, "artifact_data": {}, "uuid": "b6a38639-6dd2-4016-b9bf-8760e3feaf5b"}}
{"status": "successful", "runner_ident": "6"}
{"zipfile": 1412}
UEsDBBQAAAAIA...cBAAAAAA={"eof": true}

"api/v2/jobs/6/":

HTTP 200 OK
Allow: GET, DELETE, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept
X-API-Node: awx-69c867d6c6-vjqc4
X-API-Product-Name: AWX
X-API-Product-Version: 21.11.0
X-API-Time: 0.055s

{
    "id": 6,
    "type": "job",
...
    },
    "summary_fields": {
        "organization": {
            "id": 1,
            "name": "Default",
            "description": ""
        },
        "inventory": {
            "id": 1,
            "name": "Demo Inventory",
            "description": "",
            "has_active_failures": false,
            "total_hosts": 1,
            "hosts_with_active_failures": 0,
            "total_groups": 0,
            "has_inventory_sources": false,
            "total_inventory_sources": 0,
            "inventory_sources_with_failures": 0,
            "organization_id": 1,
            "kind": ""
        },
        "execution_environment": {
            "id": 2,
            "name": "Control Plane Execution Environment",
            "description": "",
            "image": "[private-registry/ansible/awx-ee:21.11.0](private-registry/ansible/awx-ee:21.11.0)"
        },
        "project": {
            "id": 6,
            "name": "Demo Project",
            "description": "",
            "status": "ok",
            "scm_type": "",
            "allow_override": false
        },
        "job_template": {
            "id": 7,
            "name": "Demo Job Template",
            "description": ""
        },
        "unified_job_template": {
            "id": 7,
            "name": "Demo Job Template",
            "description": "",
            "unified_job_type": "job"
        },
        "instance_group": {
            "id": 2,
            "name": "default",
            "is_container_group": true
        },
        "created_by": {
            "id": 1,
            "username": "admin",
            "first_name": "",
            "last_name": ""
        },
        "user_capabilities": {
            "delete": true,
            "start": true
        },
        "labels": {
            "count": 0,
            "results": []
        },
        "credentials": [
            {
                "id": 1,
                "name": "Demo Credential",
                "description": "",
                "kind": "ssh",
                "cloud": false
            }
        ]
    },
    "created": "2023-01-24T23:27:27.788821Z",
    "modified": "2023-01-24T23:27:28.019288Z",
    "name": "Demo Job Template",
    "description": "",
    "job_type": "run",
    "inventory": 1,
    "project": 6,
    "playbook": "hello_world.yml",
    "scm_branch": "",
    "forks": 0,
    "limit": "",
    "verbosity": 0,
    "extra_vars": "{}",
    "job_tags": "",
    "force_handlers": false,
    "skip_tags": "",
    "start_at_task": "",
    "timeout": 0,
    "use_fact_cache": false,
    "organization": 1,
    "unified_job_template": 7,
    "launch_type": "manual",
    "status": "error",
    "execution_environment": 2,
    "failed": true,
    "started": "2023-01-24T23:27:28.113893Z",
    "finished": "2023-01-24T23:27:30.271883Z",
    "canceled_on": null,
    "elapsed": 2.158,
    "job_args": "",
    "job_cwd": "",
    "job_env": {},
    "job_explanation": "Job terminated due to error",
    "execution_node": "",
    "controller_node": "awx-69c867d6c6-vjqc4",
    "result_traceback": "Finished",
    "event_processing_finished": true,
    "launched_by": {
        "id": 1,
        "name": "admin",
        "type": "user",
        "url": "/api/v2/users/1/"
    },
    "work_unit_id": "4Pmqfss9",
    "job_template": 7,
    "passwords_needed_to_start": [],
    "allow_simultaneous": false,
    "artifacts": {},
    "scm_revision": "347e44fea036c94d5f60e544de006453ee5c71ad",
    "instance_group": 2,
    "diff_mode": false,
    "job_slice_number": 0,
    "job_slice_count": 1,
    "webhook_service": "",
    "webhook_credential": null,
    "webhook_guid": "",
    "host_status_counts": null,
    "playbook_counts": {
        "play_count": 0,
        "task_count": 0
    },
    "custom_virtualenv": null
}

It's like job running without errors but it doesn't return successful status and "marked" as failed.

P.S. I saw the similar reports, but they mostly related to job running >4h. In my case it happens within seconds.

Is any way to debug that for find the issue?

AWX version

21.11.0

Select the relevant components

Installation method

kubernetes

Modifications

no

Ansible version

No response

Operating system

No response

Web browser

No response

Steps to reproduce

Issue (with logs) described above.

Expected results

No errors, see output for job.

Actual results

Job failed without output (except "Finished").

Additional information

No response

The text was updated successfully, but these errors were encountered:

TheRealHaoLiu · 2023-01-25T02:58:59Z

compiled a debug image quay.io/haoliu/awx-ee:debug-stdout

			for stdinErr == nil { // check between every line read to see if we need to stop reading
				line, err := streamReader.ReadString('\n')
				kw.Debug("Read line from pod %s/%s: %s", podNamespace, podName, line)
				if err == io.EOF {

added a simple change to log the output from the streamReader

I want to see if receptor was actually able to read from kube apiserver

TheRealHaoLiu · 2023-01-25T03:46:49Z

@Halytskyi DM me a gist containing the result

it does seem that we were able to read from the stdout stream

TheRealHaoLiu · 2023-01-25T05:01:15Z

Further debugging after preserving WorkUnit dir found that stdout file is empty

TheRealHaoLiu · 2023-01-25T05:02:00Z

In Matrix @Halytskyi told me that he's on EKS with 1.23.15

TheRealHaoLiu · 2023-01-25T05:09:46Z

Disabling kube reconnection support works around the problem

which indicate that our new code change to receptor may have contributed to this problem

yuliym · 2023-01-27T14:37:26Z

Hi.
Similar issue with AWX 21.10.2 which running on AKS with reconnection support enabled (disabling resolving issue but jobs fail in 4h)
awx.main.dispatch job 534 (error) encountered an error (rc=None), please see task stdout for details.

logs from awx-ee container:

uster-auth false}]}
DEBUG 2023/01/27 14:05:25 [nHvktzQU] Detected EOF for pod awx/automation-job-534-vz2ns. Will retry 5 more times.
DEBUG 2023/01/27 14:05:25 [nHvktzQU] Detected EOF for pod awx/automation-job-534-vz2ns. Will retry 4 more times.
DEBUG 2023/01/27 14:05:25 [nHvktzQU] Detected EOF for pod awx/automation-job-534-vz2ns. Will retry 3 more times.
DEBUG 2023/01/27 14:05:25 [nHvktzQU] Detected EOF for pod awx/automation-job-534-vz2ns. Will retry 2 more times.
DEBUG 2023/01/27 14:05:26 [nHvktzQU] Detected EOF for pod awx/automation-job-534-vz2ns. Will retry 1 more times.
DEBUG 2023/01/27 14:05:26 Stdout complete - closing channel for: nHvktzQU
WARNING 2023/01/27 14:05:26 Could not read in control service: read unix /var/run/receptor/receptor.sock->@: use of closed network connection
DEBUG 2023/01/27 14:05:26 Client disconnected from control service @
WARNING 2023/01/27 14:05:26 Could not close connection: close unix /var/run/receptor/receptor.sock->@: use of closed network connection
DEBUG 2023/01/27 14:05:26 Client connected to control service @
DEBUG 2023/01/27 14:05:26 Control service closed
DEBUG 2023/01/27 14:05:26 Client disconnected from control service @
ERROR 2023/01/27 14:05:29 Exceeded retries for reading stdout /tmp/receptor/awx-f894c6856-gslxg/nHvktzQU/stdout
DEBUG 2023/01/27 14:05:32 Client connected to control service @
DEBUG 2023/01/27 14:05:32 Control service closed
DEBUG 2023/01/27 14:05:32 Client disconnected from control service @
ERROR 2023/01/27 14:05:32 write unix /var/run/receptor/receptor.sock->@: write: broken pipe
ERROR 2023/01/27 14:05:32 Write error in control service: write unix /var/run/receptor/receptor.sock->@: write: broken pipe
DEBUG 2023/01/27 14:05:32 Client disconnected from control service @
DEBUG 2023/01/27 14:05:32 Client connected to control service @
DEBUG 2023/01/27 14:05:32 Control service closed
DEBUG 2023/01/27 14:05:32 Client disconnected from control service @
DEBUG 2023/01/27 14:05:47 Sending service advertisement: &{awx-f894c6856-gslxg control 2023-01-27 14:05:47.908558409 +0000 UTC m=+66126.078429070 1 map[type:Control Service] [{local false} {kubernetes-runtime-auth false} {kubernetes-incluster-auth false}]}

iuvooneill · 2023-01-27T14:54:39Z

I can confirm that disabling reconnect support fixed my job failures - I don't have anything that runs over 4 hours at this time.

TheRealHaoLiu · 2023-01-27T16:51:04Z

we digged a further onto this issue and we found

DEBUG 2023/01/26 13:01:44 Kubernetes version v1.24.7-eks-fb459a0 is at least v1.23.14, using reconnect support

the k8s version detection code did not function correctly and incorrectly enabled the fix

TheRealHaoLiu · 2023-01-27T16:52:24Z

another problem that we found during investigation

v1.24.8-eks-ffeb93d

does not contain the fix since long log message still have timestamp inserted in the middle of the message

$$$2023-01-26T21:30:18.149366556Z $$

TheRealHaoLiu · 2023-01-27T16:56:20Z

Hi @yuliym @iuvooneill can u run a test for us

project repo: https://github.com/ansible/test-playbooks
playbook: chatty_payload.yml
vars:

---
num_messages: 10
message_size: 20000000

in the job log look to see if there's any random timestamps in the messages

also please provide the output from kubectl version and kubectl get node

TheRealHaoLiu · 2023-01-27T17:07:29Z

i noticed something interesting on a fresh EKS clusters

➜  awx-operator git:(devel) ✗ oc version
Client Version: 4.9.0
Kubernetes Version: v1.24.8-eks-ffeb93d
➜  awx-operator git:(devel) ✗ oc get node
NAME                                           STATUS   ROLES    AGE   VERSION
ip-192-168-62-80.us-west-2.compute.internal    Ready    <none>   21h   v1.24.7-eks-fb459a0
ip-192-168-71-247.us-west-2.compute.internal   Ready    <none>   21h   v1.24.7-eks-fb459a0

k8s API server and kublet version does not match
the code change we rely on is actually in kubelet (AWX dont have permission to access node information)

yuliym · 2023-01-30T13:03:04Z

Hi @TheRealHaoLiu

num_messages: 10
message_size: 20000000

[vagrant@d72532e09d8a awx-dev]$ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.9", GitCommit:"9dd794e454ac32d97cde41ae10be801ae98f75df", GitTreeState:"clean", BuildDate:"2021-03-18T01:09:28Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.8", GitCommit:"83d00b7cbf10e530d1d4b2403f22413220c37621", GitTreeState:"clean", BuildDate:"2022-11-09T19:50:11Z", GoVersion:"go1.17.11", Compiler:"gc", Platform:"linux/amd64"}
[vagrant@d72532e09d8a awx-dev]$ kubectl get node
NAME                                STATUS   ROLES   AGE     VERSION
aks-agentpool-20093454-vmss000000   Ready    agent   125d    v1.23.8
aks-sre-27287620-vmss00000q         Ready    agent   2d19h   v1.23.8
[vagrant@d72532e09d8a awx-dev]$

playbook: chatty_payload.yml succeed however some other our jobs failed due to Error.
btw output size of chatty_payload.yml 190M. Can't attach it here
but I can confirm there is some timestamp in the log output like:
$$2023-01-30T12:41:41.600402153Z $$

UPD:
Upgraded AKS cluster to v1.24.6 still the same issue. Also re-triggered chatty_payload.yml still see timestamp in output like below:
$$$2023-01-30T15:15:27.349034966Z $$

pratik-pagade · 2023-02-03T07:27:47Z

Hi @TheRealHaoLiu ,

I'm also facing the same issues as described by @Halytskyi. I'm running awx on kubernetes(GKE) installed using AWX-Operator.

We have db refresh jobs( < 4hrs ) that are erroring out w/o log output.
To confirm I tested a simple playbook with couple of sleep commands and resulted in an error w/o log output. Also, the time for which the job runs is random.

Sleep command playbook
`---
#Sleep for 30 mins

name: Really long ssh command
shell:
executable: /bin/bash
cmd: "sleep 1800"

#Sleep for 10 mins

name: Another shorter ssh command
shell:
executable: /bin/bash
cmd: "sleep 600"`

Ansible UI O/P

AWX_EE container logs

AWX_Task container logs

Partial O/P Automation-Job Pod

AWX - v21.0.0(Installed using AWX Operator)
GKE - v1.22.15-gke.1000

alexchronopoulos · 2023-02-08T01:23:43Z

For anyone else running into this, I was able to resolve the issue on our EKS cluster by adjusting the Default Instance Group Pod specification to ensure containers launched are assigned to particular Nodes by following the instructions here.

Our EKS cluster has two Node Groups, one of which was on Kubernetes version 1.23.9 and the other on 1.23.15. The AWX Pods were all set to run on the Node Group on version 1.23.15, however the Pods launched from Jobs were being assigned to the Node Group on version 1.23.9. This resulted in the error described here.

Viewing which Nodes Pods were being assigned to by using kubectl get pods -o wide then checking the version of Kubernetes on those Nodes using kubectl get nodes tipped me off to the discrepancy.

I hope this helps others that are running into this issue with a similar setup.

yuliym · 2023-02-10T11:55:49Z

Unfortunately not our case. We are running AWX on AKS and versions are align

Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.6", GitCommit:"c86d003ea699ec4bcffee10ad563a26b63561c0e", GitTreeState:"clean", BuildDate:"2022-12-17T10:31:53Z", GoVersion:"go1.18.6", Compiler:"gc", Platform:"linux/amd64"}
[vagrant@d72532e09d8a ~]$ kubectl get nodes
NAME                                STATUS   ROLES   AGE     VERSION
aks-agentpool-20093454-vmss000000   Ready    agent   10d     v1.24.6
aks-hub-33629182-vmss00000p         Ready    agent   2m58s   v1.24.6

yuliym · 2023-02-14T20:08:19Z

@TheRealHaoLiu
Issue fixed after upgrade aks to k8s v1.24.9
Looks like fix kubernetes/kubernetes#113516 effective since k8s 1.24.7 and not merged to 1.24.6

I found another issue with reconnect partly related to #13161
Looks like after successful reconnect - reconnect counter does not reset and keep counting from 5 to 1.
So for 5 min timeout issue job will fail in ~25 min.
For 4h timeout issue job will potentially fail in 20h (5 retry * 4h)
Sounds like bug. Isn't it ?

DEBUG 2023/02/14 15:54:37 [6Wf2RDre] Detected EOF for pod awx/automation-job-1985-xmmhz. Will retry 5 more times. Error: EOF
DEBUG 2023/02/14 15:59:38 [6Wf2RDre] Detected EOF for pod awx/automation-job-1985-xmmhz. Will retry 4 more times. Error: EOF
DEBUG 2023/02/14 16:04:38 [6Wf2RDre] Detected EOF for pod awx/automation-job-1985-xmmhz. Will retry 3 more times. Error: EOF
DEBUG 2023/02/14 16:09:38 [6Wf2RDre] Detected EOF for pod awx/automation-job-1985-xmmhz. Will retry 2 more times. Error: EOF
DEBUG 2023/02/14 16:14:38 [6Wf2RDre] Detected EOF for pod awx/automation-job-1985-xmmhz. Will retry 1 more times. Error: EOF

masbahnana · 2023-07-03T20:54:43Z

hi @TheRealHaoLiu this issue is duplicated here let me know if I can close or I keep open (because the duplicated isse is with needs_triage label)

fosterseth · 2023-07-14T01:09:31Z

@yuliym do you know if the retry not resetting problem only occurs for tasks that don't emit new stdout within the 5 minute period (i.e. if there is a sleep 650 task)?

feels like this line should be resetting the count back to 5 after a successful write https://github.com/ansible/receptor/blob/4addde85f132cc555331041e9a6f7963519c542c/pkg/workceptor/kubernetes.go#L299

github-actions bot added component:ui needs_triage type:bug community labels Jan 25, 2023

TheRealHaoLiu assigned TheRealHaoLiu and fosterseth Jan 25, 2023

djyasin removed the needs_triage label Jan 25, 2023

fosterseth mentioned this issue Jan 27, 2023

Do not use reconnect by default ansible/receptor#729

Merged

AlanCoding mentioned this issue Jan 31, 2023

Consume job_explanation from runner, fix error reporting error #13482

Merged

fosterseth mentioned this issue Feb 1, 2023

awx-ee k8s container log showing warnings "Could not close connection: close unix /var/run/receptor/receptor.sock->@: use of closed network connection ansible/awx-operator#1203

Open

3 tasks

This was referenced Feb 2, 2023

Cannot do Inventory Sync due to binascii.Error: Incorrect padding #13509

Closed

AWX Office Hours Agenda - Feb 14 2023 #13431

Closed

yzhivkov mentioned this issue Feb 12, 2023

Jobs error at random in UI (with stripped log output) #13380

Open

9 tasks

hurdigurdi mentioned this issue Mar 10, 2023

[21.3.x] No Job logs for Container Groups other than default #13672

Open

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Job failed with just error and without log output #13469

Job failed with just error and without log output #13469

Halytskyi commented Jan 25, 2023 •

edited

Loading

TheRealHaoLiu commented Jan 25, 2023

TheRealHaoLiu commented Jan 25, 2023

TheRealHaoLiu commented Jan 25, 2023 •

edited

Loading

TheRealHaoLiu commented Jan 25, 2023

TheRealHaoLiu commented Jan 25, 2023 •

edited

Loading

yuliym commented Jan 27, 2023

iuvooneill commented Jan 27, 2023

TheRealHaoLiu commented Jan 27, 2023

TheRealHaoLiu commented Jan 27, 2023

TheRealHaoLiu commented Jan 27, 2023 •

edited

Loading

TheRealHaoLiu commented Jan 27, 2023 •

edited

Loading

yuliym commented Jan 30, 2023 •

edited

Loading

pratik-pagade commented Feb 3, 2023 •

edited

Loading

alexchronopoulos commented Feb 8, 2023

yuliym commented Feb 10, 2023

yuliym commented Feb 14, 2023 •

edited

Loading

masbahnana commented Jul 3, 2023

fosterseth commented Jul 14, 2023

Job failed with just error and without log output #13469

Job failed with just error and without log output #13469

Comments

Halytskyi commented Jan 25, 2023 • edited Loading

Please confirm the following

Bug Summary

AWX version

Select the relevant components

Installation method

Modifications

Ansible version

Operating system

Web browser

Steps to reproduce

Expected results

Actual results

Additional information

TheRealHaoLiu commented Jan 25, 2023

TheRealHaoLiu commented Jan 25, 2023

TheRealHaoLiu commented Jan 25, 2023 • edited Loading

TheRealHaoLiu commented Jan 25, 2023

TheRealHaoLiu commented Jan 25, 2023 • edited Loading

yuliym commented Jan 27, 2023

iuvooneill commented Jan 27, 2023

TheRealHaoLiu commented Jan 27, 2023

TheRealHaoLiu commented Jan 27, 2023

TheRealHaoLiu commented Jan 27, 2023 • edited Loading

TheRealHaoLiu commented Jan 27, 2023 • edited Loading

yuliym commented Jan 30, 2023 • edited Loading

pratik-pagade commented Feb 3, 2023 • edited Loading

alexchronopoulos commented Feb 8, 2023

yuliym commented Feb 10, 2023

yuliym commented Feb 14, 2023 • edited Loading

masbahnana commented Jul 3, 2023

fosterseth commented Jul 14, 2023

Halytskyi commented Jan 25, 2023 •

edited

Loading

TheRealHaoLiu commented Jan 25, 2023 •

edited

Loading

TheRealHaoLiu commented Jan 25, 2023 •

edited

Loading

TheRealHaoLiu commented Jan 27, 2023 •

edited

Loading

TheRealHaoLiu commented Jan 27, 2023 •

edited

Loading

yuliym commented Jan 30, 2023 •

edited

Loading

pratik-pagade commented Feb 3, 2023 •

edited

Loading

yuliym commented Feb 14, 2023 •

edited

Loading