Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding additional details for debugging about registry pushes #1558

Merged

Conversation

dgershman
Copy link
Contributor

@dgershman dgershman commented Apr 5, 2023

Description

This enables some additional logging information for pushes to registries.

Related Issue

Relates to #1218

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Other (security config, docs update, etc)

Checklist before merging

@dgershman dgershman changed the title adding additional details for debugging about registry pushes Adding additional details for debugging about registry pushes Apr 5, 2023
@netlify
Copy link

netlify bot commented Apr 5, 2023

Deploy Preview for zarf-docs canceled.

Name Link
🔨 Latest commit 8a9bb0c
🔍 Latest deploy log https://app.netlify.com/sites/zarf-docs/deploys/642db783b6d6b50008c0ce66

Copy link
Contributor

@Racer159 Racer159 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, and thanks for the PR! Requesting this line be added as well since it won't destroy the messages when the spinner completes (if it completes).

Also do you have any log snippets from when you saw the infinite spin with this in place? (was testing locally over this past weekend and while I could significantly slow the Zarf install I never saw it actually infinite spin myself yet).

src/internal/packager/images/push.go Outdated Show resolved Hide resolved
@dgershman
Copy link
Contributor Author

dgershman commented Apr 5, 2023

Looks good overall, and thanks for the PR! Requesting this line be added as well since it won't destroy the messages when the spinner completes (if it completes).

Also do you have any log snippets from when you saw the infinite spin with this in place? (was testing locally over this past weekend and while I could significantly slow the Zarf install I never saw it actually infinite spin myself yet).

I don't have any logs that I can share, but I can work toward that once this feature is released. Essentially it's not necessarily an infinite spin, but images seem to not push consistently. Sometimes the entire deployment will hang on pushing a small image like gitea or one of the flux images. It will timeout and then retry, sometimes either successfully completing or failing and requiring re-running the whole deployment process.

@dgershman dgershman force-pushed the more-debugging-crane-push-logging branch from d9c925d to a8df5be Compare April 5, 2023 17:55
@Racer159
Copy link
Contributor

Racer159 commented Apr 5, 2023

Thanks, that is good to know - I was able to see the retry behavior happen in my tests by intentionally killing the connection to trigger them manually (by deleting or otherwise manipulating the registry pod) - what is odd is that is the opposite of what was reported by @corang (and others) #1218 (comment) - more logs will definitely help since this is likely shaping up to be a multi-faceted issue

Copy link
Contributor

@Racer159 Racer159 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Racer159 Racer159 enabled auto-merge (squash) April 5, 2023 18:49
@Racer159 Racer159 merged commit 287b22e into zarf-dev:main Apr 5, 2023
@dgershman
Copy link
Contributor Author

Thanks, that is good to know - I was able to see the retry behavior happen in my tests by intentionally killing the connection to trigger them manually (by deleting or otherwise manipulating the registry pod) - what is odd is that is the opposite of what was reported by @corang (and others) #1218 (comment) - more logs will definitely help since this is likely shaping up to be a multi-faceted issue

I started a new ticket to track my findings #1568.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants