Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(RK-381) Do not recurse into symlinked dirs when finding files to purge #1233

Merged
merged 1 commit into from
Oct 14, 2021

Conversation

Magisus
Copy link
Collaborator

@Magisus Magisus commented Oct 12, 2021

Previously, if an environent contained a symlink pointing to another
location in the environment, r10k would purge the files pointed to, even
if they were being tracked by the environment. This occurred because we
would check whether such files were purgable twice: once at their real
location, and again via the symlink. The former would be flagged for
keeping, but the latter would be flagged for purging, because git does
not follow symlinks when listing tracked files. This means that the
non-symlinked path is considered desired (because Git "tracks" it), but
the symlink path is not, because Git does not report it as tracked.

This commit fixes this bug by not recursing into directories that are
also symlinks, meaning that the dir's contents will not show up in the
"purgeable" list. Note that symlinks pointing to files (not dirs) are
not impacted, because in this case the two paths are both explicitly
tracked by Git, since it tracks the symlink itself.

Please add all notable changes to the "Unreleased" section of the CHANGELOG in the format:

- (JIRA ticket) Summary of changes. [Issue or PR #](link to issue or PR)

@Magisus Magisus requested a review from a team October 12, 2021 23:57
@Magisus
Copy link
Collaborator Author

Magisus commented Oct 12, 2021

I think there's maybe one more test I need to add. And let me know if anyone else sees any other corner cases!

@Magisus
Copy link
Collaborator Author

Magisus commented Oct 13, 2021

Added tests:

  • symlink we want to keep, pointing to a file
  • symlink we want to keep, pointing to a dir
  • symlink we want to purge, pointing to a file
  • symlink we want to purge, pointing to a dir

In all cases, the things pointed to are not purged. In the latter two, the symlinks themselves are purged, leaving the source files alone.

'spec/fixtures/unit/util/purgeable/managed_one/managed_subdir_1/subdir_expected_1',
'spec/fixtures/unit/util/purgeable/managed_one/managed_subdir_1/subdir_new_1',
'spec/fixtures/unit/util/purgeable/managed_one/managed_subdir_1/managed_symlink_file',
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only "managed" symlinks appear in this list of desired contents, which is what makes them "managed".

Previously, if an environent contained a symlink pointing to another
location in the environment, r10k would purge the files pointed to, even
if they were being tracked by the environment. This occurred because we
would check whether such files were purgable twice: once at their real
location, and again via the symlink. The former would be flagged for
keeping, but the latter would be flagged for purging, because git does
not follow symlinks when listing tracked files. This means that the
non-symlinked path is considered desired (because Git "tracks" it), but
the symlink path is not, because Git does not report it as tracked.

This commit fixes this bug by not recursing into directories that are
also symlinks, meaning that the dir's contents will not show up in the
"purgeable" list. Note that symlinks pointing to files (not dirs) are
not impacted, because in this case the two paths are both explicitly
tracked by Git, since it tracks the symlink itself.
expect(File.exist?("#{managed_directory}/managed_subdir_1")).to be true
expect(File.exist?("#{managed_directory}/managed_symlink_dir")).to be true
expect(File.exist?("#{managed_directory}/managed_subdir_1/subdir_expected_1")).to be true
expect(File.exist?("#{managed_directory}/managed_subdir_1/managed_symlink_file")).to be true
Copy link
Collaborator Author

@Magisus Magisus Oct 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When run without this change, this line and the one above it fail, because managed_subdir_1 got cleaned out when managed_synlink_dir was recursed into, and the mismatched path names added to stale_contents. This reproduces what the user saw in the original escalation.

Comment on lines +27 to +31
expect(File.exist?("#{managed_directory}/expected_1")).to be true
expect(File.exist?("#{managed_directory}/managed_subdir_1")).to be true
expect(File.exist?("#{managed_directory}/managed_symlink_dir")).to be true
expect(File.exist?("#{managed_directory}/managed_subdir_1/subdir_expected_1")).to be true
expect(File.exist?("#{managed_directory}/managed_subdir_1/managed_symlink_file")).to be true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be nice to just do

Suggested change
expect(File.exist?("#{managed_directory}/expected_1")).to be true
expect(File.exist?("#{managed_directory}/managed_subdir_1")).to be true
expect(File.exist?("#{managed_directory}/managed_symlink_dir")).to be true
expect(File.exist?("#{managed_directory}/managed_subdir_1/subdir_expected_1")).to be true
expect(File.exist?("#{managed_directory}/managed_subdir_1/managed_symlink_file")).to be true
desired_contents.each do |path|
expect(File.exist?(path)).to be true
end

Copy link
Collaborator Author

@Magisus Magisus Oct 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had that first and the problem is that if it fails, all you get is:

Failure/Error: expect(File.exist?(path)).to be true

expected true
got false

Which is utterly useless.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh that's so annoying

@mwaggett mwaggett merged commit 4576138 into puppetlabs:main Oct 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants