Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue #704 #973

Merged
merged 6 commits into from
Jan 23, 2020
Merged

Conversation

cvgw
Copy link
Contributor

@cvgw cvgw commented Jan 17, 2020

Fixes #704

Description
Previously whiteout files were not extracted from cached layers and not included in the list of extracted files. This meant that if a file was deleted in a cached layer it would "reappear" in the final image as it was missing the whiteout file.

This change adds a new method GetFSFromLayers to which GetFSFromImage now delegates. GetFSFromLayers accepts a series of options, one of which IncludeWhiteout will cause whiteout files to be extracted and returned in the list of extracted files.

Additionally, this change updates CachingRunCommand and CachingCopyCommand to use GetFSFromLayers and pass the option to include the whiteout files. This results in the whiteout files being extracted from cached layers and the deleted file properly removed from the final image.

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

  • Includes unit tests
  • Adds integration tests if needed.

See the contribution guide for more details.

Reviewer Notes

  • The code flow looks good.
  • Unit tests and or integration tests added.

Release Notes

Describe any changes here so maintainer can include it in the release notes, or delete this block.

- Fix bug with deleted files and cached run and copy commands

cvgw added 2 commits January 17, 2020 13:36
* add util.GetFSFromLayers
* GetFSFromImage delegates to GetFSFromLayers
* add FSOpts and FSConfig for GetFSFromLayers
* add tests for GetFSFromLayers
* add gomock for test support
* add mock_v1 for layers
Update caching run and copy commands to use the new
GetFSFromLayers method and include the whiteout option so that
whiteout files are extracted and included in extractedFiles
@googlebot googlebot added the cla: yes CLA signed by all commit authors label Jan 17, 2020
cvgw added 4 commits January 17, 2020 14:28
for some reason hack/boilerplate.sh failed after using `addlicense`
so I copy/pasted the license header from another file
}
}

func ExtractFunc(extractFunc ExtractFunction) FSOpt {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't wrap my head around ExtractFunc taking an ExtractFunction type with parameter name extractFunc. I don't have a recommendation either since my go skills are rather poor so just fyi.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT about func ExtractFuncOpt(f ExtractFunction) FSOpt

Copy link
Contributor

@samos123 samos123 Jan 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah I now get it was well, might just be me not getting it the other night, never seen this pattern before. I think keeping as is is fine

Copy link
Contributor

@samos123 samos123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


func IncludeWhiteout() FSOpt {
return func(opts *FSConfig) {
opts.includeWhiteout = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is always true. Do you see any future this being configured via command line? i am wondering if we could make the code simpler and avoid this config.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see it ever being configured via command line since a single process will call FSFromLayers with different options.

This is always true because the zero value is false. So by not including this option it is set to false.

I might be misunderstanding

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I could see an argument for passing the value of IncludeWhitespace as an arg to the function so it could be switched from true to false but that seems like something that could be changed later if the need arisea

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i meant the opposite. Lets remove opts.includeWhiteout totally

Copy link
Contributor Author

@cvgw cvgw Jan 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand. When GetFSFromImage calls GetFSFromLayers includeWhiteout should be false. When CachingCopyCommand or CachingRunCommand calls GetFSFromLayers includeWhiteout should be true.

so includeWhiteout has to be configurable in some regards. Are you objecting to having it be configurable or this specific configuration pattern?

return GetFSFromLayers(root, layers, ExtractFunc(extract))

util.IncludeWhiteout(),

if !cfg.includeWhiteout {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, i think it did not occur to me includeWhiteout was false due to the opts .. FSOpt array.
That explains.

	return GetFSFromLayers(root, layers, ExtractFunc(extract))
}

func GetFSFromLayers(root string, layers []v1.Layer, opts ...FSOpt) ([]string, error) {

@cvgw cvgw merged commit a2aae62 into GoogleContainerTools:master Jan 23, 2020
@cvgw cvgw deleted the u/cgwippern/fix-issue-704 branch January 23, 2020 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes CLA signed by all commit authors
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Removed files re-appear when using cache
4 participants