Support windows images for create-builder

[ameyer-pivotal]

Adds support for creating Windows-based builder images. There's no guarantee at the moment that pack build works with a Windows builder yet, as that work is underway. However, this PR adds the ability to create a true OCI image (that contains buildpacks and lifecycle binaries, etc) that can be stored in a registry or on a Windows daemon. We will hold off on documenting any Windows support until pack build is fully supported.

Relies on a new feature of imgutil, an OS-specific layer writer for Windows. This PR generalizes some of the tar/layer-related code to make use of a factory that can decide which type of writer is appropriate for a given image OS.

Resolves #469

[natalieparellano]

@ameyer-pivotal @micahyoung this is so exciting! Consolidating the Windows logic in the layer writer is super clean. I left a few comments, mostly questions to further my own understanding.

[natalieparellano]

IIRC Windows requires parent paths to be created first whereas Linux does not.

[ameyer-pivotal]

I believe you're correct. Are you noticing a code defect or simply clarifying for other readers?

[natalieparellano]

Ah, the latter!

[natalieparellano]

@ameyer-pivotal @micahyoung do you have a preferred stack and/or dummy buildpack to use for user acceptance testing? I started playing around with this today - I figured I'd use the nanoserver-1809 stack in the samples repo and it appears that is not supported by pack - getting this error:

ERROR: invalid build-image: failed to parse CNB_USER_ID, value S-1-1-0 should be an integer

From what I understand, the user id in the stack image is an arbitrary value for now - could we change it to be an integer? Or should pack be more flexible to accept something that Windows would consider valid?

[ameyer-pivotal]

Hi @natalieparellano, thanks for taking a look at this :) You should be able to build the build/run images inside the acceptance/testdata/mock_stack/windows directory. That's what I've been using locally. Let me know if you still have trouble.

Side note: For now, we set the CNB_USER_ID, etc., to zero because Windows doesn't allow us to assign a user ID (it's autogenerated by the OS). We still have to figure out a better solution for that later.

[ameyer-pivotal]

@natalieparellano Also note that the version of nanoserver on that stack I mentioned is 1809. If my math is right, you have to use a version of Windows 10 that is 1809 or higher as your Docker host. We've had success with a VM running Windows 10 and using the DOCKER_HOST env var on our Macs to point to the VM. If you want to go that route, here's a helpful gist: https://gist.github.com/micahyoung/d8649ec8a5ac6493333dc6d02722d501

[codecov]

Codecov Report

Merging #571 into master will increase coverage by 0.14%.
The diff coverage is 63.95%.

@@            Coverage Diff             @@
##           master     #571      +/-   ##
==========================================
+ Coverage   70.44%   70.59%   +0.14%     
==========================================
  Files          64       66       +2     
  Lines        3695     3710      +15     
==========================================
+ Hits         2603     2619      +16     
+ Misses        762      757       -5     
- Partials      330      334       +4     

Flag	Coverage Δ
#os_linux	70.59% <63.95%> (+0.14%)	⬆️
#os_macos	66.41% <63.95%> (+0.10%)	⬆️
#os_windows	?
#unit	70.59% <63.95%> (+0.14%)	⬆️

Impacted Files	Coverage Δ
internal/layer/layer.go	0.00% <0.00%> (ø)
internal/archive/archive.go	50.56% <43.75%> (+4.57%)	⬆️
create_builder.go	76.62% <50.00%> (-0.87%)	⬇️
package_buildpack.go	56.75% <50.00%> (ø)
build.go	73.75% <66.66%> (-0.41%)	⬇️
internal/archive/tar_builder.go	55.10% <71.42%> (ø)
internal/builder/builder.go	66.84% <71.87%> (-0.18%)	⬇️
internal/layer/writer_factory.go	77.77% <77.77%> (ø)
internal/dist/buildpack.go	81.03% <100.00%> (+0.16%)	⬆️
... and 1 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 70d22be...470787a. Read the comment docs.

[ameyer-pivotal]

@jromero Note that with the CI changes coming in this PR, you'll want to mark build / test as required for both windows-wcow and windows-lcow, now that those two fanned out of windows.

[jromero]

@ameyer-pivotal and I ran through user acceptance on a macOS communicating to a WCOW docker daemon and on a Windows VM using a local docker daemon.

There were no issue which should block this PR but there were a few things to note.

• directory-based buildpacks are not supported (see code)
  • As a buildpack author on Windows, I am not able to iterate on buildpacks from directories
  • This feature is intentionally not supported on Windows due to permission issues. There may be possible changes made to enable this if users and project feels strongly in the future.
• buildpackage support (What do Windows-based buildpackages look like? #604)
  • There is no easy way to test a builder.toml with build packages due to the fact that buildpackages for Windows story is not yet complete.
• default lifecycle OS variant (WCOW: Should download Windows variant of lifecycle #605)
  • pack create-builder should download the OS variant of lifecycle for the builder OS
  • When running create-builder the default lifecycle when downloaded does not take into consideration the docker target OS of the builder image
• experimental flag (Gate WCOW behind experimental flag #607)
  • We agreed that Windows support should be labeled as experimental and thereby behind the experimental flag.