Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bundle: Remove rootfs requirement #469

Conversation

wking
Copy link
Contributor

@wking wking commented May 27, 2016

Recently b2e9154 (Remove requirement for rootfs path to be relative,
2016-04-22, #394) relaxed the relative-path requirement on root.path
to allow absolute paths (see also discussion here). The
motivation being that, while most folks will want a relative-path
pointing into the bundle directory, there are valid use
cases that don't do that, and we want to support them too.

However, allowing absolute paths but requiring them to point to a
directory lying next to config.json seems to defeat the purpose. This
commit removes the "root.path must point to a directory next to
config.json" restriction, so folks who want absolute paths in
root.path can use them without an unnecessary restriction.

And again, most folks will put their rootfs alongside config.json or
elsewhere inside the bundle directory. But this list was about "This
MUST include the following artifacts", and we don't want to require
everyone to do that.

This is the narrowly-scoped version of #423 with the minimum we need
to make #394 a useful change.

Recently b2e9154 (Remove requirement for rootfs path to be relative,
2016-04-22, opencontainers#394) relaxed the relative-path requirement on root.path
to allow absolute paths (see also discussion in [1]).  The motivation
being that, while most folks will want a relative-path pointing into
the bundle directory, there are valid use cases that don't do that,
and we want to support them too [2,3].

However, allowing absolute paths but requiring them to point to a
directory lying next to config.json seems to defeat the purpose.  This
commit removes the "root.path must point to a directory next to
config.json" restriction, so folks who want absolute paths in
root.path can use them without an unnecessary restriction.

And again, most folks will put their rootfs alongside config.json or
elsewhere inside the bundle directory.  But this list was about "This
MUST include the following artifacts", and we don't want to require
*everyone* to do that.

[1]: opencontainers#389
[2]: opencontainers#389 (comment)
[3]: opencontainers#394 (comment)

Signed-off-by: W. Trevor King <[email protected]>
@wking
Copy link
Contributor Author

wking commented May 27, 2016

Oh, and while I think the root entry should be optional, this PR doesn't change that. You still need to set root.path. You just don't have to point it at a target lying next to config.json or within the bundle directory.

@crosbymichael
Copy link
Member

crosbymichael commented May 27, 2016

Rejected

Rejected with PullApprove

@wking
Copy link
Contributor Author

wking commented May 27, 2016

On Fri, May 27, 2016 at 11:53:02AM -0700, Michael Crosby wrote:

Rejected

Reasoning? How do you square that with #389 / #394? Do you really
use absolute paths that always point to a directory next to your
config.json?

@crosbymichael
Copy link
Member

Because the original sentences are correct and describe exactly what should happen/ it should be.

@wking
Copy link
Contributor Author

wking commented May 28, 2016

On Fri, May 27, 2016 at 03:51:05PM -0700, Michael Crosby wrote:

Because the original sentences are correct and describe exactly what
should happen/ it should be.

Following references from #389 takes me to moby/moby#22256. I
haven't traced that back to figure out where spec.Root.Path is being
setup, but before the Docker PR, Docker's libcontainerd was
bind-mounting it to ${BUNDLE}/rootfs 1. After the Docker PR it is
no longer doing that. So I'm not clear on how the “these artifacts
MUST all be present in a single directory on the local filesystem…”
applies to that case.

Or more concretely, with /var/oci/config.json and the following
root.path entries:

a. "rootfs"
b. "./rootfs"
c. "my-rootfs"
d. "./my-rootfs"
e. "my/rootfs"
f. "./my/rootfs"
g. "/var/oci/rootfs"
h. "/var/oci/my-rootfs"
i. "/var/oci/my/rootfs"
j. "/tmp/rootfs

Which are valid? Before #394, I think it was a, b, c, and d. Since
#394 I think it also includes g and h. The “MUST all be present in a
single directory” contraint I'm removing here blocks e, f, i, and j.
And the “MUST include the following” constraint from before the list
blocks j (as long as the rootfs is part of the artifact list). So
after this PR, I expect all of those paths are legal.

And from my rough understanding of moby/moby#21969, I expect
Docker is now using paths like j (absolute paths pointing outside the
bundle directory).

Does that all sound accurate to you? Once we agree on the facts, it
will be easier for me to understand why you think this PR is a bad
idea. Because g and h sound much less useful to me than the
already-legal-before-#394 a, b, c, and d.

@wking
Copy link
Contributor Author

wking commented Jun 22, 2016

On Fri, May 27, 2016 at 05:06:01PM -0700, W. Trevor King wrote:

Or more concretely, with /var/oci/config.json and the following
root.path entries…

ocitools adds another option to this list
(opencontainers/runtime-tools#113):

k. "."

I think that is currently illegal as well because of the “MUST all be
present in a single directory” constraint (the rootfs has a
config.json child, while the MUST requires a config.json sibling).
With this PR, k would also be legal.

Is my understanding of that incorrect? Is k somehow legal with the
current spec wording? Are there reasons for making it illegal?

@wking
Copy link
Contributor Author

wking commented Aug 13, 2016

On Fri, May 27, 2016 at 05:06:01PM -0700, W. Trevor King wrote:

Or more concretely, with /var/oci/config.json and the following
root.path entries:


j. "/tmp/rootfs

Which are valid? Before #394, I think it was a, b, c, and d. Since
#394 I think it also includes g and h. The “MUST all be present in a
single directory” contraint I'm removing here blocks e, f, i, and j.
And the “MUST include the following” constraint from before the list
blocks j (as long as the rootfs is part of the artifact list). So
after this PR, I expect all of those paths are legal.

And from my rough understanding of moby/moby#21969, I expect
Docker is now using paths like j (absolute paths pointing outside the
bundle directory).

I setup a container with Docker version 1.11.2 (build b9f10c9) and
confirmed the j-style path:

jq .bundle /run/containerd/a88d4b17c0f9b0347eb21c3f46ae124b2b38c7cf83495d3f3f60a08972bf1616/state.json

"/var/run/docker/libcontainerd/a88d4b17c0f9b0347eb21c3f46ae124b2b38c7cf83495d3f3f60a08972bf1616"

BUNDLE=$(jq -r .bundle /run/containerd/a88d4b17c0f9b0347eb21c3f46ae124b2b38c7cf83495d3f3f60a08972bf1616/state.json)

ROOT=$(jq -r .root.path $BUNDLE/config.json)

echo $BUNDLE

/var/run/docker/libcontainerd/a88d4b17c0f9b0347eb21c3f46ae124b2b38c7cf83495d3f3f60a08972bf1616

echo $ROOT

/var/lib/docker/btrfs/subvolumes/924b6be428d9dff1b206fb1ff6d4f217608860bfc367cd2052fbb0154e04fa0a

realpath --relative-to $BUNDLE $ROOT

../../../../var/lib/docker/btrfs/subvolumes/924b6be428d9dff1b206fb1ff6d4f217608860bfc367cd2052fbb0154e04fa0a

So I still think we want this PR to drop the requirement that
root.path is a sibling of config.json.

@RobDolinMS
Copy link
Collaborator

@crosbymichael Do @wking's comments address your concern or do you still think this should be rejected?

@vbatts
Copy link
Member

vbatts commented Jan 18, 2017

after chatting about this: this is not behavior folks would expect. there is no sane assumption is there is no rootfs specified. It's a bad idea to assume it would default to '/' on the host.
I reject this too.

@vbatts vbatts closed this Jan 18, 2017
@wking
Copy link
Contributor Author

wking commented Jan 18, 2017 via email

wking added a commit to wking/ocitools-v2 that referenced this pull request Mar 28, 2017
Test the spec's [1]:

  While these artifacts MUST all be present in a single directory on
  the local filesystem, ...

which is a condition it imposes on config.json and the directory
referenced by root.path.

I think we should drop that restriction from the spec, but my attempt
to remove the restriction was rejected [2].  If a future spec drops
the restriction, we can revert this commit.

Using path/filepath for the path manipulation will break when
validating cross-platform configs (e.g. trying to validate a Windows
bundle on a Linux machine).  But that's a bigger issue than this
commit, so I've left it alone for now.

[1]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc5/bundle.md#container-format
[2]: opencontainers/runtime-spec#469

Signed-off-by: W. Trevor King <[email protected]>
wking added a commit to wking/ocitools-v2 that referenced this pull request Mar 29, 2017
Test the spec's [1]:

  While these artifacts MUST all be present in a single directory on
  the local filesystem, ...

which is a condition it imposes on config.json and the directory
referenced by root.path.

I think we should drop that restriction from the spec, but my attempt
to remove the restriction was rejected [2].  If a future spec drops
the restriction, we can revert this commit.

Using path/filepath for the path manipulation will break when
validating cross-platform configs (e.g. trying to validate a Windows
bundle on a Linux machine).  But that's a bigger issue than this
commit, so I've left it alone for now.

[1]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc5/bundle.md#container-format
[2]: opencontainers/runtime-spec#469

Signed-off-by: W. Trevor King <[email protected]>
wking added a commit to wking/ocitools-v2 that referenced this pull request Mar 29, 2017
Test the spec's [1]:

  While these artifacts MUST all be present in a single directory on
  the local filesystem, ...

which is a condition it imposes on config.json and the directory
referenced by root.path.

I think we should drop that restriction from the spec, but my attempt
to remove the restriction was rejected [2].  If a future spec drops
the restriction, we can revert this commit.

Using path/filepath for the path manipulation will break when
validating cross-platform configs (e.g. trying to validate a Windows
bundle on a Linux machine).  But that's a bigger issue than this
commit, so I've left it alone for now.

[1]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc5/bundle.md#container-format
[2]: opencontainers/runtime-spec#469

Signed-off-by: W. Trevor King <[email protected]>
@wking wking mentioned this pull request May 13, 2017
wking added a commit to wking/opencontainer-runtime-spec that referenced this pull request May 18, 2017
Anyone setting 'rootfs.path' should be aware of this advice,
regardless of whether they're the ones composing the bundle or not.

Use the RFC 2119 "SHOULD" (vs. the old lowercase "should") for this
recommendation.  The SHOULD semantics make sense and using SHOULD
avoids confusing readers ("did they mean to SHOULD this?").

Also drop the "While the name of this directory may be arbitrary"
caveat, because SHOULD already implies "but you can pick another
directory name if you want".

Note that the "MUST be present in a single directory" line which is
still in bundle.md forbids you from picking "foo/bar", "../foo", or
other paths that do not point at a direct child of the bundle
directory.  I don't like that direct-child restriction [1], but I'm
not touching it in this commit.

[1]: opencontainers#469

Signed-off-by: W. Trevor King <[email protected]>
wking added a commit to wking/opencontainer-runtime-spec that referenced this pull request May 18, 2017
Instead of repeating the config.md conditions, just make the linkage
more clear.  Now the chain is:

1. Bundles REQUIRE a config.json which is a bundle artifact.
2. If that config.json has a root.path entry (as specified in
   config.md), then add the referenced directory to the set of bundle
   artifacts.  The config.md requirements include "If defined, a
   directory MUST exist at the path declared by the field".
3. Apply the "MUST all be present in a single directory" condition to
   all bundle artifacts.  I don't like that direct-child restriction
   [1], but I'm not touching it in this commit.

So these are the same requirements as before this commit, but with
less redundancy and fewer words.

[1]: opencontainers#469

Signed-off-by: W. Trevor King <[email protected]>
wking added a commit to wking/opencontainer-runtime-spec that referenced this pull request May 22, 2017
Instead of repeating the config.md conditions, just make the linkage
more clear.  Now the chain is:

1. Bundles REQUIRE a config.json which is a bundle artifact.
2. If that config.json has a root.path entry (as specified in
   config.md), then add the referenced directory to the set of bundle
   artifacts.  The config.md requirements include "If defined, a
   directory MUST exist at the path declared by the field".
3. Apply the "MUST all be present in a single directory" condition to
   all bundle artifacts.  I don't like that direct-child restriction
   [1], but I'm not touching it in this commit.

So these are the same requirements as before this commit, but with
less redundancy and fewer words.

[1]: opencontainers#469

Signed-off-by: W. Trevor King <[email protected]>
wking added a commit to wking/opencontainer-runtime-spec that referenced this pull request Jun 1, 2017
Instead of repeating the config.md conditions, just make the linkage
more clear.  Now the chain is:

1. Bundles REQUIRE a config.json which is a bundle artifact.
2. If that config.json has a root.path entry (as specified in
   config.md), then add the referenced directory to the set of bundle
   artifacts.  The config.md requirements include "If defined, a
   directory MUST exist at the path declared by the field".
3. Apply the "MUST all be present in a single directory" condition to
   all bundle artifacts.  I don't like that direct-child restriction
   [1], but I'm not touching it in this commit.

So these are the same requirements as before this commit, but with
less redundancy and fewer words.

[1]: opencontainers#469

Signed-off-by: W. Trevor King <[email protected]>
wking added a commit to wking/opencontainer-runtime-spec that referenced this pull request Jun 16, 2017
Instead of repeating the config.md conditions, just make the linkage
more clear.  Now the chain is:

1. Bundles REQUIRE a config.json which is a bundle artifact.
2. If that config.json has a root.path entry (as specified in
   config.md), then add the referenced directory to the set of bundle
   artifacts.  The config.md requirements include "If defined, a
   directory MUST exist at the path declared by the field".
3. Apply the "MUST all be present in a single directory" condition to
   all bundle artifacts.  I don't like that direct-child restriction
   [1], but I'm not touching it in this commit.

So these are the same requirements as before this commit, but with
less redundancy and fewer words.

[1]: opencontainers#469

Signed-off-by: W. Trevor King <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants