setuid bit is lost upon install

[miketheman]

• The OS (including version) where you are running any of the Habitat commands.

# uname -a
Linux f9f3298c3c78 4.4.17-moby #1 SMP Fri Aug 12 22:26:18 UTC 2016 x86_64 GNU/Linux
# hab --version
hab 0.9.0/20160815225003

When creating a package that sets a file with a setuid on it, the locally-installed version works correctly.

However, when installing from a package, the setuid bit is lost.

[1][default:/src:0]# cd sudo
[2][default:/src/sudo:0]# build
...
[3][default:/src/sudo:0]# ls -la $(hab pkg path miketheman/sudo)/bin/sudo
-rwsr-xr-x 1 root root 158816 Aug 18 15:05 /hab/pkgs/miketheman/sudo/1.8.17p1/20160818150414/bin/sudo
[4][default:/src/sudo:0]# rm -fr /hab/pkgs/miketheman/sudo
[5][default:/src/sudo:0]# hab pkg install results/miketheman-sudo-1.8.17p1-20160818150414-x86_64-linux.hart
↓ Downloading miketheman-20160616143833 public origin key
    81 B / 81 B | [===================================================================================================================] 100.00 % 1.90 MB/s
☑ Cached miketheman-20160616143833 public origin key
→ Using core/acl/2.2.52/20160612075215
→ Using core/attr/2.4.47/20160612075207
→ Using core/coreutils/8.25/20160729192409
→ Using core/gcc-libs/5.2.0/20160612075020
→ Using core/glibc/2.22/20160612063629
→ Using core/gmp/6.1.0/20160612064724
→ Using core/libcap/2.24/20160612075226
→ Using core/linux-headers/4.3/20160612063537
→ Using core/ncurses/6.0/20160612075116
→ Using core/vim/7.4.1089/20160729215447
✓ Installed miketheman/sudo/1.8.17p1/20160818150414
★ Install of miketheman/sudo/1.8.17p1/20160818150414 complete with 1 new packages installed.
[6][default:/src/sudo:0]# ls -la $(hab pkg path miketheman/sudo)/bin/sudo
-rwxr-xr-x 1 root root 158816 Aug 18 15:05 /hab/pkgs/miketheman/sudo/1.8.17p1/20160818150414/bin/sudo

To be perfectly clear:
-rwsr-xr-x turns into -rwxr-xr-x

Possibly related to other unexpected libarchive behaviors: #1100

[stevendanna]

The libarchive crate is very well documented and has some comments that may be helpful:

https://github.com/chef/libarchive-rust/blob/master/src/archive.rs#L201-L212

[reset]

@miketheman this should be resolved by the changes @fujin made in #1187. Can you verify before I close this ticket?

[miketheman]

@reset @fujin I can confirm that after (rather confusing) rebuild of make shell off master, installing the package retains the original permissions:

root@a296e9abca2c:/src/components/hab# /src/target/debug/hab pkg install core/sudo
» Installing core/sudo
↓ Downloading core/sudo/1.8.17p1/20160818134624
....
root@a296e9abca2c:/src/components/hab# ls -la /hab/pkgs/core/sudo/1.8.17p1/20160818134624/bin/sudo
-rwsr-xr-x 1 root root 158816 Aug 18 13:47 /hab/pkgs/core/sudo/1.8.17p1/20160818134624/bin/sudo

Thanks!!!

At what point to we need to start thinking about dealing with packages as "this only really works if you are using hab 0.10.x and higher"?

[reset]

@miketheman thanks for verifying, appreciate it!

I'm not sure what you mean about that last part. Could you elaborate?

[fujin]

thanks @miketheman <3

[miketheman]

@reset Certainly, sorry if I wasn't clearer.
Currently, there exists a core/sudo package in the depot that cannot be used with hab 0.9.0, since it doesn't include this fix.
Admittedly, I wrote that package, and nobody else is likely using it yet, but it raises the issue of:
If there's a package that will only work as of a certain release of hab, should we surface this somehow?

[reset]

@miketheman it's our goal to ensure that Habitat is backwards compatible with any package created by the Habitat build tools regardless of which version used, especially once we hit 1.0. This was just a bug in our package extraction.

We can release a new patch version of Habitat which includes just this fix if it's biting people hard. Our next planned release is Thursday, September 1st

[miketheman]

@reset I agree with the goal, wholeheartedly. Habitat itself is backwards compatible with this package, but the package is not usable.

This bug has exposed a quality that makes an existing built package unusable by the current release.
This may have been discovered at any point in the release cycle, even after 1.0.
It's how we treat these kind of discoveries is what I'm trying to surface from this and hopefully develop some sort of prepared response for when it happens again (and it will, because something always comes up).

The RubyGems way is to specify the minimum required Ruby version in the gem's spec, like so:

spec.required_ruby_version = '~> 2.0'

I'm not suggesting we adopt this syntax, only pointing out how one packaging system has solved a requirement on the base-layer dependency.

I'm not asking for a release to solve this bugfix - that's too narrow a scope. I'm lobbying for figuring out how the habitat project reacts to such a discovery of a released package not working with a released version of habitat.

[reset]

@miketheman this isn't necessary for backwards compatibility, but it is for forwards compatibility and potentially for plan syntax.

We can sustain backwards compatibility because our artifact is structured, like a SQL database. And, like a SQL database, we can determine the version of it's structure and then perform a forwards migration on it. There isn't a way to guarantee forward compatibility with migrations, though, so we do have a bit of a problem on our hands there.

That isn't to say we can't make improvements when we need them, though. If we were to bake the version of Habitat used to build the .hart into metadata. This would allow older versions of the Habitat Supervisor to know that it isn't compatible with the latest package version which would not allow the package to be installed preventing both an operator error or an automatic upgrade issue. We don't have this just yet, but we can add it when we need it.

Plan syntax is a different story and might need something like the required version field. When we speak about the plan as the artifact it's potentially more difficult to provide a transform for since it's user generated, However, since we used the native language of the building OS itself, it's much easier to stay compatible. Something drastic would really need to happen since we can only define functions and not macros or keywords.

I think it's definitely worth looking at the case of forwards compatibility and plan syntax when they are about to become a problem. For now, it's safe to say they are both at v1 of their API.