Cannot manually compile with 'bmake install' due to missing signatures

[jacobvosmaer]

Hi, I am trying to manually compile lang/clang on OS X (because no binary package is provided in Darwin/trunk). This fails with the following error:

=> Installing llvm-3.8.0{,nb*} from /Users/jacobvosmaer/pkgsrc-packages/All;http://pkgsrc.joyent.com/packages/Darwin/trunk/x86_64//All
pkg_add: No valid signature found, rejected
pkg_add: 1 package addition failed
=> No binary package found for llvm-3.8.0{,nb*}; installing from source.
=> Bootstrap dependency digest>=20010302: found digest-20160304
===> Checking for vulnerabilities in llvm-3.8.0nb2
===> Installing binary package of llvm-3.8.0nb2
pkg_add: No valid signature found, rejected
pkg_add: 1 package addition failed
*** Error code 1

I tried following the instructions in https://pkgsrc.joyent.com/docs/building/ when configuring the pkgsrc tree.

I think it makes sense what happens here; bmake install creates a temporary package that it feeds to pkg_add and pkg_add uses VERIFIED_INSTALLATION=always via /opt/pkg/etc/pkg_install.conf. What I don't understand is how this is supposed to not go wrong. If I were able to locally produce packages that are trusted by /opt/pkg/etc/gnupg/pkgsrc.gpg that would surprise me. :)

Is there maybe some step missing in the 'Building packages' documentation?

[jperkin]

Yeh I need to document this better. The problem is that the pkg_install.conf that I ship by default requires signed packages (VERIFIED_INSTALLATION=always), but creating signed packages requires some additional setup and to configure pkg_add to permit the key that you setup (as obviously you can't sign as me!) I'll try to get to this at some point soon, but in the meantime you can remove the VERIFIED_INSTALLATION line and allow your unsigned packages.

[jperkin]

I'll also fix the issue causing clang to not be in the repository for the next build. It's due to the build not fitting in the default ramdisk (http://us-east.manta.joyent.com/pkgsrc/public/reports/Darwin/trunk/x86_64/20160812.1847/llvm-3.8.1/build.log) so I'll move it to local disk.

[jacobvosmaer]

@jperkin awesome!

I will poke around a bit to see if I can figure out the 'sign my own packages' thing.

[jacobvosmaer]

Some things I find hard re configuring signing packages created by bmake install, in case this helps.

• gpg --import as described in https://gist.github.com/drscream/c45419950d8af648e2c6 does not work because/when I try to use an already existing key pair: it says the pubkey is already in the (main) keyring
• getting gpg-agent to work across sudo is finnicky
• having to do anything with gpg in the first place is... challenging for me. I don't use it every day and every time I do it is no fun.

So I see why this would be hard to document. :( Sorry I can't be of more help.

[jacobvosmaer]

This works but it is not ideal:

VERIFIED_INSTALLATION=trusted

In pkg_install.conf. That throws up a prompt that lets me ignore the missing signature during bmake install.

[drscream]

@jacobvosmaer could you share your exact error by using the gpg --import?

Because I run everything as root on the build zone I need to import the key which I have created on one of my own machines. If you use your key which should be already imported in the keyring you don't need todo that again.

[jperkin]

Yeh, it can be tricky to get right and there are some GPG corner cases where e.g. it's impossible to correctly choose a signing key when you have multiple imported (especially annoying when you want to use the same machines for different builds).

[jacobvosmaer]

@drscream hi I just noticed that is your gist I link to :)

I use a primitive OS without zones (OS X) where my build user is my normal user. That gives me this:

$ /opt/pkg/bin/gpg2 --primary-keyring /opt/pkg/etc/gnupg/pkgsrc.gpg --import cjv.pub 
gpg: key A5386FBD: "Jacob Vosmaer <[email protected]>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1

So the issue here is that I'm reusing that key. Giving root it's own GPG keyring is not a crazy idea, I might try that. But to be honest I am OK with VERIFIED_INSTALLATION=trusted for my use case (personal laptop) so I am not sure if I want to spend much more time fighting GPG. :)

I am not quite sure how to resolve this GitHub issue here I opened. For me, the problem is solved, but the documentation on https://pkgsrc.joyent.com/docs/building/ is still incomplete.

Feel free to close this @jperkin

[rmustacc]

github autoclose, re-opening

[CodingMarkus]

Why not making

VERIFIED_INSTALLATION=trusted

default, then close this issue, and everyone is happy, right?

Every browser allows users to override cert failures as well and its always up to the user to put their system at risk. And in the current state, it's impossible for users to build on packets which makes using pkgsrc more or less pointless as only a small subset of packages is pre-built and users will just use macports, fink, or brew as these provide zero hassle.

[jperkin]

I have a few thoughts on this, but the most important thing to say is that my binary repositories will always attempt to build everything in pkgsrc, so if a package is not available, then by definition it's currently broken, and there will be a corresponding failure log listed in the bulk build report posted to pkgsrc-bulk. This will mean that any attempt to build from source will also fail, and the user will enter the wonderful world of pkgsrc development and having to resolve all of the issues in the third-party software. Changing the value of VERIFIED_INSTALLATION will be only one of the many steps necessary to get going, and if users are serious about getting involved in pkgsrc work then they will almost certainly be better off building their own bootstrap and packages so that they are not tied in to having to reproduce my exact environment.

I have a number of issues with setting VERIFIED_INSTALLATION=trusted. First and foremost my binary package repositories are designed to be as simple and secure as possible. Giving the user the option of potentially installing compromised software isn't one I want to delegate to them. At no point should I be shipping unsigned packages, and so always is the best option so that there is a clear failure mode when attempting to install an unsigned or invalid package. It's also unclear how the pkg_add prompt would work from the pkgin environment (which certainly isn't designed for interaction), especially when running in unattended modes (e.g. pkgin -y install or even when being called from ansible/salt/chef/etc).

Since this issue was opened I've added a bunch more docs on the wiki, one of which is how to configure signed packages, so I think we can just close this now. As always if there are parts of the documentation that aren't clear then please let me know.