attempt to download signing keys upon artifact installation, new origin keys Depot API

[bookshelfdave]

Overview

Upon artifact install, check to see if we have the signing origin key for an artifact in our local key cache (HAB_CACHE_KEY_PATH). If not, try to download it from the origin and continue with artifact verification. Initially, I had implemented package install to download all keys for an origin. Upon discussion with the team, we decided it would be best for the depot to download a single key at a time, since we have the exact key w/ revision in the .hart header.

This PR adds 4 new routes to the depot API for uploading/download origin keys. Origin keys are stored in a hashed file structure on disk, and metadata is stored as a Redis set with origin_keys as the prefix. For example, origin_keys:core is a set of all keys for the core origin. Redis-backed storage for keys buys us the ability to list all keys for a given origin.

New hab commands:

• hab origin key upload /path/to/some_origin.pub
  • upload an origin key by filename.
  • Upload will not overwrite an existing key.
• hab origin key download some_origin
  • this command downloads all public keys for a specific origin into the HAB_CACHE_KEY_PATH directory.
• hab origin key download some_origin revision
  • this command downloads a single public key w/ revision into the HAB_CACHE_KEY_PATH directory.

Depot API

The new routes in the Depot API are as follows:

List all keys

GET /origins/:origin/keys

Return a list of all keys for an origin in the following format:

Example response:

[
    {
      "origin": "core",
      "revision": "20160423193732",
      "location": "/origins/core/keys/20160423193732"
    },
    {
      "origin": "core",
      "revision": "20160423193733",
      "location": "/origins/core/keys/20160423193733"
    }
]

Get a specific origin key by revision

• GET /origins/:origin/keys/:revision

Returns a plain-text origin public key:

Example response:

SIG-PUB-1
core-20160423193745

Jpmj1gD9oTFCgz3wSLltt/QB6RTmNRWoUTe+xhDTIHc=

Get the newest key revision for an origin

• GET /origins/:origin/keys/latest

Returns a plain-text origin public key:

Example response:

SIG-PUB-1
core-20160423193745

Jpmj1gD9oTFCgz3wSLltt/QB6RTmNRWoUTe+xhDTIHc=

Upload a new origin key with revision

• POST /origins/:origin/keys/:revision

RAML

This PR includes an initial implementation of RAML for the new /origins key routes. The rest of the routes are marked as TODO. I'd love for this to help test and document our API's in the future.

Misc Notes

• set Content-Disposition for package + key download
• removed GPG key installation from test setup
• The crypto module is getting huge and probably deserves a refactor soon.

Sadly, this PR is light on testing. We'll need to get some integration test stuff together soonish. Perhaps a testing spike is in order across the board.

[thesentinels]

By analyzing the blame information on this pull request, we identified @reset, @adamhjk and @fnichol to be potential reviewers

[reset]

It's conventional to put all tests within a module called tests inside the file you're writing the tests for

#[cfg(test)]
mod tests {
  // this line will get you access to the outer module's functions/structs
  use super::*;
  #[test]
  fn my_test() {  }
}

[reset]

@thesentinels r+

[thesentinels]

📌 Commit c3fbab3 has been approved by reset

[thesentinels]

⌛ Testing commit c3fbab3 with merge 032c592...

[thesentinels]

☀️ Test successful - travis