Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added some documentation for encrypting with age #1001

Closed
wants to merge 6 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 26 additions & 5 deletions README.rst
Original file line number Diff line number Diff line change
@@ -189,13 +189,13 @@ the ``--age`` option or the **SOPS_AGE_RECIPIENTS** environment variable:

.. code:: bash
$ sops --age age1yt3tfqlfrwdwx0z0ynwplcr6qxcxfaqycuprpmy89nr83ltx74tqdpszlw test.yaml > test.enc.yaml
$ sops --encrypt --age age1yt3tfqlfrwdwx0z0ynwplcr6qxcxfaqycuprpmy89nr83ltx74tqdpszlw test.yaml > test.enc.yaml
When decrypting a file with the corresponding identity, sops will look for a
text file name ``keys.txt`` located in a ``sops`` subdirectory of your user
configuration directory. On Linux, this would be ``$XDG_CONFIG_HOME/sops/keys.txt``.
On macOS, this would be ``$HOME/Library/Application Support/sops/keys.txt``. On
Windows, this would be ``%AppData%\sops\keys.txt``. You can specify the location
configuration directory. On Linux, this would be ``$XDG_CONFIG_HOME/sops/age/keys.txt``.
On macOS, this would be ``$HOME/Library/Application Support/sops/age/keys.txt``. On
Windows, this would be ``%AppData%\sops\age\keys.txt``. You can specify the location
of this file manually by setting the environment variable **SOPS_AGE_KEY_FILE**.

The contents of this key file should be a list of age X25519 identities, one
@@ -204,7 +204,28 @@ identity will be tried in sequence until one is able to decrypt the data.

Encrypting with SSH keys via age is not yet supported by sops.

A list of age recipients can be added to the ``.sops.yaml``:

.. code:: yaml
creation_rules:
- age: >-
age1s3cqcks5genc6ru8chl0hkkd04zmxvczsvdxq99ekffe4gmvjpzsedk23c,
age1qe5lxzzeppw5k79vxn3872272sgy224g2nzqlzy3uljs84say3yqgvd0sw
It is also possible to use ``updatekeys``, when adding or removing age recipients. For example:

.. code:: sh
$ sops updatekeys secret.enc.yaml
2022/02/09 16:32:02 Syncing keys for file /iac/solution1/secret.enc.yaml
The following changes will be made to the file's groups:
Group 1
age1s3cqcks5genc6ru8chl0hkkd04zmxvczsvdxq99ekffe4gmvjpzsedk23c
+++ age1qe5lxzzeppw5k79vxn3872272sgy224g2nzqlzy3uljs84say3yqgvd0sw
Is this okay? (y/n):y
2022/02/09 16:32:04 File /iac/solution1/secret.enc.yaml synced with new keys
Encrypting using GCP KMS
~~~~~~~~~~~~~~~~~~~~~~~~
GCP KMS uses `Application Default Credentials
@@ -650,7 +671,7 @@ and its KMS and PGP keys are used to encrypt the file. It should be noted that
the looking up of ``.sops.yaml`` is from the working directory (CWD) instead of
the directory of the encrypting file (see `Issue 242 <https://github.com/mozilla/sops/issues/242>`_).
The path_regex checks the full path of the encrypting file. Here is another example:
The path_regex checks the path of the encrypting file relative to the .sops.yaml config file. Here is another example:
* files located under directory **development** should use one set of KMS A
* files located under directory **production** should use another set of KMS B
15 changes: 13 additions & 2 deletions config/config.go
Original file line number Diff line number Diff line change
@@ -8,7 +8,9 @@ import (
"io/ioutil"
"os"
"path"
"path/filepath"
"regexp"
"strings"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR now contains changes that should not be part of it. I guess something went wrong during rebasing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I must have messed it up. Will give it a look again - sorry about the mess @felixfontein

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries, such things happen ;-) Better look at the "File changed" view on GitHub (https://github.com/getsops/sops/pull/1001/files) next time after rebasing to see whether the result looks as expected.


"github.com/sirupsen/logrus"
"go.mozilla.org/sops/v3"
@@ -313,12 +315,20 @@ func parseDestinationRuleForFile(conf *configFile, filePath string, kmsEncryptio
return config, nil
}

func parseCreationRuleForFile(conf *configFile, filePath string, kmsEncryptionContext map[string]*string) (*Config, error) {
func parseCreationRuleForFile(conf *configFile, confPath, filePath string, kmsEncryptionContext map[string]*string) (*Config, error) {
// If config file doesn't contain CreationRules (it's empty or only contains DestionationRules), assume it does not exist
if conf.CreationRules == nil {
return nil, nil
}

configDir, err := filepath.Abs(filepath.Dir(confPath))
if err != nil {
return nil, err
}

// compare file path relative to path of config file
filePath = strings.TrimPrefix(filePath, configDir + string(filepath.Separator))

var rule *creationRule

for _, r := range conf.CreationRules {
@@ -356,7 +366,8 @@ func LoadCreationRuleForFile(confPath string, filePath string, kmsEncryptionCont
if err != nil {
return nil, err
}
return parseCreationRuleForFile(conf, filePath, kmsEncryptionContext)

return parseCreationRuleForFile(conf, confPath, filePath, kmsEncryptionContext)
}

// LoadDestinationRuleForFile works the same as LoadCreationRuleForFile, but gets the "creation_rule" from the matching destination_rule's
51 changes: 34 additions & 17 deletions config/config_test.go
Original file line number Diff line number Diff line change
@@ -75,6 +75,15 @@ creation_rules:
hc_vault_uris: https://foz:443/v1/foz/keys/foz
`)

var sampleConfigWithAmbiguousPath = []byte(`
creation_rules:
- path_regex: foo/*
kms: "1"
pgp: "2"
gcp_kms: "3"
hc_vault_uris: http://4:8200/v1/4/keys/4
`)

var sampleConfigWithGroups = []byte(`
creation_rules:
- path_regex: foobar*
@@ -299,12 +308,12 @@ func TestLoadConfigFileWithGroups(t *testing.T) {
}

func TestLoadConfigFileWithNoMatchingRules(t *testing.T) {
_, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithNoMatchingRules, t), "foobar2000", nil)
_, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithNoMatchingRules, t), "/conf/path", "foobar2000", nil)
assert.NotNil(t, err)
}

func TestLoadConfigFileWithInvalidComplicatedRegexp(t *testing.T) {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithInvalidComplicatedRegexp, t), "stage/prod/api.yml", nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithInvalidComplicatedRegexp, t), "/conf/path", "stage/prod/api.yml", nil)
assert.Equal(t, "can not compile regexp: error parsing regexp: invalid escape sequence: `\\K`", err.Error())
assert.Nil(t, conf)
}
@@ -315,58 +324,58 @@ func TestLoadConfigFileWithComplicatedRegexp(t *testing.T) {
"stage/dev/feature-foo.yml": "dev-feature",
"stage/dev/api.yml": "dev",
} {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithComplicatedRegexp, t), filePath, nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithComplicatedRegexp, t), "/conf/path", filePath, nil)
assert.Nil(t, err)
assert.Equal(t, k, conf.KeyGroups[0][0].ToString())
}
}

func TestLoadEmptyConfigFile(t *testing.T) {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleEmptyConfig, t), "foobar2000", nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleEmptyConfig, t), "/conf/path", "foobar2000", nil)
assert.Nil(t, conf)
assert.Nil(t, err)
}

func TestLoadConfigFileWithEmptyCreationRules(t *testing.T) {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithEmptyCreationRules, t), "foobar2000", nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithEmptyCreationRules, t), "/conf/path", "foobar2000", nil)
assert.Nil(t, conf)
assert.Nil(t, err)
}

func TestLoadConfigFileWithOnlyDestinationRules(t *testing.T) {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithOnlyDestinationRules, t), "foobar2000", nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithOnlyDestinationRules, t), "/conf/path", "foobar2000", nil)
assert.Nil(t, conf)
assert.Nil(t, err)
}

func TestKeyGroupsForFile(t *testing.T) {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfig, t), "foobar2000", nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfig, t), "/conf/path", "foobar2000", nil)
assert.Nil(t, err)
assert.Equal(t, "2", conf.KeyGroups[0][0].ToString())
assert.Equal(t, "1", conf.KeyGroups[0][1].ToString())
conf, err = parseCreationRuleForFile(parseConfigFile(sampleConfig, t), "whatever", nil)
conf, err = parseCreationRuleForFile(parseConfigFile(sampleConfig, t), "/conf/path", "whatever", nil)
assert.Nil(t, err)
assert.Equal(t, "bar", conf.KeyGroups[0][0].ToString())
assert.Equal(t, "foo", conf.KeyGroups[0][1].ToString())
}

func TestKeyGroupsForFileWithPath(t *testing.T) {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithPath, t), "foo/bar2000", nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithPath, t), "/conf/path", "foo/bar2000", nil)
assert.Nil(t, err)
assert.Equal(t, "2", conf.KeyGroups[0][0].ToString())
assert.Equal(t, "1", conf.KeyGroups[0][1].ToString())
conf, err = parseCreationRuleForFile(parseConfigFile(sampleConfigWithPath, t), "somefilename.yml", nil)
conf, err = parseCreationRuleForFile(parseConfigFile(sampleConfigWithPath, t), "/conf/path", "somefilename.yml", nil)
assert.Nil(t, err)
assert.Equal(t, "baggins", conf.KeyGroups[0][0].ToString())
assert.Equal(t, "bilbo", conf.KeyGroups[0][1].ToString())
conf, err = parseCreationRuleForFile(parseConfigFile(sampleConfig, t), "whatever", nil)
conf, err = parseCreationRuleForFile(parseConfigFile(sampleConfig, t), "/conf/path", "whatever", nil)
assert.Nil(t, err)
assert.Equal(t, "bar", conf.KeyGroups[0][0].ToString())
assert.Equal(t, "foo", conf.KeyGroups[0][1].ToString())
}

func TestKeyGroupsForFileWithGroups(t *testing.T) {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithGroups, t), "whatever", nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithGroups, t), "/conf/path", "whatever", nil)
assert.Nil(t, err)
assert.Equal(t, "bar", conf.KeyGroups[0][0].ToString())
assert.Equal(t, "foo", conf.KeyGroups[0][1].ToString())
@@ -375,31 +384,39 @@ func TestKeyGroupsForFileWithGroups(t *testing.T) {
}

func TestLoadConfigFileWithUnencryptedSuffix(t *testing.T) {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithSuffixParameters, t), "foobar", nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithSuffixParameters, t), "/conf/path", "foobar", nil)
assert.Nil(t, err)
assert.Equal(t, "_unencrypted", conf.UnencryptedSuffix)
}

func TestLoadConfigFileWithEncryptedSuffix(t *testing.T) {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithSuffixParameters, t), "barfoo", nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithSuffixParameters, t), "/conf/path", "barfoo", nil)
assert.Nil(t, err)
assert.Equal(t, "_enc", conf.EncryptedSuffix)
}

func TestLoadConfigFileWithUnencryptedRegex(t *testing.T) {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithRegexParameters, t), "barbar", nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithRegexParameters, t), "/conf/path", "barbar", nil)
assert.Equal(t, nil, err)
assert.Equal(t, "^dec:", conf.UnencryptedRegex)
}

func TestLoadConfigFileWithEncryptedRegex(t *testing.T) {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithRegexParameters, t), "barbar", nil)
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithRegexParameters, t), "/conf/path", "barbar", nil)
assert.Equal(t, nil, err)
assert.Equal(t, "^enc:", conf.EncryptedRegex)
}

func TestLoadConfigFileWithInvalidParameters(t *testing.T) {
_, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithInvalidParameters, t), "foobar", nil)
_, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithInvalidParameters, t), "/conf/path", "foobar", nil)
assert.NotNil(t, err)
}

func TestLoadConfigFileWithAmbiguousPath(t *testing.T) {
config := parseConfigFile(sampleConfigWithAmbiguousPath, t)
_, err := parseCreationRuleForFile(config, "/foo/config", "/foo/foo/bar", nil)
assert.Nil(t, err)
_, err = parseCreationRuleForFile(config, "/foo/config", "/foo/fuu/bar", nil)
assert.NotNil(t, err)
}

5 changes: 4 additions & 1 deletion pgp/keysource_test.go
Original file line number Diff line number Diff line change
@@ -44,7 +44,10 @@ func TestPGPKeySourceFromString(t *testing.T) {
}

func TestRetrievePGPKey(t *testing.T) {
fingerprint := "FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4"
// Requires a key available in https://keys.openpgp.org/ *with identity information* (that is, an email address).
// See https://keys.openpgp.org/about/faq#verify-multiple for details about identity information.
// We use the key of [email protected] for here.
fingerprint := "14F26682D0916CDD81E37B6D61B7B526D98F0353"
_, err := getKeyFromKeyServer(fingerprint)
assert.NoError(t, err)
}
2 changes: 1 addition & 1 deletion stores/yaml/store.go
Original file line number Diff line number Diff line change
@@ -70,7 +70,7 @@ func (store Store) nodeToTreeValue(node *yaml.Node, commentsWereHandled bool) (i
return result, nil
case yaml.MappingNode:
branch := make(sops.TreeBranch, 0)
return store.appendYamlNodeToTreeBranch(node, branch, false)
return store.appendYamlNodeToTreeBranch(node, branch, commentsWereHandled)
case yaml.ScalarNode:
var result interface{}
node.Decode(&result)
30 changes: 30 additions & 0 deletions stores/yaml/store_test.go
Original file line number Diff line number Diff line change
@@ -91,6 +91,26 @@ var COMMENT_5 = []byte(`# foo
key: value
`)

// The following is a regression test for https://github.com/mozilla/sops/issues/865
var COMMENT_6 = []byte(`a:
- a
# I no longer get duplicated
- {}
`)

var COMMENT_6_BRANCHES = sops.TreeBranches{
sops.TreeBranch{
sops.TreeItem{
Key: "a",
Value: []interface{}{
"a",
sops.Comment{" I no longer get duplicated"},
sops.TreeBranch{},
},
},
},
}

func TestUnmarshalMetadataFromNonSOPSFile(t *testing.T) {
data := []byte(`hello: 2`)
_, err := (&Store{}).LoadEncryptedFile(data)
@@ -178,6 +198,16 @@ func TestEmpty2(t *testing.T) {
}
*/

func TestComment6(t *testing.T) {
branches, err := (&Store{}).LoadPlainFile(COMMENT_6)
assert.Nil(t, err)
assert.Equal(t, COMMENT_6_BRANCHES, branches)
bytes, err := (&Store{}).EmitPlainFile(branches)
assert.Nil(t, err)
assert.Equal(t, string(COMMENT_6), string(bytes))
assert.Equal(t, COMMENT_6, bytes)
}

func TestEmitValue(t *testing.T) {
// First iteration: load and store
bytes, err := (&Store{}).EmitValue(BRANCHES[0])