Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

security: Encrypt region boundary keys, Part 1 - helper functions #2931

Merged
merged 22 commits into from
Sep 16, 2020
Merged

security: Encrypt region boundary keys, Part 1 - helper functions #2931

merged 22 commits into from
Sep 16, 2020

Conversation

yiwu-arbug
Copy link

Signed-off-by: Yi Wu [email protected]

What problem does this PR solve?

This is part 1 for adding TDE support to PD. pingcap/tidb#18262 It contains helper methods for encryption.

What is changed and how it works?

This PR adds utility types and functions:

  • encrypt/decrypt data using aes-ctr (for encrypting region boundary keys) and aes-gcm (for encrypting data encryption keys)
  • helper methods to encrypt/decrypt region boundary keys in-place
  • wrapper of master key, and helper method to read a master key (must be a 256 bit encryption key, stored as hex-string) from a local file

Check List

Tests

  • Unit test

Related changes

Release note

  • No release note

yiwu-arbug
yiwu-arbug commented Sep 9, 2020
View reviewed changes
go.mod Outdated Show resolved Hide resolved
@yiwu-arbug yiwu-arbug changed the title Encrypt region boundary keys, part 1 Encrypt region boundary keys, Part 1 Sep 9, 2020
@nolouch nolouch added the component/security Security logic. label Sep 9, 2020
HunDunDM
HunDunDM reviewed Sep 9, 2020
View reviewed changes
go.mod Outdated Show resolved Hide resolved
Yisaer
Yisaer reviewed Sep 9, 2020
View reviewed changes
pkg/encryption/region_crypter_test.go Outdated Show resolved Hide resolved
pkg/encryption/region_crypter.go Outdated Show resolved Hide resolved
pkg/encryption/region_crypter.go Show resolved Hide resolved
pkg/encryption/master_key_test.go Show resolved Hide resolved
pkg/encryption/master_key_test.go Show resolved Hide resolved
pkg/encryption/crypter.go Outdated Show resolved Hide resolved
Yi Wu added 6 commits September 15, 2020 12:18
@yiwu-arbug
Copy link
Author

comments addressed or answered. PTAL.

@yiwu-arbug yiwu-arbug changed the title Encrypt region boundary keys, Part 1 Encrypt region boundary keys, Part 1 - helper functions Sep 15, 2020
Yisaer
Yisaer approved these changes Sep 16, 2020
View reviewed changes
Copy link
Contributor

@Yisaer Yisaer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 16, 2020
@Yisaer Yisaer requested a review from HunDunDM September 16, 2020 04:30
@yiwu-arbug yiwu-arbug merged commit 0a15643 into tikv:master Sep 16, 2020
@yiwu-arbug yiwu-arbug deleted the enc_util branch September 16, 2020 21:29
@yiwu-arbug yiwu-arbug changed the title Encrypt region boundary keys, Part 1 - helper functions security: Encrypt region boundary keys, Part 1 - helper functions Sep 16, 2020
@yiwu-arbug yiwu-arbug mentioned this pull request Sep 16, 2020
Merged
HunDunDM
HunDunDM reviewed Sep 17, 2020
View reviewed changes
pkg/encryption/crypter.go Show resolved Hide resolved
pkg/encryption/crypter.go Show resolved Hide resolved
pkg/encryption/crypter.go Show resolved Hide resolved
pkg/encryption/crypter.go Show resolved Hide resolved
pkg/encryption/crypter.go Show resolved Hide resolved
pkg/encryption/crypter.go Show resolved Hide resolved
pkg/encryption/master_key.go Show resolved Hide resolved
pkg/encryption/master_key.go Show resolved Hide resolved
@yiwu-arbug yiwu-arbug mentioned this pull request Oct 12, 2020
Merged
@yiwu-arbug yiwu-arbug mentioned this pull request Nov 3, 2020
Merged
ti-chi-bot pushed a commit that referenced this pull request Nov 13, 2020
security: Encrypt region boundary keys, Part 4 - KMS (#3141)
3c62563 
* Encrypt region boundary keys, Part 2 - server changes

Signed-off-by: Yi Wu <[email protected]>

* update errno and config template

Signed-off-by: Yi Wu <[email protected]>

* fix typo

Signed-off-by: Yi Wu <[email protected]>

* fix tests

Signed-off-by: Yi Wu <[email protected]>

* fix tests

Signed-off-by: Yi Wu <[email protected]>

* fix tests

Signed-off-by: Yi Wu <[email protected]>

* fix tests

Signed-off-by: Yi Wu <[email protected]>

* address comment in #2931

Signed-off-by: Yi Wu <[email protected]>

* fix loadRegion

Signed-off-by: Yi Wu <[email protected]>

* rename encryption_key_manager package

Signed-off-by: Yi Wu <[email protected]>

* fix lint

Signed-off-by: Yi Wu <[email protected]>

* fix lint

Signed-off-by: Yi Wu <[email protected]>

* fix comments

Signed-off-by: Yi Wu <[email protected]>

* fix comment

Signed-off-by: Yi Wu <[email protected]>

* use option pattern

Signed-off-by: Yi Wu <[email protected]>

* move loadRegion and saveRegion

Signed-off-by: Yi Wu <[email protected]>

* revert changes

Signed-off-by: Yi Wu <[email protected]>

* fix doc

Signed-off-by: Yi Wu <[email protected]>

* address comment

Signed-off-by: Yi Wu <[email protected]>

* key manager

Signed-off-by: Yi Wu <[email protected]>

* add test and refactor

Signed-off-by: Yi Wu <[email protected]>

* make EncryptionKeysPath a const

Signed-off-by: Yi Wu <[email protected]>

* fix region_crypter key manager nil check

Signed-off-by: Yi Wu <[email protected]>

* save conflict test

Signed-off-by: Yi Wu <[email protected]>

* sanity check keys revision

Signed-off-by: Yi Wu <[email protected]>

* kms

Signed-off-by: Yi Wu <[email protected]>

* test set ciphertextKey

Signed-off-by: Yi Wu <[email protected]>

* test set ciphertextKey

Signed-off-by: Yi Wu <[email protected]>

* clone region only when needed

Signed-off-by: Yi Wu <[email protected]>

* add test for config

Signed-off-by: Yi Wu <[email protected]>

* update errors

Signed-off-by: Yi Wu <[email protected]>

* fix lint

Signed-off-by: Yi Wu <[email protected]>

* fix lint

Signed-off-by: Yi Wu <[email protected]>

* make errdoc

Signed-off-by: Yi Wu <[email protected]>

* address comment

Signed-off-by: Yi Wu <[email protected]>

* fix error type

Signed-off-by: Yi Wu <[email protected]>

* fix test

Signed-off-by: Yi Wu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HunDunDM HunDunDM HunDunDM left review comments

@Yisaer Yisaer Yisaer approved these changes

@nolouch nolouch Awaiting requested review from nolouch

Assignees
No one assigned
Labels
component/security Security logic. status/LGT1 Indicates that a PR has LGTM 1.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@yiwu-arbug @Yisaer @HunDunDM @nolouch @ti-srebot