PS-4976: InnoDB undo log encryption #2626
Conversation
| eval INSTALL PLUGIN keyring_file SONAME '$KEYRING_PLUGIN'; | ||
|
|
||
| SET GLOBAL innodb_undo_log_encrypt = ON; | ||
| --sleep 2 |
| --source include/shutdown_mysqld.inc | ||
| # Search for particular string to confirm the encryption metadata is | ||
| # stored. | ||
| --sleep 2 |
There was a problem hiding this comment.
I have the correct answer for this now: log (redo and undo too) encryption starts up a bit after the server is already started / after the setting is changed - if commands are executed too soon, the logs will contain unencrypted data, and in this case, it's possible that it doens't contain the encryption magic flag.
This can be solved by a debug sync point, I'll open a separate commit in a few days for improved undo/redo tests - with a bit more and better verification that the logs are actually encrpyted.
There was a problem hiding this comment.
That's a bad design, and a debug_sync point will not be good enough. It needs some release build visibility, through I_S/PFS table, status var, or similar
| --source include/have_innodb.inc | ||
|
|
||
| --disable_query_log | ||
| call mtr.add_suppression("\\[ERROR\\] InnoDB: Encryption can't find master key, please check the keyring plugin is loaded."); |
There was a problem hiding this comment.
Later: suppressions are already substring matches, easier to read and maintain if dropped those common "[ERROR] InnoDB:" prefixes
| # show that it's writable | ||
| # | ||
| set global innodb_undo_log_encrypt=1; | ||
| --sleep 2 |
| bool | ||
| fsp_is_undo_tablespace(uint32 space_id) | ||
| { | ||
| return srv_is_undo_tablespace(space_id); |
There was a problem hiding this comment.
Later: why such wrapper?
| << " in read-only mode."; | ||
| return; | ||
| } | ||
| ulint undo_spaces[TRX_SYS_N_RSEGS + 1]; |
There was a problem hiding this comment.
Later: space_id_t
| { | ||
| /* Skip system tablespace, since it's also shared | ||
| tablespace. */ | ||
| const ulint space_id = undo_spaces[undo_idx]; |
There was a problem hiding this comment.
Later: space_id_t
| } | ||
| mtr_commit(&mtr); | ||
| } | ||
| //undo::spaces->s_unlock(); |
There was a problem hiding this comment.
Later: no commented out code
| || strlen(server_uuid) == 0) { | ||
| return; | ||
| } | ||
| ulint undo_spaces[TRX_SYS_N_RSEGS + 1]; |
There was a problem hiding this comment.
Later: space_id_t
| const ulint undo_spaces_no = trx_rseg_get_n_undo_tablespaces(undo_spaces); | ||
| for (ulint undo_idx = 0; undo_idx < undo_spaces_no; ++undo_idx) | ||
| { | ||
| const ulint space_id = undo_spaces[undo_idx]; |
There was a problem hiding this comment.
Later: space_id_t
This PR implements Encryption threads for encryption/re-encryption/decryption of innodb tablespaces. It is based on MariaDB implementation ported from Google's patch. With this WL user can: 1) Change default KEYRING encryption key 2) Create a new table with KEYRING 3) Encrypt offline already existing tables with KEYRING (with alter) 4) Encrypt/re-encrypt online already existing tables with KEYRING (with innodb_online_encryption, innodb_online_encryption_rotate_key_age variables and innodb_online_encryption_threads) – including tables already encrypted with Master Key encryption. 5) Re-encrypt online already encrypted tables with newer version of encryption key (with variables innodb_online_encryption variable, innodb_online_encryption_threads, innodb_online_encryption_rotate_key_age). This WL also fixed the following bugs reported to MariaDB: (MDEV-17231) Encryption threads keep re-encrypting/encrypting corrupted pages (MDEV-17235) Data dictonary is not updated with encryption flags (MDEV-17234) In memory crypt_data is updated before page0 is updated (MDEV-17233) Page 0 is updated more than once when encryption completes (MDEV-17230) encryption_key_id from alter is ignored by encryption threads (MDEV-17229) Encryption threads ignore innodb_default_encryption_key_id
This is based on WL#9290 in MySQL 8.0, while also adding a keyring based rotated key encryption. The encryption setting supports two options: * master_key, which works like the redo log encryption in MySQL 8.0 * keyring_key, which encrypts each new redo log tablespace with the latest percona_redo key from the keyring. Related commits in upstream 8.0: * 1a6dd57 * 8facbb0
dutow
force-pushed
the
ps-5.7-undo-log-encryption
branch
from
165991b to
491215c
Compare
This commit implements undo log encryption loosely based on WL#9289 (71e656a) for MySQL 8.0. Undo log encryption requires the use of separate undo tablepsaces with innodb_undo_tablespaces=N.
dutow
force-pushed
the
ps-5.7-undo-log-encryption
branch
from
491215c to
2332d36
Compare
Note: this PR currently contains 3 new commits compared to the master branch, but only the last, about the undo log encryption, is actually part of this pull requests - there are 2 other open pull requests about the other 2.
PS-4976: InnoDB undo log encryption
This commit implements undo log encryption loosely based on WL#9289 (71e656a) for MySQL 8.0.
Undo log encryption requires the use of separate undo tablespaces with innodb_undo_tablespaces=N.