crypto: use address32 flag #8542

joyqvq · 2023-02-22T19:41:31Z

EDIT: as per #9262 the hash function is changed from SHA3-256 to BLAKE2b!

Description

SuiAddress and ObjectIDs are now 32 bytes along instead of 20 bytes (in hex, the len increases from 40 to 64)
Match Secp256k1.deriveKeypair SDK API with Ed25519: takes in mnemonics and an optional path string.
Specifics:
in move: use address::length() as the source of truth for 32
in rust: use ObjectID::LENGTH as the source of truth for 32
in ts: a hardcode const SUI_ADDRESS_LENGTH
shifted object ID changed a few test assumptions, hence changes in move integration tests, randomness test, crates/sui-adapter-transactional-tests fixes thanks to Todd N 🙏

How to retrieve new address

We cannot derive 32-byte address from the legacy 20-byte address. Instead, you can derive directly from the flag and the private key using the following command. Alternatively, you can construct from the flag and public key (address = last 32 bytes of sha3_256(flag || pubkey)).
0. Build/download latest sui (0.28.0 or newer)

cargo build --bin sui

In CLI, Use mnemonics

target/debug/sui keytool import "$MNEMONICS" ed25519                                                                                                                                                                               
2023-02-28T15:32:10.675501Z  INFO sui::keytool: Key imported for address [$NEW_32_BYTE_ADDRESS]

In CLI, Use flag || privkey base64 string

target/debug/sui keytool unpack AJrA997C1eVz6wYIp7bO8dpITSRBXpvg1m70/P3gusu2                                                                                                                                                                                                                                
Address, keypair and key scheme written to $NEW_32_BYTE_ADDRESS.key

In Typescript, use latest toSuiAddress()
In Rust, use latest pk.into()

Test vectors

We provide three test cases for each signature scheme in three arrays. In each array, the first element is the mnemonics, the second element is a base64 encoded flag || privkey (same as what you see in sui.keystore), the third element is the derived 32-bytes address in hex.

For Ed25519:

sui/sdk/typescript/test/unit/cryptography/ed25519-keypair.test.ts

Line 12 in 4adfbff

const TEST_CASES = [

For Secp256k1

sui/sdk/typescript/test/unit/cryptography/secp256k1-keypair.test.ts

Line 37 in 4adfbff

const TEST_CASES = [

Test Plan

See all unit tests. e2e tests see:

If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process.

Type of Change (Check all that apply)

user-visible impact

Any 20 byte old address is invalidated.

breaking change for a client SDKs

When 0.29.0 is released for sui, the user MUST upgrade the sui wallet and SDK to the latest version to interact with the network. This change is backward incompatible. User needs to reimport their mnemonics or private key to see an valid 32-byte address.

breaking change for FNs (FN binary must upgrade)

All move packages and FN can only understand a 32-byte address and fail all 20-byte addresses.

breaking change for validators or node operators (must upgrade binaries)
breaking change for on-chain data layout

All system objects and IDs are using the new 32-byte representation.

necessitate either a data wipe or data migration

All old object IDs and Sui Addresses are invalidated.

vercel · 2023-02-22T19:41:38Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated
explorer	✅ Ready (Inspect)	Visit Preview	💬 Add your feedback	Feb 28, 2023 at 8:00PM (UTC)
explorer-storybook	✅ Ready (Inspect)	Visit Preview	💬 Add your feedback	Feb 28, 2023 at 8:00PM (UTC)
wallet-adapter	✅ Ready (Inspect)	Visit Preview	💬 Add your feedback	Feb 28, 2023 at 8:00PM (UTC)

1 Ignored Deployment

Name	Status	Preview	Comments	Updated
frenemies	⬜️ Ignored (Inspect)			Feb 28, 2023 at 8:00PM (UTC)

sblackshear · 2023-02-22T23:58:01Z

Flagging this as one other spot that will need to change: https://github.com/MystenLabs/sui/blob/main/crates/sui-framework/sources/address.move#L11

joyqvq · 2023-02-24T14:21:23Z

crates/sui-framework/sources/address.move

@@ -8,17 +8,14 @@ module sui::address {
    use std::string;

    /// The length of an address, in bytes
-    const LENGTH: u64 = 20;


cc @sblackshear to review address.move and address_test.move

joyqvq · 2023-02-24T16:16:13Z

sdk/typescript/src/types/common.ts

@@ -76,7 +76,7 @@ export function isValidTransactionDigest(
 // which uses the Move account address length
 // https://github.com/move-language/move/blob/67ec40dc50c66c34fd73512fcc412f3b68d67235/language/move-core/types/src/account_address.rs#L23 .

-export const SUI_ADDRESS_LENGTH = 20;
+export const SUI_ADDRESS_LENGTH = 32;


cc @Jordan-Mysten i tried to make all the typescript changes too (at least all unit tests are passing, but not any e2e), feel free to push any more

We can follow-up with this at a later point, but would probably be ideal for BCS + SDK to use the same constant so this can easily be configured in a single place.

crates/sui-adapter-transactional-tests/tests/size_limits/move_object_size_limit.move

crates/sui-sdk/examples/transfer_coins.rs

kchalkias · 2023-02-24T17:03:18Z

sdk/typescript/src/cryptography/secp256k1-publickey.ts

@@ -70,6 +71,6 @@ export class Secp256k1PublicKey implements PublicKey {
    let tmp = new Uint8Array(SECP256K1_PUBLIC_KEY_SIZE + 1);
    tmp.set([SIGNATURE_SCHEME_TO_FLAG['Secp256k1']]);
    tmp.set(this.toBytes(), 1);
-    return sha3.sha3_256(tmp).slice(0, 40);
+    return sha3.sha3_256(tmp).slice(0, SUI_ADDRESS_LENGTH * 2);


I'd a comment here that this double size is because of hex being 100% longer (each hex char represents half a byte of information)

joyqvq · 2023-02-24T17:22:07Z

crates/sui-types/src/unit_tests/base_types_tests.rs

@@ -353,12 +362,16 @@ fn test_move_package_size_for_gas_metering() {
    let serialized = bcs::to_bytes(&package).unwrap();
    // If the following assertion breaks, it's likely you have changed MovePackage's fields.
    // Make sure to adjust `object_size_for_gas_metering()` to include those changes.
-    assert_eq!(size + 2, serialized.len());
+    assert_eq!(size - 2, serialized.len());


is this the correct thing to do to change the test? from what i see, the object_size_for_gas_metering seems to be an approx to begin with, and no need to tweak there.

let meta_data_size = size_of::<Owner>() + size_of::<TransactionDigest>() + size_of::<u64>();

this should take into account the size of owner aka the increased address size

Jordan-Mysten

TS Changes LGTM

Jordan-Mysten · 2023-02-28T02:08:21Z

sdk/typescript/src/types/common.ts

@@ -76,7 +76,7 @@ export function isValidTransactionDigest(
 // which uses the Move account address length
 // https://github.com/move-language/move/blob/67ec40dc50c66c34fd73512fcc412f3b68d67235/language/move-core/types/src/account_address.rs#L23 .

-export const SUI_ADDRESS_LENGTH = 20;
+export const SUI_ADDRESS_LENGTH = 32;


We can follow-up with this at a later point, but would probably be ideal for BCS + SDK to use the same constant so this can easily be configured in a single place.

kchalkias

Excellent job, thanks both to @joyqvq and all reviewers.

kchalkias · 2023-02-28T06:12:27Z

crates/sui-core/src/unit_tests/move_integration_tests.rs

-    assert_eq!(effects.events.len(), 6);
+    assert_eq!(effects.events.len(), 7);
+    // TODO: figure out why an extra event is emitted.
+    // assert_eq!(effects.events.len(), 6);


will create an issue to track the effects issues, i guess @patrickkuo's work might be relevant

joyqvq force-pushed the bench-address32 branch 6 times, most recently from f7ee3c5 to 66dddea Compare February 23, 2023 22:09

vercel bot had a problem deploying to Preview – explorer February 23, 2023 22:24 Failure

vercel bot temporarily deployed to Preview – wallet-adapter February 23, 2023 22:24 Inactive

vercel bot temporarily deployed to Preview – explorer-storybook February 23, 2023 22:25 Inactive

joyqvq force-pushed the bench-address32 branch from 1456a8d to c7a9a86 Compare February 23, 2023 22:36

vercel bot deployed to Preview – explorer February 23, 2023 22:36 View deployment

vercel bot temporarily deployed to Preview – explorer-storybook February 23, 2023 22:36 Inactive

vercel bot temporarily deployed to Preview – wallet-adapter February 23, 2023 22:36 Inactive

joyqvq force-pushed the bench-address32 branch from 70e6992 to 9f82b72 Compare February 24, 2023 04:36

vercel bot temporarily deployed to Preview – wallet-adapter February 24, 2023 04:37 Inactive

vercel bot temporarily deployed to Preview – explorer-storybook February 24, 2023 04:37 Inactive

vercel bot deployed to Preview – explorer February 24, 2023 04:37 View deployment

joyqvq requested review from sblackshear and kchalkias February 24, 2023 14:20

joyqvq commented Feb 24, 2023

View reviewed changes

joyqvq requested a review from Jordan-Mysten February 24, 2023 14:22

joyqvq force-pushed the bench-address32 branch from 9f82b72 to abd204e Compare February 24, 2023 16:15

vercel bot temporarily deployed to Preview – wallet-adapter February 24, 2023 16:15 Inactive

vercel bot temporarily deployed to Preview – explorer-storybook February 24, 2023 16:16 Inactive

joyqvq commented Feb 24, 2023

View reviewed changes

vercel bot deployed to Preview – explorer February 24, 2023 16:16 View deployment

kchalkias reviewed Feb 24, 2023

View reviewed changes

joyqvq commented Feb 24, 2023

View reviewed changes

vercel bot deployed to Preview – explorer February 28, 2023 00:46 View deployment

vercel bot temporarily deployed to Preview – explorer-storybook February 28, 2023 00:46 Inactive

vercel bot temporarily deployed to Preview – wallet-adapter February 28, 2023 00:46 Inactive

joyqvq force-pushed the bench-address32 branch from 7be39ba to d658bd9 Compare February 28, 2023 00:56

vercel bot temporarily deployed to Preview – wallet-adapter February 28, 2023 00:56 Inactive

vercel bot deployed to Preview – explorer February 28, 2023 00:57 View deployment

vercel bot temporarily deployed to Preview – explorer-storybook February 28, 2023 00:57 Inactive

Jordan-Mysten approved these changes Feb 28, 2023

View reviewed changes

joyqvq force-pushed the bench-address32 branch from d658bd9 to 6f6f6f3 Compare February 28, 2023 02:26

vercel bot temporarily deployed to Preview – explorer-storybook February 28, 2023 02:27 Inactive

vercel bot temporarily deployed to Preview – wallet-adapter February 28, 2023 02:27 Inactive

vercel bot deployed to Preview – explorer February 28, 2023 02:28 View deployment

kchalkias approved these changes Feb 28, 2023

View reviewed changes

kchalkias mentioned this pull request Feb 28, 2023

Extra event emitted after switching to 32B addresses #8707

Open

joyqvq added the breaking change label Feb 28, 2023

joyqvq force-pushed the bench-address32 branch from 6f6f6f3 to abf8cc5 Compare February 28, 2023 18:39

vercel bot temporarily deployed to Preview – explorer-storybook February 28, 2023 18:40 Inactive

vercel bot deployed to Preview – explorer February 28, 2023 18:40 View deployment

vercel bot temporarily deployed to Preview – wallet-adapter February 28, 2023 18:40 Inactive

joyqvq enabled auto-merge (squash) February 28, 2023 19:00

crypto: Switch from 20 to 32 byte address

5d3f521

joyqvq force-pushed the bench-address32 branch from abf8cc5 to 5d3f521 Compare February 28, 2023 19:59

vercel bot temporarily deployed to Preview – wallet-adapter February 28, 2023 20:00 Inactive

vercel bot deployed to Preview – explorer February 28, 2023 20:00 View deployment

vercel bot temporarily deployed to Preview – explorer-storybook February 28, 2023 20:00 Inactive

joyqvq merged commit a0955c4 into main Feb 28, 2023

joyqvq deleted the bench-address32 branch February 28, 2023 20:28

This was referenced Mar 3, 2023

Investigate extra gas effect event #8805

Closed

Official address encoding base58check Vs bech32 vs base64 MystenLabs/fastcrypto#1

Closed

GrapeBaBa mentioned this pull request Mar 5, 2023

Tracking sui 0.28 breaking changes GrapeBaBa/sui4j#94

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

crypto: use address32 flag #8542

crypto: use address32 flag #8542

joyqvq commented Feb 22, 2023 •

edited

Loading

vercel bot commented Feb 22, 2023 •

edited

Loading

sblackshear commented Feb 22, 2023

joyqvq Feb 24, 2023

joyqvq Feb 24, 2023 •

edited

Loading

Jordan-Mysten Feb 28, 2023

kchalkias Feb 24, 2023

joyqvq Feb 24, 2023

Jordan-Mysten left a comment

Jordan-Mysten Feb 28, 2023

kchalkias left a comment

kchalkias Feb 28, 2023

crypto: use address32 flag #8542

crypto: use address32 flag #8542

Conversation

joyqvq commented Feb 22, 2023 • edited Loading

Description

How to retrieve new address

Test vectors

Test Plan

Type of Change (Check all that apply)

vercel bot commented Feb 22, 2023 • edited Loading

sblackshear commented Feb 22, 2023

joyqvq Feb 24, 2023

Choose a reason for hiding this comment

joyqvq Feb 24, 2023 • edited Loading

Choose a reason for hiding this comment

Jordan-Mysten Feb 28, 2023

Choose a reason for hiding this comment

kchalkias Feb 24, 2023

Choose a reason for hiding this comment

joyqvq Feb 24, 2023

Choose a reason for hiding this comment

Jordan-Mysten left a comment

Choose a reason for hiding this comment

Jordan-Mysten Feb 28, 2023

Choose a reason for hiding this comment

kchalkias left a comment

Choose a reason for hiding this comment

kchalkias Feb 28, 2023

Choose a reason for hiding this comment

joyqvq commented Feb 22, 2023 •

edited

Loading

vercel bot commented Feb 22, 2023 •

edited

Loading

joyqvq Feb 24, 2023 •

edited

Loading