Geth keys with ciphertext of 31 bytes (62 hex digits) length

[NikVolf]

Originally reported on slack

Not only they fail to load individually, such keys also prevent loading of any key in the same directory (listAccount fails)

It's also unclear how the geth produces and reads this kind of keys

User with large amount of keys reports that around 0.37% of all keys have 62 hex digits (252 bits) in the ciphertext section of the key file

[Gustav-Simonsson]

It's due to the code here: https://github.com/ethereum/go-ethereum/blob/master/accounts/key_store_passphrase.go#L110

The privkey-bytes-to-be-encrypted are retrieved using https://github.com/ethereum/go-ethereum/blob/master/crypto/crypto.go#L98 which calls golang stdlib https://golang.org/pkg/math/big/#Int.Bytes

It returns the minimal byte array needed to represent the integer, which for private keys can be less than 32 bytes.

While private keys are often defined to be 32 bytes, ECDSA specifies them as integers in range [1, secp256k1n − 1] (also YP eq 205 & 208).

Since private keys are randomly generated, sometimes their integer value can be represented as a big-endian byte array of length shorter than 32 bytes - and this is what happens in this case.

Logging the length of keyBytes (see code link above) in geth and running one million key generations confirms the expected statistics:

// this runs 2 key generations per count
godep go test -run TestKeyStorePassphrase -count 500000 -timeout 30m &>debug

grep keyBytes debug | wc -l
1000000

grep keyBytes debug | grep 32 | wc -l
996164

grep keyBytes debug | grep 31 | wc -l
3818

grep keyBytes debug | grep 30 | wc -l
18

So we have 3818 of 1M (~2^8) key generations returning privkey with length 31 and
18 of 1M (~2^16) with 30 bytes.

Arguably Web3 Secret Storage Definition should define padding if the private key to-be-encrypted can be represented as less than 32 bytes. Since Ethereum uses big-endian would suggest we define higher-order (left) side zero byte padding to always get 32 bytes for encryption.

geth PR: ethereum/go-ethereum#3032

[knocte]

We're also affected by this, thanks for the investigation. In our case, the error is: Invalid hash at line 1 column 164 when importing. I figured I should post it here so that it's easily googlable.

[rphmeier]

Thanks for the breakdown @Gustav-Simonsson -- this should be a pretty simple fix.