Replace heavy crypto packages for lighter noble implementations via upgrading ethereumjs-util to latest (now called @ethereumjs/util)

[adonesky1]

Removes secp256k1 and keccak via as dependencies by upgrading from [email protected] to its latest (beta) version @ethereumjs/[email protected] which uses noble implementations for crypto. Replaces keccak hashing with keccak256 from ethereum-cryptography/keccak

To achieve this I've copied functions from ethereumjs-abi (rawEncode and solidityPack) that we use to pack the values before hashing them with the newly added keccak256 implementation from ethereum-cryptography/keccak, and removed the ethereumjs-abi dependency. I've extracted these functions and more or less preserved their implementations (solidityPack, rawEncode) minus the references to ethereumjs/[email protected] methods, which, if left intact would keep secp256k1 and keccak in our dependency tree.

Next steps:

• Swap out bn.js for BigInt (per this comment: Replace heavy crypto packages for lighter noble implementations via upgrading ethereumjs-util to latest (now called @ethereumjs/util) #260 (comment))

[kumavis]

as possible try to remove toBuffer calls where we know the input type

[adonesky1]

@kumavis this currently appears to be increasing the size of the package... 🤦

[adonesky1]

@kumavis this currently appears to be increasing the size of the package... 🤦

This is just true of the dist/ folder, not the overall impact on the consumer and its dependency tree, so yeah its gonna be bigger with all those added utility functions

I'm trying to evaluate the impact upstream using yarn why looking at Disk size with transitive dependencies and Disk size with unique dependencies but not sure how valuable these are of indicators.

[wbt]

Updating ethereumjs-util is also useful for the reasons noted here around getting consistent types from bn.js after a recent breaking-change update adopted elsewhere.

[paulmillr]

Why are you keeping bn.js if all the internal methods of new @ethereumjs libraries switched to native bigints? For keccak256, you can use ethereum-cryptography/keccak256 since you're already using it.

[adonesky1]

Why are you keeping bn.js if all the internal methods of new @ethereumjs libraries switched to native bigints

I'm using bn.js here to replicate the functions from ethereumjs-abi (rawEncode and solidityPack) that we use to pack the values before hashing them with the keccak256 implementation from ethereum-cryptography/keccak. I've extracted these functions and more or less preserved their implementations (solidityPack, rawEncode) - including use of bn.js - minus the references to old ethereumjs/[email protected] methods, which would keep secp256k1 and keccak in our dependency tree. cc @kumavis

[paulmillr]

@adonesky1 understood, so it's basically copy-pasting code from other lib. I've noticed about 5 lines that use BN. Think it would be really easy to replace it with native BigInt. In the current case you're adding 3547 lines of unnecessary code.

[adonesky1]

I've noticed about 5 lines that use BN. Think it would be really easy to replace it with native BigInt. In the current case you're adding 3547 lines of unnecessary code.

Yeah looking into this now. Will try to make it work in a subsequent PR.

addressed feedback