abi.encodePacked for tightly-packed arrays

[mudgen]

Abstract

Currently abi.encodePacked adds padding to arrays. abi.encode can be used for that.

Please change abi.encodePacked so that it returns bytes of arrays without padding. I ran into a situation where abi.encodePacked would have been really useful if it did not pad arrays with zeros.

It would be great if the following was true:

uint8 value1 = 1;
uint8 value2 = 2;
uint8 value3 = 3;
uint8[] memory threeValuesArray = new uint8[](3);
threeValuesArray[0] = value1;
threeValuesArray[1] = value2;
threeValuesArray[2] = value3;

abi.encodePacked(value1, value2, value3) == abi.encodePacked(threeValuesArray)

[chriseth]

The functio abi.encodePacked was implemented to mimik the behaviour of several implicit operations in old Solidity versions. We actually planned to not touch it anymore. Since I somehow see the need, we could introduce abi.encodeTightlyPacked, though.

[mudgen]

@chriseth That would be awesome, thanks!

[tarik0]

I've found a workaround for this but It requires the BytesLib.sol. Since encodePacked does not use any padding you can just append to each other. (There might be a better approach for this and I don't know If it's gas efficient or not.)

pragma solidity ^0.8.0;

import "https://github.com/GNSPS/solidity-bytes-utils/blob/master/contracts/BytesLib.sol";

contract Test {
    using BytesLib for bytes;

    /**
     * Workaround for the `https://github.com/ethereum/solidity/issues/8441`.
     * @notice Requires `BytesLib.sol` at `https://github.com/GNSPS/solidity-bytes-utils/blob/master/contracts/BytesLib.sol`
     */ 
    function encodeTightlyPacked(address[] memory path) internal pure returns(bytes memory encoded) {
        for (uint i = 0; i < path.length; i++) {
            encoded = encoded.concat(
                abi.encodePacked(path[i])
            );
        }
    }

    function test() public view returns (bytes memory t1, bytes memory t2) {
        address[] memory path = new address[](3);
        path[0] = 0x7a250d5630B4cF539739dF2C5dAcb4c659F2488D;
        path[1] = 0xE592427A0AEce92De3Edee1F18E0157C05861564;
        path[2] = 0xaD6D458402F60fD3Bd25163575031ACDce07538D;

        t1 = abi.encodePacked(path[0], path[1], path[2]);
        t2 = encodeTightlyPacked(path);
    }
}

The result;

t1 = 0x7a250d5630b4cf539739df2c5dacb4c659f2488de592427a0aece92de3edee1f18e0157c05861564ad6d458402f60fd3bd25163575031acdce07538d
t2 = 0x7a250d5630b4cf539739df2c5dacb4c659f2488de592427a0aece92de3edee1f18e0157c05861564ad6d458402f60fd3bd25163575031acdce07538d

[axic]

@tarik0 the language supports bytes.concat() since a while, which I believe would allow you to write:

        uint len = arr.length;
        for (uint i = 0; i < len; i++) {
            encoded = bytes.concat(
                encoded,
                abi.encodePacked(path[i])
            );
        }

It is not super optimal as it will waste memory quite a bit.

[dk1a]

My current workaround (very verbose, but generic and decently gas-efficient):
https://github.com/latticexyz/mud/blob/cb731e0937e614bb316e6bc824813799559956c8/packages/store/src/tightcoder/EncodeArray.sol

On a similar note, an abi.decodeTightlyPacked coupled with memory slices would be useful as well to replace this:
https://github.com/latticexyz/mud/blob/cb731e0937e614bb316e6bc824813799559956c8/packages/store/src/tightcoder/DecodeSlice.sol

[github-actions]

This issue has been marked as stale due to inactivity for the last 90 days.
It will be automatically closed in 7 days.

[github-actions]

Hi everyone! This issue has been automatically closed due to inactivity.
If you think this issue is still relevant in the latest Solidity version and you have something to contribute, feel free to reopen.
However, unless the issue is a concrete proposal that can be implemented, we recommend starting a language discussion on the forum instead.