Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output the storage layout of a contract via storageLayout artifact #7589

Merged
merged 1 commit into from
Nov 14, 2019
Merged

Output the storage layout of a contract via storageLayout artifact #7589

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Changelog.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Compiler Features:
* SMTChecker: Add break/continue support to the CHC engine.
* SMTChecker: Support assignments to multi-dimensional arrays and mappings.
* SMTChecker: Support inheritance and function overriding.
* Standard JSON Interface: Output the storage layout of a contract when artifact ``storageLayout`` is requested.
* TypeChecker: List possible candidates when overload resolution fails.


Expand Down
265 changes: 265 additions & 0 deletions docs/miscellaneous.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ Miscellaneous
Layout of State Variables in Storage
************************************

.. _storage-inplace-encoding:
chriseth marked this conversation as resolved.
Show resolved Hide resolved

Statically-sized variables (everything except mapping and dynamically-sized array types) are laid out contiguously in storage starting from position ``0``. Multiple, contiguous items that need less than 32 bytes are packed into a single storage slot if possible, according to the following rules:

- The first item in a storage slot is stored lower-order aligned.
Expand Down Expand Up @@ -49,6 +51,8 @@ The elements of structs and arrays are stored after each other, just as if they
Mappings and Dynamic Arrays
===========================

.. _storage-hashed-encoding:

Due to their unpredictable size, mapping and dynamically-sized array types use a Keccak-256 hash
computation to find the starting position of the value or the array data. These starting positions are always full stack slots.

Expand Down Expand Up @@ -88,6 +92,267 @@ by checking if the lowest bit is set: short (not set) and long (set).
.. note::
Handling invalidly encoded slots is currently not supported but may be added in the future.

JSON Output
===========

.. _storage-layout-top-level:

The storage layout of a contract can be requested via the :ref:`standard JSON interface <compiler-api>`. The output is a JSON object containing two keys,
``storage`` and ``types``. The ``storage`` object is an array where each
element has the following form:


.. code::


{
"astId": 2,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed these to be numbers again and added some spaces.

"contract": "fileA:A",
"label": "x",
"offset": 0,
"slot": "0",
"type": "t_uint256"
}

where the example above is the storage layout of ``contract A { uint x; }`` from source unit ``fileA``
and

- ``astId`` is the id of the AST node of the state variable's declaration
- ``contract`` is the name of the contract including its path as prefix
- ``label`` is the name of the state variable
- ``offset`` is the offset in bytes within the storage slot according to the encoding
- ``slot`` is the storage slot where the state variable resides or starts. This
number may be very large and therefore its JSON value is represented as a
string.
- ``type`` is an identifier used as key to the variable's type information (described in the following)

The given ``type``, in this case ``t_uint256`` represents an element in
``types``, which has the form:


.. code::

{
"encoding": "inplace",
"label": "uint256",
"numberOfBytes": "32",
}

where

- ``encoding`` how the data is encoded in storage, where the possible values are:

- ``inplace``: data is laid out contiguously in storage (see :ref:`above <storage-inplace-encoding>`).
- ``mapping``: Keccak-256 hash-based method (see :ref:`above <storage-hashed-encoding>`).
- ``dynamic_array``: Keccak-256 hash-based method (see :ref:`above <storage-hashed-encoding>`).
- ``bytes``: single slot or Keccak-256 hash-based depending on the data size (see :ref:`above <bytes-and-string>`).

- ``label`` is the canonical type name.
- ``numberOfBytes`` is the number of used bytes (as a decimal string). Note that if ``numberOfBytes > 32`` this means that more than one slot is used.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added string clarification.


Some types have extra information besides the four above. Mappings contain
ekpyron marked this conversation as resolved.
Show resolved Hide resolved
its ``key`` and ``value`` types (again referencing an entry in this mapping
of types), arrays have its ``base`` type, and structs list their ``members`` in
the same format as the top-level ``storage`` (see :ref:`above
<storage-layout-top-level>`).

.. note ::
The JSON output format of a contract's storage layout is still considered experimental
and is subject to change in non-breaking releases of Solidity.

The following example shows a contract and its storage layout, containing
value and reference types, types that are encoded packed, and nested types.


.. code::

pragma solidity >=0.4.0 <0.7.0;
contract A {
struct S {
uint128 a;
uint128 b;
uint[2] staticArray;
uint[] dynArray;
}

uint x;
uint y;
S s;
address addr;
mapping (uint => mapping (address => bool)) map;
uint[] array;
string s1;
bytes b1;
}

.. code::

"storageLayout": {
"storage": [
{
"astId": 14,
"contract": "fileA:A",
"label": "x",
"offset": 0,
"slot": "0",
"type": "t_uint256"
},
{
"astId": 16,
"contract": "fileA:A",
"label": "y",
"offset": 0,
"slot": "1",
"type": "t_uint256"
},
{
"astId": 18,
"contract": "fileA:A",
"label": "s",
"offset": 0,
"slot": "2",
"type": "t_struct(S)12_storage"
},
{
"astId": 20,
"contract": "fileA:A",
"label": "addr",
"offset": 0,
"slot": "6",
"type": "t_address"
},
{
"astId": 26,
"contract": "fileA:A",
"label": "map",
"offset": 0,
"slot": "7",
"type": "t_mapping(t_uint256,t_mapping(t_address,t_bool))"
},
{
"astId": 29,
"contract": "fileA:A",
"label": "array",
"offset": 0,
"slot": "8",
"type": "t_array(t_uint256)dyn_storage"
},
{
"astId": 31,
"contract": "fileA:A",
"label": "s1",
"offset": 0,
"slot": "9",
"type": "t_string_storage"
},
{
"astId": 33,
"contract": "fileA:A",
"label": "b1",
"offset": 0,
"slot": "10",
"type": "t_bytes_storage"
}
],
"types": {
"t_address": {
"encoding": "inplace",
"label": "address",
"numberOfBytes": "20"
},
"t_array(t_uint256)2_storage": {
"base": "t_uint256",
"encoding": "inplace",
"label": "uint256[2]",
"numberOfBytes": "64"
},
"t_array(t_uint256)dyn_storage": {
"base": "t_uint256",
"encoding": "dynamic_array",
"label": "uint256[]",
"numberOfBytes": "32"
},
"t_bool": {
"encoding": "inplace",
"label": "bool",
"numberOfBytes": "1"
},
"t_bytes_storage": {
"encoding": "bytes",
"label": "bytes",
"numberOfBytes": "32"
},
"t_mapping(t_address,t_bool)": {
"encoding": "mapping",
"key": "t_address",
"label": "mapping(address => bool)",
"numberOfBytes": "32",
"value": "t_bool"
},
"t_mapping(t_uint256,t_mapping(t_address,t_bool))": {
"encoding": "mapping",
"key": "t_uint256",
"label": "mapping(uint256 => mapping(address => bool))",
"numberOfBytes": "32",
"value": "t_mapping(t_address,t_bool)"
},
"t_string_storage": {
"encoding": "bytes",
"label": "string",
"numberOfBytes": "32"
},
"t_struct(S)12_storage": {
"encoding": "inplace",
"label": "struct A.S",
"members": [
{
"astId": 2,
"contract": "fileA:A",
"label": "a",
"offset": 0,
"slot": "0",
"type": "t_uint128"
},
{
"astId": 4,
"contract": "fileA:A",
"label": "b",
"offset": 16,
"slot": "0",
"type": "t_uint128"
},
{
"astId": 8,
"contract": "fileA:A",
"label": "staticArray",
"offset": 0,
"slot": "1",
"type": "t_array(t_uint256)2_storage"
},
{
"astId": 11,
"contract": "fileA:A",
"label": "dynArray",
"offset": 0,
"slot": "3",
"type": "t_array(t_uint256)dyn_storage"
}
],
"numberOfBytes": "128"
},
"t_uint128": {
"encoding": "inplace",
"label": "uint128",
"numberOfBytes": "16"
},
"t_uint256": {
"encoding": "inplace",
"label": "uint256",
"numberOfBytes": "32"
}
}
}

.. index: memory layout

****************
Expand Down
3 changes: 3 additions & 0 deletions docs/using-the-compiler.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,7 @@ Input Description
// metadata - Metadata
// ir - Yul intermediate representation of the code before optimization
// irOptimized - Intermediate representation after optimization
// storageLayout - Slots, offsets and types of the contract's state variables.
// evm.assembly - New assembly format
// evm.legacyAssembly - Old-style assembly format in JSON
// evm.bytecode.object - Bytecode object
Expand Down Expand Up @@ -376,6 +377,8 @@ Output Description
"devdoc": {},
// Intermediate representation (string)
"ir": "",
// See the Storage Layout documentation.
"storageLayout": {"storage": [...], "types": {...} },
// EVM-related outputs
"evm": {
// Assembly (string)
Expand Down
2 changes: 2 additions & 0 deletions libsolidity/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ set(sources
interface/ReadFile.h
interface/StandardCompiler.cpp
interface/StandardCompiler.h
interface/StorageLayout.cpp
interface/StorageLayout.h
interface/Version.cpp
interface/Version.h
parsing/DocStringParser.cpp
Expand Down
23 changes: 23 additions & 0 deletions libsolidity/interface/CompilerStack.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
#include <libsolidity/interface/ABI.h>
#include <libsolidity/interface/Natspec.h>
#include <libsolidity/interface/GasEstimator.h>
#include <libsolidity/interface/StorageLayout.h>
#include <libsolidity/interface/Version.h>
#include <libsolidity/parsing/Parser.h>

Expand Down Expand Up @@ -667,6 +668,28 @@ Json::Value const& CompilerStack::contractABI(Contract const& _contract) const
return *_contract.abi;
}

Json::Value const& CompilerStack::storageLayout(string const& _contractName) const
{
if (m_stackState < AnalysisPerformed)
BOOST_THROW_EXCEPTION(CompilerError() << errinfo_comment("Analysis was not successful."));

return storageLayout(contract(_contractName));
}

Json::Value const& CompilerStack::storageLayout(Contract const& _contract) const
{
if (m_stackState < AnalysisPerformed)
BOOST_THROW_EXCEPTION(CompilerError() << errinfo_comment("Analysis was not successful."));

solAssert(_contract.contract, "");

// caches the result
if (!_contract.storageLayout)
_contract.storageLayout.reset(new Json::Value(StorageLayout().generate(*_contract.contract)));

return *_contract.storageLayout;
}

Json::Value const& CompilerStack::natspecUser(string const& _contractName) const
{
if (m_stackState < AnalysisPerformed)
Expand Down
9 changes: 9 additions & 0 deletions libsolidity/interface/CompilerStack.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,10 @@ class CompilerStack: boost::noncopyable
/// Prerequisite: Successful call to parse or compile.
Json::Value const& contractABI(std::string const& _contractName) const;

/// @returns a JSON representing the storage layout of the contract.
/// Prerequisite: Successful call to parse or compile.
Json::Value const& storageLayout(std::string const& _contractName) const;

/// @returns a JSON representing the contract's user documentation.
/// Prerequisite: Successful call to parse or compile.
Json::Value const& natspecUser(std::string const& _contractName) const;
Expand Down Expand Up @@ -319,6 +323,7 @@ class CompilerStack: boost::noncopyable
eth::LinkerObject eWasmObject; ///< Experimental eWasm code
mutable std::unique_ptr<std::string const> metadata; ///< The metadata json that will be hashed into the chain.
mutable std::unique_ptr<Json::Value const> abi;
mutable std::unique_ptr<Json::Value const> storageLayout;
mutable std::unique_ptr<Json::Value const> userDocumentation;
mutable std::unique_ptr<Json::Value const> devDocumentation;
mutable std::unique_ptr<std::string const> sourceMapping;
Expand Down Expand Up @@ -382,6 +387,10 @@ class CompilerStack: boost::noncopyable
/// This will generate the JSON object and store it in the Contract object if it is not present yet.
Json::Value const& contractABI(Contract const&) const;

/// @returns the storage layout of the contract as a JSON object.
/// This will generate the JSON object and store it in the Contract object if it is not present yet.
Json::Value const& storageLayout(Contract const&) const;

/// @returns the Natspec User documentation as a JSON object.
/// This will generate the JSON object and store it in the Contract object if it is not present yet.
Json::Value const& natspecUser(Contract const&) const;
Expand Down
Loading