Add Storage Keys to ABI

[skilesare]

I've run across a couple of times where I've wanted to monkey around with the internals of a contract or intercept things as they are running in a VM. One issue I keep running into is that there is not an easy way to know which key a solidity storage var is stored at. This would be a fantastic thing to have in the ABI or in some other output of the compiler. I would assume there is already some function that does this analysis and assigns storage locations and indexes based on the vars that are declared.

I know mappings are a bit strange as you have to keccak the position plus the mapping key(or something like that), but you still need the piece that identifies the storage location of the mapping.

[VoR0220]

@skilesare there is a "dump state" method for each contract if you look on the client and can usually parse down for that. This gets you the current state of an address on a client. You can parse it down from there, but remember that this is fairly out in the open.

[skilesare]

Good to know. I'm not so much interested in the current state as sometimes I'm trying to build the state manually using a state manager so I need to know what key to use.

For example: I'm sideloading a contract into ethereurmjs-vm and I've put the contract code using putContractCode and now I need to set the owner in storage so I call putContractStorage(address, key, value). How do I know what key is? Well if I wrote the contract and owner was the first var I declared it goes in 0x0. What if I have a mapping(address=>int). I know there is some crazy math, but it would be nice to have this in the ABI. I also know that short uints can be compacted into one slot. It would be great to know what slot and index each var is at.

[VoR0220]

I know what you're talking about but a simpler solution is to put a function like a public view getter for the index of your mapping. Plus it's free. I suppose that the ABI could support a length pointer...not sure why that shouldn't be doable (outside of the cheapness factor...which tbf is a big factor). Length and from there you just add the index with each transaction separately....Either way. Take a look at how Remix does this. They currently already have a working implementation of this.

[chriseth]

@skilesare remix can do that for you. The problem is that this works for simple value-type variables, but you need more logic for mappings, arrays and structs. If you implement that logic, then you can already do everything on your own without any help from the compiler.

[skilesare]

@chriseth

Re: remix- can you point me to where remix does this? I don’t see any mappings on the detail screen.

Re: logic - can we use this issue to document that logic? It seems to exist elsewhere but it’s pretty spread out. I may have access to a CPP developer in the near future and may be able to add this in.

My understanding is that the first storage key is 0x0 and corosponds to the first variable.

If that var is less than 32 bytes the second var may get put in 0x0 as well if it plus the first are less than 32 bytes.

Mapping are the storage slot hashed with the key, so if the first var is unit and the second is mapping address to unit the values would be put at keccak(0x0...1,adres)

I’m not sure what happens it’s bytes or arrays

[chriseth]

@skilesare you have to use the debugger on a transaction and then the variables are shown in the "Solidity state" section.

The layout should be documented in https://solidity.readthedocs.io/en/v0.4.21/miscellaneous.html#layout-of-state-variables-in-storage - please feel free to improve that.

[livnev]

@leonardoalt Would it be possible to revive this heroïc effort? @MrChico and I and many others who are using the solidity metadata for formal analysis, currently mapping out their contract storage by hand, would be forever grateful...

[leonardoalt]

@livnev I guess this would need some discussion to decide what exactly it would mean.

1. This would be quite complicated for non-value types. When I talked to @MrChico about that, he mentioned that having that even for value types only would be good already.
2. I'm not sure having it as part of the ABI is a good idea. Maybe a command line option?

@chriseth @axic any thoughts?

[MrChico]

It doesn't need to be in the ABI. Just having a way to extract it somehow from the compiler metadata (like an additional entry in the combined json) would work. And when it comes to non-value types, I understand that they may occupy more than one storage slot, and that there are different encoding schemes. But this information would also be valuable to obtain. I'm imagining something like this

{
  "storageLocations": [
    "totalSupply": {
      "slot": 0,
      "type": "uint256",
      "encoding": "direct"
    },
    "balanceOf": {
      "slot": 1,
      "type": "mapping(address => uint)",
      "encoding": "hashList",
      "dimension": 1
    },
    "TokenName": {
      "slot": 2,
      "type": "string",
      "encoding": "packedBytes"
    },
    "whitelist": {
      "slot": 3,
      "type": "address[]",
      "encoding": "dynamicArray"
    },
    "startDate": {
      "slot": 4,
      "type": "uint32",
      "encoding": "packedValue",
      "offSet": 28
    }
    "stopped": {
      "slot": 4,
      "type": "bool",
      "encoding": "packedValue",
      "offSet": 27
    }
  ]
}

[livnev]

where the above would be the result for a contract with storage variables declared something like this:

  uint256                      public totalSupply;
  mapping (address => uint256) public balanceOf;
  string                       public tokenName;
  address[]                    public whitlist;
  uint32                       public startDate;
  bool                         public stopped;

[axic]

#4017 had a lot of discussion on this topic with an agreed JSON format.

I think this was the final one: #4017 (comment)