Allow Modules to Store and Serve Data outside of the JMT

[preston-evans98]

Background

Currently, modules have access to #[state] variables. Under the hood, these variables are stored in the authenticated data store backed by the JMT.

Recently, the EVM and Optimistic rollup workstreams have both encountered the need to serve data via RPC which does not need to be authenticated. (For the EVM, this is receipt and log data; for ORUs, this is storage proofs). While we always can store extra data in state, it's significantly preferable to avoid the overhead in these cases. Instead, we should add a new field type to modules which is write-only within the state machine, but can be read in RPC calls.

Desired Outcome

Modules should be able to efficiently store and retrieve arbitrary auxiliary data using a non-provable data store. The store should be accessible for reading in RPC handlers but must not be readable during execution.

Potential Designs

Off the top of my head, I can see at least three possible designs:

Option 1: Add a new API for registering module-defined DBs

In this design, we create a new trait which allows modules to bring their own (non-provable) database and have it opened/initialized during node startup. The motivating example for this design is to allow the EVM module to use Reth's existing LMDB/MDBX database rather than attempting to port their storage model to RocksDB.

A very rough sketch of the design is like this:

// sov-modules-api/lib.rs
pub trait UnprovableStore: Clone { 
  type Config: DeserializeOwned;
  fn open(Config) -> Self;
  fn put<S: Schema>(&self, S::Key, S::Value) -> Result<()>;
  #[cfg(feature = "native")]
  fn get<S: Schema>(&self, S::Key) -> Result<Option<S::Value>>;
} 

#[derive(Default)]
pub struct DbRegistry { 
  dbs: HashMap<Address, Box<dyn UnprovableStore>>
}

//my-modules/lib.rs
/// A new initailization function to create the module store. This is invoked
/// during node startup
fn init(config: Config) -> Option<UnprovableStore>;

//runtime.rs
/// A new (macro-generated?) function to initalize all module dbs.
fn init(config: Config) ->  {
  let db_registry = Default::default();
  if let Some(db) = MyModule::init(config.my_module) { 
    registry.add(MyModules::new().address(), db)
  } 
}

//scratchpad.rs
pub struct WorkingSet<S> { 
  // ...
  module_dbs: DbRegistry,
} 

impl<S> WorkingSet<S> { 
  pub fn get_unprovable(...);
  pub fn put_unprovable(...);
} 

Option 2: Allow modules to define and write to their own RocksDB tables

In this design, we don't allow modules to bring their own DBs, but we do allow them to create and write to custom tables in our current DB. This will require us to separate our current SchemaDB crate into at least two pieces, and expose the schema definition traits/macros from sov-modules-api. We will also have to modify our DB intialization code...

1. Generate a list of module-defined column families from the runtime
2. Open all of these DBs on startup

Option 3: Allow modules to read/write to their own prefix in a single (predefined) RocksDB table

This is the least flexible option, but the simplest to implement. We can simply create a new database schema and add a couple of methods to the working set.

[preston-evans98]

cc @bkolad

[bkolad]

The functionality has to also supportWrokingSet reverts. In my opinion, option 2 does not bring a lot of advantage over 3, 1 would be great, but it brings some complexity, and it looks like all our use cases can be achieved with 3. Let's go with 3?

[LukaszRozmej]

I would say that if possible it would be natural to use a column family here. We probably don't need 1 right now.

[bkolad]

We should ensure that the DB writes are atomic across the JMT and non-JMT, so the data remains consistent even if the rollup shuts down in the middle of the db-write.