Try json optimization

[ethanfrey]

@hashedone asked some questions about ownership of variables and passing String vs &str. I realised much of the API was designed when my Rust knowledge was much weaker. I would like to allow novice developers to use the simple case, but we should allow contract developers to use &str in InstantiateMsg, ExecuteMsg, and QueryMsg to avoid copies, without affecting anyone else using this framework. (opt-in per contract).

I tried to make a simple demo on Hackatom to see what affect it would have on gas cost (as we currently meter this, so we have a nice comparison).

I had to make a few changes open to more lifetimes in cosmwasm-std, but eventually hit an issue in serde-json-wasm:

pub fn from_slice<T>(v: &[u8]) -> Result<T>
where
    T: de::DeserializeOwned,
{
    let mut de = Deserializer::new(v);
    let value = de::Deserialize::deserialize(&mut de)?;
    de.end()?;

    Ok(value)
}

https://github.com/CosmWasm/serde-json-wasm/blob/main/src/de/mod.rs#L574-L584

Before I go deeper into this rabbit hole, I would like some feedback. Seems like a rather simple change to make it more general, but let devs still use existing code as DeserializedOwned without any changes.

[webmaster128]

At very high level, Deserialize with reference fields can only be used if the data source outlives the target structure. This is the case in our usage:

fn _do_execute<M, C, E>(
    execute_fn: &dyn Fn(DepsMut, Env, MessageInfo, M) -> Result<Response<C>, E>,
    env_ptr: *mut Region,
    info_ptr: *mut Region,
    msg_ptr: *mut Region,
) -> ContractResult<Response<C>>
where
    M: DeserializeOwned + JsonSchema,
    C: Serialize + Clone + fmt::Debug + PartialEq + JsonSchema,
    E: ToString,
{
    let env: Vec<u8> = unsafe { consume_region(env_ptr) };
    let info: Vec<u8> = unsafe { consume_region(info_ptr) };
    let msg: Vec<u8> = unsafe { consume_region(msg_ptr) };

    let env: Env = try_into_contract_result!(from_slice(&env));
    let info: MessageInfo = try_into_contract_result!(from_slice(&info));
    let msg: M = try_into_contract_result!(from_slice(&msg)); // msg outlives msg (variable shadowing is evil)

    let mut deps = make_dependencies();
    execute_fn(deps.as_mut(), env, info, msg).into()
}

DeserializeOwned is needed for deserializers with a vanishing data source, such as streaming deserializers.

So I don't see anything conceptually wrong with that. I just wonder if it is really with the effort to save a few string copies.

[webmaster128]

I also do not know to what degree schemars supports deriving JSON schema from reference fields. I did not find anything in the repo. This should be tested.

[ethanfrey]

I don't think it is that much code, good point about checking json serializer.

Mainly I want to see if we can make the std lib support such a case without too much work and mainly seems like I just need to annotate lifetimes

[ethanfrey]

Wait, now I remember why there are no &str. We can't properly do that in JSON, as it may need to escape strings, so it gets quite complicated CoW if we try to maybe avoid heap-allocations. Nothing easy.

You commented this in serde-json-wasm CosmWasm/serde-json-wasm#16 (comment)

[ethanfrey]

I'm going to close this approach, we need String for all JSON deserialization.