Metadata V15: Generate Runtime APIs

[lexnv]

This PR adds support for the runtime API using the metadata exposed by substrate in paritytech/substrate#13302.

API Design

• runtime_apis.rs generates a new runtime API interface that allows users to call directly into the runtime of substrate-based chains

• The Metadata_metadata runtime function call is accessible via metadata().metadata(). This preserves two indirections: one for the runtime API trait (identified by the first metadata()) and one for the concrete function to call from the runtime trait. This follows the design decisions in subxt to utilize empty struct objects instead of paths for improved ergonomics and formatting.

• Arguments can be passed to the runtime functions that accept them. The provided arguments to the runtime function call are scale-encoded in the codegen directly.

• codegen API using the subxt macro

pub struct RuntimeApi;
impl RuntimeApi {
    pub fn metadata(&self) -> Metadata {
        Metadata
    }
    ...
    
 #[doc = " The `Metadata` api trait that returns metadata for the runtime."]
pub struct Metadata;
impl Metadata {
    #[doc = " Returns the metadata of a runtime."]
    pub fn metadata(
        &self,
    ) -> ::subxt::runtime_api::StaticRuntimeApiPayload<runtime_types::sp_core::OpaqueMetadata>
    {
        let mut result = Vec::new();
        ::subxt::runtime_api::StaticRuntimeApiPayload::new_static(
            "Metadata_metadata",
            result,
            [
                158u8, 223u8, 229u8, 177u8, 228u8, 49u8, 24u8, 219u8, 143u8, 82u8, 255u8,
                132u8, 245u8, 210u8, 160u8, 197u8, 218u8, 10u8, 63u8, 165u8, 115u8, 91u8,
                70u8, 151u8, 246u8, 51u8, 18u8, 235u8, 91u8, 143u8, 27u8, 127u8,
            ],
        )
    }

• API is similar to submitting extrinsics: users must first construct the API payload and then submit it to the chain.

    // Create a runtime API payload that calls into
    // `Core_version` function.
    let call = polkadot::apis().core().version();

    // Submit the runtime API call.
    let version = api.runtime_api().at_latest().await?.call(call).await;
    println!("Core_version: {:?}", version);

Runtime API Payload

This PR introduces two types of payloads: StaticRuntimeApiPayload and DynamicRuntimeApiPayload.
The payload that these types are built around takes in two generic parameters:

• Argument data
• Return type

Each payload must implement the following trait:

pub trait RuntimeApiPayload {
    /// The target type of the return value.
    type ReturnType: DecodeWithMetadata;

    /// The runtime API function name.
    fn fn_name(&self) -> &str;

    /// Scale encode arguments data to the provided output.
    fn encode_args_to(&self, metadata: &Metadata, out: &mut Vec<u8>) -> Result<(), Error>;

    /// Returns the details needed to validate the runtime API call, which
    /// include a statically generated validation hash.
    fn validation_details(&self) -> Option<ValidationDetails>;
}

The associated ReturnType is used for decoding the result of the function back to a provided type (either by codegen or "dynamically" by the user).

StaticRuntimeApiPayload

This payload leverages the API exposed by the codegen and is strongly typed.

The arguments are provided by the codegen as scale-encoded bytes, which makes the generic argument data Vec<u8>. In this case encode_args_to ignores the Metadata parameters and passes through the bytes
obtained from the codegen directly.

Validation is enabled by default for these payloads, users can opt-out however using the unvalidated() method.

DynamicRuntimeApiPayload

This offers users more control when it comes to making dynamic calls. Similar to transactions,
users can construct a dynamic runtime API payload from various objects and the subxt
will do its best to encode those objects to the shape of the runtime metadata.

Build on top of #912.

Closes paritytech/subxt-team-milestones#21.

[jsdw]

Nit: maybe just manually have an into_codegen_metadata_version(metadata_version: Option<MetadataVersion>) -> CodegenMetadataVersion so we don't have to hardcode the 14 here (I'm trying to get away from assuming anything about what the node has in terms of metadata versions :))?

[jsdw]

Or clap has default_value = <str> which we could use so that MetadataVersion could have a Latest variant and clap will default to using that. (maybe we can just use CodegenMetadataVersion directly in that case?)

[lexnv]

That makes sense! Have ended up using just the CodegenMetadataVersion here :D Thanks!

[jsdw]

Does the ? here lead to us exiting early if this call fails due to "Metadata_metadata_versions" not existing?

[lexnv]

The code here seems a bit complex after another look, I'll take a note to go back and do a plain-old match to make it easier to follow:

• we initially do a .ok() which throws away the Result<String, Error> from the client and wraps into a Option<String>
• we transpose an Option<Result<..>>, out of which the Result can only be part of the closure: hex decoding or coded::decod failed

I believe we should be safe here, I'll be a bit more explicit with a simple match and such :D

[jsdw]

Looks awesome; good job on this!