Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Commit

Permalink
Metadata V15: Add Runtime API metadata (#13302)
Browse files Browse the repository at this point in the history
* impl_runtime_apis: Generate getters for `metadata_at` functions

Signed-off-by: Alexandru Vasile <[email protected]>

* runtime: Implement new `Metadata` runtime trait

Signed-off-by: Alexandru Vasile <[email protected]>

* runtime: Move `metadata_at` functions to construct_runtime macro

Signed-off-by: Alexandru Vasile <[email protected]>

* contruct_runtime: Use `OpaqueMetadata` from hidden imports

Signed-off-by: Alexandru Vasile <[email protected]>

* Adjust testing

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Add tests for the new API

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/proc-macro: Helper to extract documentation literals

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/proc-macro: Helper to filter all `cfg` attributes

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/proc-macro: Generate documentation getters for metadata

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/proc-macro: Avoid trait collision with snake case methods

Signed-off-by: Alexandru Vasile <[email protected]>

* proc-macro/tests: Check doc getters

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/proc-macro: Generate metadata for runtime methods

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/api: Export scale-info and frame-metadata

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/proc-macro: Generate metadata for runtime traits

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/runtime: Expose metadata v15 internally

Signed-off-by: Alexandru Vasile <[email protected]>

* test: Use metadata v15 from `lexnv/md_v15_test` branch

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/proc-macro: Generate crate access one module up

Signed-off-by: Alexandru Vasile <[email protected]>

* frame: Implement `runtime_metadata` for mocks and tests

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/proc-macro: Fix warnings

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/proc-macro: Add no-docs flag

Signed-off-by: Alexandru Vasile <[email protected]>

* frame: Adjust more tests

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Check runtime metadata correctness

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/benchmarking: Adjust benchmarks

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/benchmarks: Adjust more benchmarks

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/api: Fix clippy

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/proc-macro: Generate runtime metadata on the `decl_runtime_apis`

Signed-off-by: Alexandru Vasile <[email protected]>

* frame: Abuse Deref to resolve `runtime_metadata`

Signed-off-by: Alexandru Vasile <[email protected]>

* Revert "frame: Implement `runtime_metadata` for mocks and tests"

This reverts commit e4782de.

Revert "frame: Adjust more tests"

This reverts commit de1352c.

Revert "frame/benchmarking: Adjust benchmarks"

This reverts commit ae85bbe.

Signed-off-by: Alexandru Vasile <[email protected]>

Revert "frame/benchmarks: Adjust more benchmarks"

This reverts commit d37aa22.

* primitives/proc-macro: Remove unused imports and function

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Adjust runtime metadata test

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/tests: Remove doc getter test

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/support: Enable `no-metadata-docs` feature from `sp-api`

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/tests: Add `TypeInfo` for test::extrinsic

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/api: Expose scale-info and frame-metadata

Signed-off-by: Alexandru Vasile <[email protected]>

* Update frame-metadata to include v15

Signed-off-by: Alexandru Vasile <[email protected]>

* Fix merge conflicts

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata_ir: Add IR for runtime API metadata

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/metadata_ir: Convert IR to V15

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/api: Collect IR metadata for runtime API

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/api: Move `metadata_ir` from frame/support

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Adjust testing

Signed-off-by: Alexandru Vasile <[email protected]>

* frame/tests: Adjust `metadata_versions` test

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/runtime_metadata: Exclude default type parameters from methods

Signed-off-by: Alexandru Vasile <[email protected]>

* Update primitives/api/proc-macro/src/runtime_metadata.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/src/metadata_ir/types.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/src/metadata_ir/mod.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/proc-macro/src/utils.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/proc-macro/src/runtime_metadata.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/proc-macro/src/runtime_metadata.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/proc-macro/src/runtime_metadata.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/proc-macro/src/runtime_metadata.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/proc-macro/src/runtime_metadata.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/proc-macro/src/runtime_metadata.rs

Co-authored-by: Bastian Köcher <[email protected]>

* primitives: Fix build

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives/metadata-ir: Move IR to dedicated crate

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives: Reexport metadata-ir and frame-metadata

Signed-off-by: Alexandru Vasile <[email protected]>

* frame: Use apis field instead of runtime

Signed-off-by: Alexandru Vasile <[email protected]>

* Better documentation for the `Deref` abstraction

Signed-off-by: Alexandru Vasile <[email protected]>

* ui-tests: Check empty `impl_runtime_apis`

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives: Remove unneeded bounds on generic params

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives: Rename `collect_where_bounds` to `get_argument_type_param`

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives: Generate crate access per fn call

Signed-off-by: Alexandru Vasile <[email protected]>

* Revert "primitives: Remove unneeded bounds on generic params"

This reverts commit 5178e38.

* metadata-ir: Add no-std

Signed-off-by: Alexandru Vasile <[email protected]>

* primitives: Adjust where bounds

Signed-off-by: Alexandru Vasile <[email protected]>

* Change `frame-metadata` branch to "origin/main"

Signed-off-by: Alexandru Vasile <[email protected]>

* Update to `main` from origin

Signed-off-by: Alexandru Vasile <[email protected]>

* Update frame-metadata to crates.io v15.1

Signed-off-by: Alexandru Vasile <[email protected]>

* Revert "ui-tests: Check empty `impl_runtime_apis`"

This reverts commit cf78a71.

* Move ui test to primitives/ui

Signed-off-by: Alexandru Vasile <[email protected]>

* Update frame/support/test/tests/runtime_metadata.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update primitives/api/proc-macro/src/runtime_metadata.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Test already covered by `empty_impl_runtime_apis_call.stderr`

This reverts commit 3bafb29.

* Retriger CI

Signed-off-by: Alexandru Vasile <[email protected]>

* Import `TokenStream` as `TokenStream2`

Signed-off-by: Alexandru Vasile <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <[email protected]>
  • Loading branch information
lexnv and bkchr authored Apr 11, 2023
1 parent 932f820 commit 416b0f5
Show file tree
Hide file tree
Showing 25 changed files with 993 additions and 31 deletions.
19 changes: 17 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,7 @@ members = [
"primitives/keystore",
"primitives/maybe-compressed-blob",
"primitives/merkle-mountain-range",
"primitives/metadata-ir",
"primitives/npos-elections",
"primitives/npos-elections/fuzzer",
"primitives/offchain",
Expand Down
4 changes: 2 additions & 2 deletions frame/support/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ targets = ["x86_64-unknown-linux-gnu"]
serde = { version = "1.0.136", optional = true, features = ["derive"] }
codec = { package = "parity-scale-codec", version = "3.2.2", default-features = false, features = ["derive", "max-encoded-len"] }
scale-info = { version = "2.1.1", default-features = false, features = ["derive"] }
frame-metadata = { version = "15.0.0", default-features = false, features = ["v14"] }
frame-metadata = { version = "15.1.0", default-features = false, features = ["v14", "v15-unstable"] }
sp-api = { version = "4.0.0-dev", default-features = false, path = "../../primitives/api" }
sp-std = { version = "5.0.0", default-features = false, path = "../../primitives/std" }
sp-io = { version = "7.0.0", default-features = false, path = "../../primitives/io" }
Expand Down Expand Up @@ -74,7 +74,7 @@ runtime-benchmarks = []
try-runtime = []
# By default some types have documentation, `no-metadata-docs` allows to reduce the documentation
# in the metadata.
no-metadata-docs = ["frame-support-procedural/no-metadata-docs"]
no-metadata-docs = ["frame-support-procedural/no-metadata-docs", "sp-api/no-metadata-docs"]
# By default some types have documentation, `full-metadata-docs` allows to add documentation to
# more types in the metadata.
full-metadata-docs = ["scale-info/docs"]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,24 @@ pub fn expand_runtime_metadata(
quote! {
impl #runtime {
fn metadata_ir() -> #scrate::metadata_ir::MetadataIR {
// Each runtime must expose the `runtime_metadata()` to fetch the runtime API metadata.
// The function is implemented by calling `impl_runtime_apis!`.
//
// However, the `construct_runtime!` may be called without calling `impl_runtime_apis!`.
// Rely on the `Deref` trait to differentiate between a runtime that implements
// APIs (by macro impl_runtime_apis!) and a runtime that is simply created (by macro construct_runtime!).
//
// Both `InternalConstructRuntime` and `InternalImplRuntimeApis` expose a `runtime_metadata()` function.
// `InternalConstructRuntime` is implemented by the `construct_runtime!` for Runtime references (`& Runtime`),
// while `InternalImplRuntimeApis` is implemented by the `impl_runtime_apis!` for Runtime (`Runtime`).
//
// Therefore, the `Deref` trait will resolve the `runtime_metadata` from `impl_runtime_apis!`
// when both macros are called; and will resolve an empty `runtime_metadata` when only the `construct_runtime!`
// is called.
//
// `Deref` needs a reference for resolving the function call.
let rt = #runtime;

#scrate::metadata_ir::MetadataIR {
pallets: #scrate::sp_std::vec![ #(#pallets),* ],
extrinsic: #scrate::metadata_ir::ExtrinsicMetadataIR {
Expand All @@ -95,7 +113,8 @@ pub fn expand_runtime_metadata(
})
.collect(),
},
ty: #scrate::scale_info::meta_type::<#runtime>()
ty: #scrate::scale_info::meta_type::<#runtime>(),
apis: (&rt).runtime_metadata(),
}
}

Expand Down
25 changes: 25 additions & 0 deletions frame/support/procedural/src/construct_runtime/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,31 @@ fn construct_runtime_final_expansion(
type RuntimeBlock = #block;
}

// Each runtime must expose the `runtime_metadata()` to fetch the runtime API metadata.
// The function is implemented by calling `impl_runtime_apis!`.
//
// However, the `construct_runtime!` may be called without calling `impl_runtime_apis!`.
// Rely on the `Deref` trait to differentiate between a runtime that implements
// APIs (by macro impl_runtime_apis!) and a runtime that is simply created (by macro construct_runtime!).
//
// Both `InternalConstructRuntime` and `InternalImplRuntimeApis` expose a `runtime_metadata()` function.
// `InternalConstructRuntime` is implemented by the `construct_runtime!` for Runtime references (`& Runtime`),
// while `InternalImplRuntimeApis` is implemented by the `impl_runtime_apis!` for Runtime (`Runtime`).
//
// Therefore, the `Deref` trait will resolve the `runtime_metadata` from `impl_runtime_apis!`
// when both macros are called; and will resolve an empty `runtime_metadata` when only the `construct_runtime!`
// is called.

#[doc(hidden)]
trait InternalConstructRuntime {
#[inline(always)]
fn runtime_metadata(&self) -> #scrate::sp_std::vec::Vec<#scrate::metadata_ir::RuntimeApiMetadataIR> {
Default::default()
}
}
#[doc(hidden)]
impl InternalConstructRuntime for &#name {}

#outer_event

#outer_origin
Expand Down
2 changes: 1 addition & 1 deletion frame/support/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ pub use paste;
pub use scale_info;
#[cfg(feature = "std")]
pub use serde;
pub use sp_api::metadata_ir;
pub use sp_core::{OpaqueMetadata, Void};
#[doc(hidden)]
pub use sp_core_hashing_proc_macro;
Expand Down Expand Up @@ -80,7 +81,6 @@ pub mod error;
pub mod crypto;
pub mod dispatch_context;
pub mod instances;
pub mod metadata_ir;
pub mod migrations;
pub mod traits;
pub mod weights;
Expand Down
2 changes: 2 additions & 0 deletions frame/support/test/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ targets = ["x86_64-unknown-linux-gnu"]
serde = { version = "1.0.136", default-features = false, features = ["derive"] }
codec = { package = "parity-scale-codec", version = "3.2.2", default-features = false, features = ["derive"] }
scale-info = { version = "2.1.1", default-features = false, features = ["derive"] }
sp-api = { version = "4.0.0-dev", default-features = false, path = "../../../primitives/api" }
sp-arithmetic = { version = "6.0.0", default-features = false, path = "../../../primitives/arithmetic" }
sp-io = { version = "7.0.0", path = "../../../primitives/io", default-features = false }
sp-state-machine = { version = "0.13.0", optional = true, path = "../../../primitives/state-machine" }
Expand Down Expand Up @@ -47,6 +48,7 @@ std = [
"sp-state-machine",
"sp-arithmetic/std",
"sp-version/std",
"sp-api/std",
]
try-runtime = ["frame-support/try-runtime"]
# WARNING:
Expand Down
2 changes: 1 addition & 1 deletion frame/support/test/tests/pallet.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1671,7 +1671,7 @@ fn metadata_at_version() {

#[test]
fn metadata_versions() {
assert_eq!(vec![LATEST_METADATA_VERSION], Runtime::metadata_versions());
assert_eq!(vec![LATEST_METADATA_VERSION, u32::MAX], Runtime::metadata_versions());
}

#[test]
Expand Down
Loading

0 comments on commit 416b0f5

Please sign in to comment.