Skip to content
This repository has been archived by the owner on May 22, 2023. It is now read-only.

Feature/update to polkadot v0 9 18 #169

Merged
merged 93 commits into from
Dec 1, 2022

Conversation

MRamanenkau
Copy link

@MRamanenkau MRamanenkau commented Oct 21, 2022

pallet-whitelist and pallet_state_trie_migration are disabled. Details are in notion

yarikbratashchuk and others added 30 commits February 11, 2022 07:46
* poc logic

* set_code_hash impl, tests, benchmark

* Address @xgreenx's comments

* Move func defs closer to set_storage

* Check if code exists

- increment/decrement codehash refcount

* Document error for non-existing code hash

* Revert unrelated change

* Changes due to @athei's review

* Fix error handling

- comment errors: ReturnCodes
- update mock ext implementation
- return Error::CodeNotFound when no code for such hash

* Emit ContractCodeUpdated when setting new code_hash

* Address @athei's comments

* Move related defs to the bottom

* Minor comment update

Co-authored-by: Alexander Theißen <[email protected]>

* Improve docs

* Improve docs

* Update frame/contracts/src/wasm/runtime.rs

Co-authored-by: Alexander Theißen <[email protected]>

* Refactor set_code_hash test

* Minor change to benchmark

Co-authored-by: Alexander Theißen <[email protected]>

* Minor change to benchmark

Co-authored-by: Alexander Theißen <[email protected]>

* Minor comment refactor

Co-authored-by: Alexander Theißen <[email protected]>

* Address @HCastano's comments

* Update seal_set_code_hash comment

Co-authored-by: Hernando Castano <[email protected]>

* Move set_code_hash after delegate_call

* Move function to the bottom

* Moved and changed banchmark, added verify block

* Bring back previous benchmark

* Remove skip_meta for seal_set_code_hash

* Bring back skip_meta for seal_set_storage_per_new_kb

* Apply weights

Co-authored-by: Alexander Theißen <[email protected]>
Co-authored-by: Hernando Castano <[email protected]>
…les (paritytech#10563)

* Moves wasmi specific `ImportResolver` and `MemoryTransfer` impls to submodule

* Splits context store environmental, moves impl `Externals` to wasmi backend

* Adds wasmer sandbox backend stub module

* Move sandbox impl code to backend specific modules

* Moves wasmi stuff

* Fixes value conversion

* Makes it all compile

* Remove `with_context_store`

* Moves `WasmerBackend` to the impl

* Reformat the source

* Moves wasmer MemoryWrapper

* Reformats the source

* Fixes mutability

* Moves backend impls to a submodule

* Fix visibility

* Reformat the source

* Feature gate wasmer backend module

* Moves wasmi memory allocation to backend module

* Rename WasmerBackend to Backend

* Refactor dispatch result decoding, get rid of Wasmi types in common sandbox code

* Reformat the source

* Remove redundant prefixes in backend functions

* Remove wasmer-sandbox from default features

* Post-review changes

* Add conversion soundness proof

* Remove redundant prefix

* Removes now redundant clone_inner

* Add `Error::SandboxBackend`, refactor invoke result

* Fix comments

* Rename `Error::SandboxBackend` to `Sandbox`

* Simplifies logic in `wasmer_backend::invoke`

* Fixes memory management
* assets: execute `died` hook outside of mutate

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* assets: extend tests for `died` hook

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* assets: update doc of FrozenBalance::died

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* assets: review fixes

- fix cases where `died` should not have been called
- use `Option<DeadConsequence>` instead of `DeadConsequence`

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* assets: update comment in mock.rs

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* assets: return `Remove` in dead_account

The return value is ignored in the only case that it is produced
by a call, but having it this way makes it more understandable.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
…aritytech#10845)

* build-script-utils: allow reading the git commit hash from env var

* build-script-utils: make the env var name substrate specific
* Don't create DB leaves when syncing historic blocks

* Changed leaves check and added test

* fmt
* subkey: Support `--version` cli command

* FMT 🤦
* Remove `u32_trait`

This trait only existed because there wasn't any const generic support at time of creation. However,
we now have support for it :)

* FMT
* wasm-builder: Support latest nightly

With latest nightly, aka rust version 1.60+ namespaced features are added. This changes the handling
of optional dependencies. We currently have features that enable optional dependencies when `std` is
enabled. This was before no problem, but now the wasm-builder detects them as enabled. To support
the transition period until 1.60 is released as stable, this pr adds an heuristic to not enable these
optional crates in the wasm build when they are enabled in the `std` feature. This heuristic fails
when someones enables these optional dependencies from the outside as well as via the `std` feature,
however we hope that no one is doing this at the moment. When namespaced features are enabled, these
dependencies needs to be enabled using `dep:dependency-name` to solve this properly.

https://doc.rust-lang.org/cargo/reference/unstable.html#namespaced-features

* Remove accidentally added features
* Fix call weights

* Fix instantiate benchmark

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Remove stale and superflous comments

* `decrement_refcount` should be infallible

* Don't hardcode increment_refcount, decrement_refcount

* Rename CopyIn/CopyOut

* Fix warning in tests

Co-authored-by: Parity Bot <[email protected]>
Compressed ECDSA keys requires to have 0x02 or 0x03 as their first byte
in order to allow public key recovery.

Nevertheless the test was working because of the `unwrap_or_default()`
at the end of the conversion routine (i.e. the invalid keys were
converted to an empty vector).
* Mark sync_state_genSyncSpec JSON-RPC as safe

* Note that parameter is unused

* Ideally I'd wait for compilation to be finished before pushing, but it's really taking a long time

* Remove deny_unsafe parameter

* Remove unused dependency
… function uniformly (paritytech#10835)

* sp-core-hashing: use sha3 instead of tiny-keccak

Signed-off-by: koushiro <[email protected]>

* use blake2 instead of blake2-rfc

Signed-off-by: koushiro <[email protected]>

* improve som hashing of sp-core and sp-api-proc-macro

Signed-off-by: koushiro <[email protected]>

* Some nits

Signed-off-by: koushiro <[email protected]>

* cargo fmt

Signed-off-by: koushiro <[email protected]>
* refactor election score

* Test for ord

* remove reference

* vec -> slice

* change iter to iter_by_significance

* improve doc

* fix typo

* add explanation about [u128; 3]

* consolidate threshold and epsilon

* random fixes

* rename

* remove Into

* make iter_by_sig private

* remove vec

* Fix tests
* add test

* Assorted refactorings

* complete test

* saturating math

* final check

* use `default`

Co-authored-by: Gav Wood <[email protected]>
Improves the logging by switching to `tracing` for a better log output. Besides that, it also adds a
trace for the function being executed.
* Track allowed requests for state/warp sync

* Added missing allowed_requests resets

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* fmt

Co-authored-by: Bastian Köcher <[email protected]>
* Simplify `num_connected_peers`

* Track requested peer counts

* Revert "Track requested peer counts"

This reverts commit 9f1c870.

* Remove `substrate_sub_libp2p_peerset_num_requested` metric

* Remove two unused functions that I forgot to get rid of in previous commit
* Introduce iters into BoundedVec

* Fix

* Remove unneeded funcs

* Update frame/support/src/storage/bounded_vec.rs

* Update frame/support/src/storage/bounded_vec.rs

Co-authored-by: Bastian Köcher <[email protected]>
arkpar and others added 14 commits March 4, 2022 11:30
* Refactored tx storage database scheme

* Bump parity-db

* fmt

* Fix handling invalid index size + test

* Removed superflous result

* Minor changes

* fmt
…aritytech#10971)

* stabilize `seal_is_contract`

* stabilize `seal_caller_is_origin`
* starting

* Updated from other branch.

* setting flag

* flag in storage struct

* fix flagging to access and insert.

* added todo to fix

* also missing serialize meta to storage proof

* extract meta.

* Isolate old trie layout.

* failing test that requires storing in meta when old hash scheme is used.

* old hash compatibility

* Db migrate.

* runing tests with both states when interesting.

* fix chain spec test with serde default.

* export state (missing trie function).

* Pending using new branch, lacking genericity on layout resolution.

* extract and set global meta

* Update to branch 4

* fix iterator with root flag (no longer insert node).

* fix trie root hashing of root

* complete basic backend.

* Remove old_hash meta from proof that do not use inner_hashing.

* fix trie test for empty (force layout on empty deltas).

* Root update fix.

* debug on meta

* Use trie key iteration that do not include value in proofs.

* switch default test ext to use inner hash.

* small integration test, and fix tx cache mgmt in ext.
test  failing

* Proof scenario at state-machine level.

* trace for db upgrade

* try different param

* act more like iter_from.

* Bigger batches.

* Update trie dependency.

* drafting codec changes and refact

* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.

* alt hashing only for branch with value.

* fix trie tests

* Hash of value include the encoded size.

* removing fields(broken)

* fix trie_stream to also include value length in inner hash.

* triedbmut only using alt type if inner hashing.

* trie_stream to also only use alt hashing type when actually alt hashing.

* Refactor meta state, logic should work with change of trie treshold.

* Remove NoMeta variant.

* Remove state_hashed trigger specific functions.

* pending switching to using threshold, new storage root api does not
make much sense.

* refactoring to use state from backend (not possible payload changes).

* Applying from previous state

* Remove default from storage, genesis need a special build.

* rem empty space

* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.

* fix compilation

* Right logic to avoid switch on reencode when default layout.

* Clean up some todos

* remove trie meta from root upstream

* update upstream and fix benches.

* split some long lines.

* UPdate trie crate to work with new design.

* Finish update to refactored upstream.

* update to latest triedb changes.

* Clean up.

* fix executor test.

* rust fmt from master.

* rust format.

* rustfmt

* fix

* start host function driven versioning

* update state-machine part

* still need access to state version from runtime

* state hash in mem: wrong

* direction likely correct, but passing call to code exec for genesis
init seem awkward.

* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.

* stateversion from runtime version (core api >= 4).

* update trie, fix tests

* unused import

* clean some TODOs

* Require RuntimeVersionOf for executor

* use RuntimeVersionOf to resolve genesis state version.

* update runtime version test

* fix state-machine tests

* TODO

* Use runtime version from storage wasm with fast sync.

* rustfmt

* fmt

* fix test

* revert useless changes.

* clean some unused changes

* fmt

* removing useless trait function.

* remove remaining reference to state_hash

* fix some imports

* Follow chain state version management.

* trie update, fix and constant threshold for trie layouts.

* update deps

* Update to latest trie pr changes.

* fix benches

* Verify proof requires right layout.

* update trie_root

* Update trie deps to  latest

* Update to latest trie versioning

* Removing patch

* update lock

* extrinsic for sc-service-test using layout v0.

* Adding RuntimeVersionOf to CallExecutor works.

* fmt

* error when resolving version and no wasm in storage.

* use existing utils to instantiate runtime code.

* migration pallet

* Patch to delay runtime switch.

* Revert "Patch to delay runtime switch."

This reverts commit d35f273.

* fix test

* fix child migration calls.

* useless closure

* remove remaining state_hash variables.

* Fix and add more tests

* Remove outdated comment

* useless inner hash

* fmt

* remote tests

* finally ksm works

* batches are broken

* clean the benchmarks

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Joshy Orndorff <[email protected]>

* Update frame/state-trie-migration/src/lib.rs

* brand new version

* fix build

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Update primitives/storage/src/lib.rs

Co-authored-by: cheme <[email protected]>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <[email protected]>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <[email protected]>

* fmt and opt-in feature to apply state change.

* feature gate core version, use new test feature for node and test node

* Use a 'State' api version instead of Core one.

* fix merge of test function

* use blake macro.

* Fix state api (require declaring the api in runtime).

* Opt out feature, fix macro for io to select a given version
instead of latest.

* run test nodes on new state.

* fix

* new test structure

* new testing stuff from emeric

* Add commit_all, still not working

* Fix all tests

* add comment

* we have PoV tracking baby

* document stuff, but proof size is still wrong

* FUCK YEAH

* a big batch of review comments

* add more tests

* tweak test

* update config

* some remote-ext stuff

* delete some of the old stuff

* sync more files with master to minimize the diff

* Fix all tests

* make signed migration a bit more relaxed

* add witness check to signed submissions

* allow custom migration to also go above limit

* Fix these pesky tests

* ==== removal of the unsigned stuff ====

* Make all tests work again

* separate the tests from the logic so it can be reused easier

* fix overall build

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <[email protected]>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <[email protected]>

* Slightly better termination

* some final tweaks

* Fix tests

* Restrict access to signed migrations

* address most of the review comments

* fix defensive

* New simplified code

* Fix weights

* fmt

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* make the tests correctly fail

* Fix build

* Fix build

* try and fix the benchmarks

* fix build

* Fix cargo file

* Fix runtime deposit

* make rustdoc happy

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_state_trie_migration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/state-trie-migration/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: cheme <[email protected]>
Co-authored-by: Guillaume Thiolliere <[email protected]>
Co-authored-by: Joshy Orndorff <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Parity Bot <[email protected]>
…tech#10974)

std::fmt::Display shows formats as reduced hash (e.g. 0xb0a8…dafe)

Use hex::encode to format full hash.

Signed-off-by: acatangiu <[email protected]>
* Move block+ext weights to own mod

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Unused import

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
* add block usage logs to system pallet

* add Debug

* use % instead of default Debug impl

* change formatting

* revert
…paritytech#10915)

* Add a test case for the determinism of recorded proof

* Replace HashMap with BTreeMap for the actual proof records

* cargo +nightly fmt --all

* Store the trie nodes in BTreeSet for StorageProof

* Nit

* Revert the BTreeMap changes and sort when converting to storage proof

* Remove PartialEq from StorageProof

* Remove unnecessary change

* Add `compare` method to StorageProof

* FMT

* Dummy change to trigger CI

* Use `BTreeSet` for StorageProof and keep using `Vec` for CompactProof

* Update comment on `iter_nodes`

* Revert `PartialEq` removal
* Update clap to the latest version

Besides that it also removes some `structopt` leftovers from some docs.

* Fix compile errors

* More fixes
@MRamanenkau MRamanenkau force-pushed the feature/update-to-polkadot-v0-9-18 branch from d1d397e to 92b9b94 Compare October 21, 2022 12:47
@MRamanenkau MRamanenkau marked this pull request as ready for review October 24, 2022 14:22
@MRamanenkau MRamanenkau changed the base branch from feature/update-to-polkadot-v0-9-17 to dev-cere December 1, 2022 16:02
@MRamanenkau MRamanenkau dismissed AndreiNavoichyk’s stale review December 1, 2022 16:02

The base branch was changed.

@MRamanenkau MRamanenkau merged commit b8cb48b into dev-cere Dec 1, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.