test: Add more high-level methods to MockChain (#807)

* feat: High level mockchain methods; re-enable and fix kernel tests

CHANGELOG.md

@@ -10,6 +10,7 @@
 - [BREAKING] Increase of nonce does not require changes in account state any more (#796).
 - Added `CHANGELOG.md` warning message on CI (#799).
 - Account deltas can now be merged (#797).
+- Added high-level methods for `MockChain` and related structures (#807).
 - Changed `AccountCode` procedures from merkle tree to sequential hash + added storage_offset support (#763).
 - [BREAKING] Renamed `NoteExecutionHint` to `NoteExecutionMode` and added new `NoteExecutionHint` to `NoteMetadata` (#812, #816).
 - [BREAKING] Refactored and simplified `NoteOrigin` and `NoteInclusionProof` structs (#810, #814).

README.md

@@ -33,13 +33,13 @@ Polygon Miden is currently on release v0.5. This is an early version of the prot
 - **Public notes**. With public notes, the users are be able to store all note details on-chain, thus, eliminating the need to communicate note details via side-channels.
 - **Local transactions**. Users can execute and prove transactions locally on their devices. The Miden Operator verifies the proofs and if the proofs are valid, updates the state of the rollup accordingly.
 - **Standard account**. Users can create accounts using a small number of standard account interfaces (e.g., basic wallet). In the future, the set of standard smart contracts will be expanded.
-- **Standard notes**. Can create notes using standardized note scripts such as pay-to-ID (`P2ID`) and atomic swap (`SWAP`). In the future, the set of standardized notes will be expanded.
+- **Standard notes**. Can create notes using standardized note scripts such as Pay-to-ID (`P2ID`) and atomic swap (`SWAP`). In the future, the set of standardized notes will be expanded.
+- **Delegated note inclusion proofs**. By delegating note inclusion proofs, users can create chains of dependent notes which are included into a block as a single batch.
 
 ### Planned features
 
 - **More storage types**. In addition to simple storage slots and storage maps, the accounts will be able to store data in storage arrays.
 - **Transaction recency conditions**. Users will be able to specify how close to the chain tip their transactions are to be executed. This will enable things like rate limiting and oracles.
-- **Delegated note inclusion proofs**. By delegating note inclusion proofs, users will be able to create chains of dependent notes which are would be included into a block as a single batch.
 - **Network transactions**. Users will be able to create notes intended for network execution. Such notes will be included into transactions executed and proven by the Miden operator.
 - **Encrypted notes**. With encrypted notes users will be able to put all note details on-chain, but the data contained withing the notes would be encrypted with the recipients key.
 

docs/introduction/get-started/create-account-use-faucet.md

@@ -21,9 +21,9 @@ The Miden client facilitates interaction with the Miden rollup and provides a wa
 2. Install the Miden client.
 
       ```shell
-      cargo install miden-client --features testing,concurrent
+      cargo install miden-cli --features testing,concurrent
       ```
-      You can now use the `miden --version` command, and you should see `Miden 0.3.0`.
+      You can now use the `miden --version` command, and you should see `Miden 0.4.0`.
 
 3. Initialize the client. This creates the `miden-client.toml` file.
 
@@ -91,7 +91,7 @@ Save the account ID for a future step.
       ![Result of viewing miden notes](../../img/get-started/note-view.png)
 
 !!! tip "The importance of syncing"
-      - As you can see, the listed note is lacking a `commit-height`. 
+      - As you can see, the note is listed as `Expected`.
       - This is because you have received a private note but have not yet synced your view of the rollup to check that the note is the result of a valid transaction.
       - Hence, before consuming the note we will need to update our view of the rollup by syncing.
       - Many users could have received the same private note, but only one user can consume the note in a transaction that gets verified by the Miden operator.
@@ -108,11 +108,16 @@ You will see something like this as output:
 
 ```sh
 State synced to block 179672
+New public notes: 0
+Tracked notes updated: 1
+Tracked notes consumed: 0
+Tracked accounts updated: 0
+Commited transactions: 0
 ```
 
 ## Consume the note & receive the funds
 
-1. Now that we have synced the client, the input-note imported from the faucet should have a `Commit Height` confirming it exists at the rollup level:
+1. Now that we have synced the client, the input-note imported from the faucet should have a `Committed` status, confirming it exists at the rollup level:
 
       ```shell
       miden notes
@@ -135,7 +140,36 @@ State synced to block 179672
       miden consume-notes --account <Account-Id> <Note-Id>
       ```
 
-  Amazing! You just have created a client-side zero-knowledge proof locally on your machine. 
+5. You should see a confirmation message like this:
+
+      ![Transaction confirmation message](../../img/get-started/transaction-confirmation.png)
+
+6. After confirming you can view the new note status by running the following command:
+
+      ```shell
+      miden notes
+      ```
+
+7. You should see something like this:
+
+      ![Viewing process info](../../img/get-started/processing-note.png)
+
+8. The note is `Processing`. This means that the proof of the transaction was sent, but there is no network confirmation yet. You can update your view of the rollup by syncing again:
+
+      ```shell
+      miden sync
+      ```
+
+9. After syncing, you should have received confirmation of the consumed note. You should see the note as `Consumed` after listing the notes:
+
+      ```shell
+      miden notes
+      ```
+
+      ![Viewing consumed note](../../img/get-started/consumed-note.png)
+
+
+  Amazing! You just have created a client-side zero-knowledge proof locally on your machine and submitted it to the Miden rollup.
 
 !!! tip
       You only need to copy the top line of characters of the Note ID.

miden-lib/build.rs

@@ -133,6 +133,20 @@ fn compile_tx_kernel(source_dir: &Path, target_dir: &Path) -> Result<Assembler>
     let masb_file_path = target_dir.join("tx_kernel.masb");
     kernel_main.write_to_file(masb_file_path).into_diagnostic()?;
 
+    #[cfg(feature = "testing")]
+    {
+        // Build kernel as a library and save it to file.
+        // This is needed in test assemblers to access individual procedures which would otherwise
+        // be hidden when using KernelLibrary (api.masm)
+        let namespace = "kernel".parse::<LibraryNamespace>().expect("invalid base namespace");
+        let test_lib =
+            Library::from_dir(source_dir.join("lib"), namespace, assembler.clone()).unwrap();
+
+        let masb_file_path =
+            target_dir.join("kernel_library").with_extension(Library::LIBRARY_EXTENSION);
+        test_lib.write_to_file(masb_file_path).into_diagnostic()?;
+    }
+
     Ok(assembler)
 }
 

miden-lib/src/transaction/mod.rs

@@ -249,3 +249,34 @@ impl TransactionKernel {
         Ok(TransactionOutputs { account, output_notes })
     }
 }
+
+#[cfg(feature = "testing")]
+impl TransactionKernel {
+    const KERNEL_TESTING_LIB_BYTES: &'static [u8] =
+        include_bytes!(concat!(env!("OUT_DIR"), "/assets/kernels/kernel_library.masl"));
+
+    pub fn kernel_as_library() -> miden_objects::assembly::Library {
+        miden_objects::assembly::Library::read_from_bytes(Self::KERNEL_TESTING_LIB_BYTES)
+            .expect("failed to deserialize transaction kernel library")
+    }
+
+    /// Contains code to get an instance of the [Assembler] that should be used in tests.
+    ///
+    /// This assembler is similar to the assembler used to assemble the kernel and transactions,
+    /// with the difference that it also includes an extra library on the namespace of `kernel`.
+    /// The `kernel` library is added separately because even though the library (`api.masm`) and
+    /// the kernel binary (`main.masm`) include this code, it is not exposed explicitly. By adding it
+    /// separately, we can expose procedures from `/lib` and test them individually.
+    pub fn assembler_testing() -> Assembler {
+        let source_manager = Arc::new(DefaultSourceManager::default());
+        let kernel_library = Self::kernel_as_library();
+
+        Assembler::with_kernel(source_manager, Self::kernel())
+            .with_library(StdLibrary::default())
+            .expect("failed to load std-lib")
+            .with_library(MidenLib::default())
+            .expect("failed to load miden-lib")
+            .with_library(kernel_library)
+            .expect("failed to load kernel library (/lib)")
+    }
+}

miden-tx/src/executor/mast_store.rs

@@ -49,7 +49,7 @@ impl TransactionMastStore {
         }
     }
 
-    fn insert(&self, mast_forest: Arc<MastForest>) {
+    pub fn insert(&self, mast_forest: Arc<MastForest>) {
         let mut mast_forests = self.mast_forests.borrow_mut();
 
         // only register procedures that are local to this forest

miden-tx/src/lib.rs

@@ -31,9 +31,8 @@ pub mod auth;
 #[cfg(feature = "testing")]
 pub mod testing;
 
-// TODO: re-enable
-//#[cfg(test)]
-//mod tests;
+#[cfg(test)]
+mod tests;
 
 // RE-EXPORTS
 // ================================================================================================

miden-tx/src/testing/executor.rs

@@ -1,4 +1,3 @@
-#[cfg(not(target_family = "wasm"))]
 use miden_lib::transaction::TransactionKernel;
 #[cfg(feature = "std")]
 use vm_processor::{
@@ -37,14 +36,8 @@ impl<H: Host> CodeExecutor<H> {
     }
 
     /// Compiles and runs the desired code in the host and returns the [Process] state
-    ///
-    /// If a module file path was set, its contents will be inserted between `self.imports` and
-    /// `code` before execution.
-    /// Otherwise, `self.imports` and `code` will be concatenated and the result will be executed.
     pub fn run(self, code: &str) -> Result<Process<H>, ExecutionError> {
-        let assembler = TransactionKernel::assembler().with_debug_mode(true);
-
-        let program = assembler.assemble_program(code).unwrap();
+        let program = TransactionKernel::assembler_testing().assemble_program(code).unwrap();
         self.execute_program(program)
     }
 
@@ -65,7 +58,9 @@ where
     A: AdviceProvider,
 {
     pub fn with_advice_provider(adv_provider: A) -> Self {
-        let host = DefaultHost::new(adv_provider);
+        let mut host = DefaultHost::new(adv_provider);
+        let test_lib = TransactionKernel::kernel_as_library();
+        host.load_mast_forest(test_lib.mast_forest().clone());
         CodeExecutor::new(host)
     }
 }