Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Upgrades] Module upgrades and core param changes via authz txs #702

Merged
merged 20 commits into from
Aug 2, 2024

Conversation

okdas
Copy link
Member

@okdas okdas commented Jul 23, 2024

Summary

Introduces protocol upgrades (consensus-breaking and not) and documents the upgrade process.

Issue

Type of change

Select one or more:

  • New feature, functionality or library
  • Bug fix
  • Code health or cleanup
  • Documentation
  • Other (specify)

Testing

Documentation changes (only if making doc changes)

  • make docusaurus_start; only needed if you make doc changes

Local Testing (only if making code changes)

  • Unit Tests: make go_develop_and_test
  • LocalNet E2E Tests: make test_e2e
  • See quickstart guide for instructions

PR Testing (only if making code changes)

  • DevNet E2E Tests: Add the devnet-test-e2e label to the PR.
    • THIS IS VERY EXPENSIVE, so only do it after all the reviews are complete.
    • Optionally run make trigger_ci if you want to re-trigger tests without any code changes
    • If tests fail, try re-running failed tests only using the GitHub UI as shown here

Sanity Checklist

  • I have tested my changes using the available tooling
  • I have commented my code
  • I have performed a self-review of my own code; both comments & source code
  • I create and reference any new tickets, if applicable
  • I have left TODOs throughout the codebase, if applicable

Summary by CodeRabbit

  • New Features

    • Introduced a centralized Keepers structure for improved management of application components.
    • Implemented a structured mechanism for handling protocol upgrades.
    • Added scripts to automate upgrade transactions and block size modifications.
    • Enhanced API functionality with new proof and session management structures.
  • Documentation

    • Launched new documentation outlining protocol upgrades for the Pocket Network.
  • Bug Fixes / Improvements

    • Refactored access patterns for various keepers, enhancing clarity and maintainability.
  • Chores

    • Created new scripts to facilitate the submission of upgrade transactions and block size updates.

@okdas okdas added protocol General core protocol related changes governance Governance related changes on-chain On-chain business logic tooling Tooling - CLI, scripts, helpers, off-chain, etc... labels Jul 23, 2024
@okdas okdas added this to the Shannon Beta TestNet Launch milestone Jul 23, 2024
@okdas okdas self-assigned this Jul 23, 2024
"grantee": "pokt1eeeksh2tvkh7wzmfrljnhw4wrhs55lcuvmekkw",
"authorization": {
"@type": "\/cosmos.authz.v1beta1.GenericAuthorization",
"msg": "\/cosmos.upgrade.v1beta1.MsgSoftwareUpgrade"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

THE magic line

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way we can leave a comment somehow or call it out? I know it's JSON so curious what ideas you have.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Olshansk with more grants introduced in #717 I think we need a separate document describing the available parameters and transactions.. There are too many. Separate small ticket?

Also, I've noticed some of our own parameters were removed but traces (tests, makefile target, etc) still present (for example min_relay_difficulty_bits - I assume there could be others). It can be tackled on the same ticket.

@okdas okdas changed the title [Upgrades] Module upgrades via authz txs [Upgrades] Module and param upgrades via authz txs Jul 24, 2024
@okdas okdas requested a review from Olshansk July 25, 2024 01:46
@okdas okdas marked this pull request as ready for review July 25, 2024 01:46
Copy link

coderabbitai bot commented Jul 25, 2024

Walkthrough

The recent updates introduce a significant restructuring of the application’s codebase by consolidating various keeper components into a unified Keepers struct. This enhancement improves code organization and maintainability, facilitating easier access to keeper functionalities across the application. Additionally, new scripts and documentation files have been added to support blockchain upgrades and parameter modifications, streamlining operational processes and governance within the Pocket Network.

Changes

Files Change Summary
app/app.go, app/export.go, app/ibc.go Refactored keeper management by consolidating individual keeper fields into a single Keepers struct, enhancing code organization and maintainability. Updated access patterns throughout.
app/keepers/types.go Introduced a new Keepers struct to centralize keeper instances, improving modularity and code structure.
app/upgrades.go, app/upgrades/historical.go Implemented upgrade management functionality, including upgrade handlers and a structure for defining upgrades and their parameters.
docs/static/openapi.yml Significant modifications to the OpenAPI spec, including removal of outdated endpoints and introduction of new structures for managing proof and session parameters.
tools/scripts/upgrades/cosmovisor-start-node.sh, increase-block-size.sh Introduced scripts for managing consensus parameter updates and simulating network upgrades, streamlining operations within the blockchain environment.
tools/scripts/params/*.json, tools/scripts/authz/*.json Configured JSON files for updating consensus parameters and software upgrade messages, facilitating smoother upgrade processes.
docusaurus/docs/protocol/upgrades/index.md Updated documentation on protocol upgrades, detailing implementation processes, types of upgrades, and implications for network participants.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@okdas
Copy link
Member Author

okdas commented Jul 25, 2024

@Olshansk can I get a preliminary review, please?

I'm still thinking how/what can we test here. It will probably be a good idea to write a test that uses cosmovisor for each particular upgrade we will have in the future.

And we should be able to test core modules param changes via authz.. Looking into that now.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

Outside diff range, codebase verification and nitpick comments (5)
tools/scripts/upgrades/increase-block-size.sh (1)

12-12: Address the discussion comment.

The discussion comment suggests considering authorization for changes to other core module parameters. This should be addressed to ensure comprehensive parameter management.

docusaurus/docs/protocol/upgrades/upgrade_list.md (1)

23-33: TestNet table is incomplete.

The TestNet table in the upgrade_list.md file is confirmed to be incomplete. The warning present in the file and the absence of a comprehensive list of TestNet upgrades in the repository support this conclusion.

  • The table should be updated to include all recent TestNet upgrades.
Analysis chain

LGTM! Verify the completeness of the TestNet table.

The TestNet section is clear and correctly formatted.

However, ensure that the TestNet table is complete and accurate.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the completeness of the TestNet table.

# Test: Check if there are additional protocol upgrades not listed in the table.
# Expect: No additional upgrades found.

# Assuming the upgrades are listed in a specific file or directory.
if grep -q "v0.0.4" "docusaurus/docs/protocol/upgrades/testnet_upgrades.md"; then
  echo "TestNet table is complete."
else
  echo "TestNet table is incomplete."
fi

Length of output: 224


Script:

#!/bin/bash
# Search for all references to TestNet upgrades in the repository to verify the completeness of the table.
rg -i "TestNet" --context 5

Length of output: 40492

tools/scripts/upgrades/start-node.sh (1)

1-12: Paths and directories do not exist.

The directories /Users/dk/pocket/pokrtoll-for-releases and /Users/dk/pocket/poktroll do not exist. Please verify or update the paths and directories in the script to ensure they are correct.

  • /Users/dk/pocket/pokrtoll-for-releases
  • /Users/dk/pocket/poktroll
Analysis chain

LGTM! Verify the paths and configurations.

The header and environment variables are clear and correctly formatted.

However, ensure that the paths and configurations are correct.


To further investigate the issue with the paths, we need to check if the directories exist and list their contents. This will help us determine if there are any files that might indicate a typo or a different file name.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the paths and configurations.

# Test: Check if the paths to the binaries exist.
# Expect: All paths should exist.

if [ -f "/Users/dk/pocket/pokrtoll-for-releases/poktrolld" ] && [ -f "/Users/dk/pocket/poktroll/poktrolld" ]; then
  echo "Paths are correct."
else
  echo "Paths are incorrect."
fi

Length of output: 112


Script:

#!/bin/bash
# Description: Verify if the directories exist and list their contents.

# Check if the directories exist
if [ -d "/Users/dk/pocket/pokrtoll-for-releases" ]; then
  echo "Directory /Users/dk/pocket/pokrtoll-for-releases exists."
  echo "Contents of /Users/dk/pocket/pokrtoll-for-releases:"
  ls -l /Users/dk/pocket/pokrtoll-for-releases
else
  echo "Directory /Users/dk/pocket/pokrtoll-for-releases does not exist."
fi

if [ -d "/Users/dk/pocket/poktroll" ]; then
  echo "Directory /Users/dk/pocket/poktroll exists."
  echo "Contents of /Users/dk/pocket/poktroll:"
  ls -l /Users/dk/pocket/poktroll
else
  echo "Directory /Users/dk/pocket/poktroll does not exist."
fi

Length of output: 342

docusaurus/docs/operate/run_a_node/appgate_server.md (1)

28-28: Approved: Consistent reference update.

The tip section has been correctly updated to reference full_node_docker.md. Consider using "before" instead of "prior to" for conciseness.

- prior to setting up a RelayMiner.
+ before setting up a RelayMiner.
Tools
LanguageTool

[style] ~28-~28: ‘prior to’ might be wordy. Consider a shorter alternative.
Context: ...ploy a Full Node prior to setting up a RelayMiner. This ensures t...

(EN_WORDINESS_PREMIUM_PRIOR_TO)

docusaurus/docs/protocol/upgrades/upgrade_procedure.md (1)

38-61: Address minor style issues.

There are some minor style issues flagged by static analysis tools:

  1. The phrase "specific details" is redundant. Consider writing "details".
  2. Use a comma before "and" if it connects two independent clauses.
  3. The verb "does not support" should agree with the subject "DevNets".
- with specific details about the upgrade.
+ with details about the upgrade.

- The height at which an upgrade should be executed and the node will be restarted.
+ The height at which an upgrade should be executed, and the node will be restarted.

- DevNets currently do not support `cosmovisor`.
+ DevNets currently does not support `cosmovisor`.
Tools
LanguageTool

[style] ~38-~38: This phrase is redundant. Consider writing “details”.
Context: ...upgrade/v1beta1/upgrade.proto#L14) with specific details about the upgrade. This information hel...

(SPECIFIC_DETAILS)


[uncategorized] ~59-~59: Use a comma before “and” if it connects two independent clauses (unless they are closely connected and short).
Context: ...t at which an upgrade should be executed and the node will be restarted. info: W...

(COMMA_COMPOUND_SENTENCE_2)

Gitleaks

46-46: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

tools/scripts/params/consensus_increase_block_size.json Outdated Show resolved Hide resolved
tools/scripts/upgrades/submit-upgrade.sh Outdated Show resolved Hide resolved
tools/scripts/upgrades/increase-block-size.sh Show resolved Hide resolved
tools/scripts/upgrades/submit-upgrade.sh Outdated Show resolved Hide resolved
docusaurus/docs/operate/run_a_node/full_node_cosmovisor.md Outdated Show resolved Hide resolved
app/upgrades.go Outdated Show resolved Hide resolved
app/upgrades/historical.go Outdated Show resolved Hide resolved
@okdas okdas changed the title [Upgrades] Module and param upgrades via authz txs [Upgrades] Module upgrades and core param changes via authz txs Jul 30, 2024
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (3)
docusaurus/docs/protocol/upgrades/index.md (3)

8-14: Fix repeated verb in the table of contents.

There's a repeated verb in the table of contents.

-  - [Planned vs. Unplanned Upgrades](#planned-vs-unplanned-upgrades)
-  - [Breaking vs. Non-breaking Upgrades](#breaking-vs-non-breaking-upgrades)
-  - [Manual Interventions](#manual-interventions)
+ - [Planned vs. Unplanned Upgrades](#planned-vs-unplanned-upgrades)
+ - [Breaking vs. Non-breaking Upgrades](#breaking-vs-non-breaking-upgrades)
+ - [Manual Interventions](#manual-interventions)
Tools
LanguageTool

[grammar] ~10-~10: You’ve repeated a verb. Did you mean to only write one of them?
Context: ...#what-is-a-protocol-upgrade) - List of Upgrades - Upgrade Types - [Planned vs....

(REPEATED_VERBS)


17-19: Consider adding more details about the DAO process and consensus-breaking changes.

The section is clear but could benefit from more details about the DAO process and consensus-breaking changes to provide a better understanding.

- The Pocket Network is continuously evolving through regular protocol upgrades. We implement software upgrades via a DAO process, allowing validator nodes to incorporate consensus-breaking changes. These upgrades can be automatically applied when using [Cosmovisor](../../operate/run_a_node/full_node_cosmovisor.md), or manually if not using `cosmovisor`.
+ The Pocket Network is continuously evolving through regular protocol upgrades. We implement software upgrades via a DAO (Decentralized Autonomous Organization) process, which involves community voting and governance decisions, allowing validator nodes to incorporate consensus-breaking changes. These upgrades can be automatically applied when using [Cosmovisor](../../operate/run_a_node/full_node_cosmovisor.md), or manually if not using `cosmovisor`.

25-27: Consider adding a direct link to the upgrade list.

The section is clear but could benefit from a direct link to the upgrade list in the documentation.

- While you can find a comprehensive list of [poktroll releases](https://github.com/pokt-network/poktroll/releases) on our GitHub, we also maintain a [list of upgrades](./upgrade_list.md) in our documentation. This list provides valuable information, including whether an upgrade involves breaking changes and if manual intervention is required from operators.
+ While you can find a comprehensive list of [poktroll releases](https://github.com/pokt-network/poktroll/releases) on our GitHub, we also maintain a [list of upgrades](./upgrade_list.md) in our documentation. This list provides valuable information, including whether an upgrade involves breaking changes and if manual intervention is required from operators. You can directly access the upgrade list [here](./upgrade_list.md).

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Outside diff range, codebase verification and nitpick comments (3)
tools/scripts/upgrades/cosmovisor-start-node.sh (1)

2-5: Improve script documentation.

The comments provide useful information, but consider adding more details about the purpose and usage of the script.

docusaurus/docs/protocol/upgrades/upgrade_procedure.md (2)

49-49: Refine the phrase for conciseness.

The phrase "specific details" is redundant. Consider writing "details".

- with specific details about the upgrade.
+ with details about the upgrade.
Tools
LanguageTool

[style] ~49-~49: This phrase is redundant. Consider writing “details”.
Context: ...upgrade/v1beta1/upgrade.proto#L14) with specific details about the upgrade. This information hel...

(SPECIFIC_DETAILS)


70-70: Use a comma before "and" in compound sentences.

Use a comma before "and" if it connects two independent clauses.

- The height at which an upgrade should be executed and the node will be restarted.
+ The height at which an upgrade should be executed, and the node will be restarted.
Tools
LanguageTool

[uncategorized] ~70-~70: Use a comma before “and” if it connects two independent clauses (unless they are closely connected and short).
Context: ...t at which an upgrade should be executed and the node will be restarted. info: W...

(COMMA_COMPOUND_SENTENCE_2)

"messages": [
{
"@type": "/cosmos.upgrade.v1beta1.MsgSoftwareUpgrade",
"authority": "pokt10d07y265gmmuvt4z0w9aw880jnsr700j8yv32t",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential security issue: Detected a Generic API Key.

The value "pokt10d07y265gmmuvt4z0w9aw880jnsr700j8yv32t" appears to be a generic API key. Ensure this is not exposing sensitive information.

Tools
Gitleaks

57-57: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

tools/scripts/upgrades/cosmovisor-start-node.sh Outdated Show resolved Hide resolved
tools/scripts/upgrades/cosmovisor-start-node.sh Outdated Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Comment on lines +27174 to +27351
name:
type: string
description: (Optional) Semantic human readable name for the service
title: >-
TODO_BETA: Name is currently unused but acts as a reminder
that an optional onchain representation of the service is
necessary
session_id:
type: string
description: A unique pseudoranom ID for this session
title: >-
NOTE: session_id can be derived from the above values using
on-chain but is included in the header for convenience
session_start_block_height:
type: string
format: int64
title: The height at which this session started
session_end_block_height:
type: string
format: int64
description: >-
Note that`session_end_block_height` is a derivative of (`start` +
`num_blocks_per_session`)

as goverened by on-chain params at the time of the session start.

It is stored as an additional field to simplofy business logic in
case

the number of blocks_per_session changes during the session.


The height at which this session ended, this is the last block of
the session
description: >-
SessionHeader is a lightweight header for a session that can be passed
around.

It is the minimal amount of data required to hydrate & retrieve all
data relevant to the session.
closest_merkle_proof:
type: string
format: byte
description: The serialized SMST proof from the `#ClosestProof()` method.
poktroll.session.SessionHeader:
type: object
properties:
application_address:
type: string
title: >-
The Bech32 address of the application using cosmos' ScalarDescriptor
to ensure deterministic encoding
service:
title: The service this session is for
type: object
properties:
id:
type: string
description: Unique identifier for the service
title: >-
For example, what if we want to request a session for a certain
service but with some additional configs that identify it?
name:
type: string
description: (Optional) Semantic human readable name for the service
title: >-
TODO_BETA: Name is currently unused but acts as a reminder that an
optional onchain representation of the service is necessary
session_id:
type: string
description: A unique pseudoranom ID for this session
title: >-
NOTE: session_id can be derived from the above values using on-chain
but is included in the header for convenience
session_start_block_height:
type: string
format: int64
title: The height at which this session started
session_end_block_height:
type: string
format: int64
description: >-
Note that`session_end_block_height` is a derivative of (`start` +
`num_blocks_per_session`)

as goverened by on-chain params at the time of the session start.

It is stored as an additional field to simplofy business logic in case

the number of blocks_per_session changes during the session.


The height at which this session ended, this is the last block of the
session
description: >-
SessionHeader is a lightweight header for a session that can be passed
around.

It is the minimal amount of data required to hydrate & retrieve all data
relevant to the session.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix duplication errors in object definitions.

There are duplication errors in the object definitions for poktroll.proof.Params, poktroll.proof.Proof, and poktroll.session.SessionHeader.

-  poktroll.proof.Params:
-    type: object
-    properties:
-      min_relay_difficulty_bits:
-        type: string
-        format: uint64
-        description: >-
-          min_relay_difficulty_bits is the minimum difficulty in bits for a
-          relay to
-          be included in a Merkle proof.
-      proof_request_probability:
-        type: number
-        format: float
-        description: >-
-          proof_request_probability is the probability of a session requiring a
-          proof
-          if it's cost (i.e. compute unit consumption) is below the
-          ProofRequirementThreshold.
-      proof_requirement_threshold:
-        type: string
-        format: uint64
-        description: >-
-          proof_requirement_threshold is the session cost (i.e. compute unit
-          consumption)
-          threshold which asserts that a session MUST have a corresponding proof
-          when its cost
-          is equal to or above the threshold. This is in contrast to the this
-          requirement
-          being determined probabilistically via ProofRequestProbability.
-      proof_missing_penalty:
-        description: >-
-          proof_missing_penalty is the number of tokens (uPOKT) which should be
-          slashed from a supplier
-          when a proof is required (either via proof_requirement_threshold or
-          proof_missing_penalty)
-          but is not provided.
-        type: object
-        properties:
-          denom:
-            type: string
-          amount:
-            type: string
-    description: Params defines the parameters for the module.
-  poktroll.proof.Proof:
-    type: object
-    properties:
-      supplier_address:
-        type: string
-      session_header:
-        type: object
-        properties:
-          application_address:
-            type: string
-            title: >-
-              The Bech32 address of the application using cosmos'
-              ScalarDescriptor to ensure deterministic encoding
-          service:
-            title: The service this session is for
-            type: object
-            properties:
-              id:
-                type: string
-                description: Unique identifier for the service
-                title: >-
-                  For example, what if we want to request a session for a
-                  certain service but with some additional configs that identify
-                  it?
-              name:
-                type: string
-                description: (Optional) Semantic human readable name for the service
-                title: >-
-                  TODO_BETA: Name is currently unused but acts as a reminder
-                  that an optional onchain representation of the service is
-                  necessary
-          session_id:
-            type: string
-            description: A unique pseudoranom ID for this session
-            title: >-
-              NOTE: session_id can be derived from the above values using
-              on-chain but is included in the header for convenience
-          session_start_block_height:
-            type: string
-            format: int64
-            title: The height at which this session started
-          session_end_block_height:
-            type: string
-            format: int64
-            description: >-
-              Note that`session_end_block_height` is a derivative of (`start` +
-              `num_blocks_per_session`)
-              as goverened by on-chain params at the time of the session start.
-              It is stored as an additional field to simplofy business logic in
-              case
-              the number of blocks_per_session changes during the session.
-          description: >-
-            SessionHeader is a lightweight header for a session that can be passed
-            around.
-            It is the minimal amount of data required to hydrate & retrieve all
-            data relevant to the session.
-      closest_merkle_proof:
-        type: string
-        format: byte
-        description: The serialized SMST proof from the `#ClosestProof()` method.
-  poktroll.session.SessionHeader:
-    type: object
-    properties:
-      application_address:
-        type: string
-        title: >-
-          The Bech32 address of the application using cosmos' ScalarDescriptor
-          to ensure deterministic encoding
-      service:
-        title: The service this session is for
-        type: object
-        properties:
-          id:
-            type: string
-            description: Unique identifier for the service
-            title: >-
-              For example, what if we want to request a session for a certain
-              service but with some additional configs that identify it?
-          name:
-            type: string
-            description: (Optional) Semantic human readable name for the service
-            title: >-
-              TODO_BETA: Name is currently unused but acts as a reminder that an
-              optional onchain representation of the service is necessary
-      session_id:
-        type: string
-        description: A unique pseudoranom ID for this session
-        title: >-
-          NOTE: session_id can be derived from the above values using on-chain
-          but is included in the header for convenience
-      session_start_block_height:
-        type: string
-        format: int64
-        title: The height at which this session started
-      session_end_block_height:
-        type: string
-        format: int64
-        description: >-
-          Note that`session_end_block_height` is a derivative of (`start` +
-          `num_blocks_per_session`)
-          as goverened by on-chain params at the time of the session start.
-          It is stored as an additional field to simplofy business logic in case
-          the number of blocks_per_session changes during the session.
-      description: >-
-        SessionHeader is a lightweight header for a session that can be passed
-        around.
-        It is the minimal amount of data required to hydrate & retrieve all data
-        relevant to the session.
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
poktroll.proof.Params:
type: object
properties:
min_relay_difficulty_bits:
type: string
format: uint64
description: >-
min_relay_difficulty_bits is the minimum difficulty in bits for a
relay to
be included in a Merkle proof.
proof_request_probability:
type: number
format: float
description: >-
proof_request_probability is the probability of a session requiring a
proof
if it's cost (i.e. compute unit consumption) is below the
ProofRequirementThreshold.
proof_requirement_threshold:
type: string
format: uint64
description: >-
proof_requirement_threshold is the session cost (i.e. compute unit
consumption)
threshold which asserts that a session MUST have a corresponding proof
when its cost
is equal to or above the threshold. This is in contrast to the this
requirement
being determined probabilistically via ProofRequestProbability.
TODO_MAINNET: Consider renaming this to
`proof_requirement_threshold_compute_units`.
proof_missing_penalty:
description: >-
proof_missing_penalty is the number of tokens (uPOKT) which should be
slashed from a supplier
when a proof is required (either via proof_requirement_threshold or
proof_missing_penalty)
but is not provided.
type: object
properties:
denom:
type: string
amount:
type: string
description: Params defines the parameters for the module.
poktroll.proof.Proof:
type: object
properties:
supplier_address:
type: string
session_header:
type: object
properties:
application_address:
type: string
title: >-
The Bech32 address of the application using cosmos'
ScalarDescriptor to ensure deterministic encoding
service:
title: The service this session is for
type: object
properties:
id:
type: string
description: Unique identifier for the service
title: >-
For example, what if we want to request a session for a
certain service but with some additional configs that identify
it?
name:
type: string
description: (Optional) Semantic human readable name for the service
title: >-
TODO_BETA: Name is currently unused but acts as a reminder
that an optional onchain representation of the service is
necessary
session_id:
type: string
description: A unique pseudoranom ID for this session
title: >-
NOTE: session_id can be derived from the above values using
on-chain but is included in the header for convenience
session_start_block_height:
type: string
format: int64
title: The height at which this session started
session_end_block_height:
type: string
format: int64
description: >-
Note that`session_end_block_height` is a derivative of (`start` +
`num_blocks_per_session`)
as goverened by on-chain params at the time of the session start.
It is stored as an additional field to simplofy business logic in
case
the number of blocks_per_session changes during the session.
The height at which this session ended, this is the last block of
the session
description: >-
SessionHeader is a lightweight header for a session that can be passed
around.
It is the minimal amount of data required to hydrate & retrieve all
data relevant to the session.
closest_merkle_proof:
type: string
format: byte
description: The serialized SMST proof from the `#ClosestProof()` method.
poktroll.session.SessionHeader:
type: object
properties:
application_address:
type: string
title: >-
The Bech32 address of the application using cosmos' ScalarDescriptor
to ensure deterministic encoding
service:
title: The service this session is for
type: object
properties:
id:
type: string
description: Unique identifier for the service
title: >-
For example, what if we want to request a session for a certain
service but with some additional configs that identify it?
name:
type: string
description: (Optional) Semantic human readable name for the service
title: >-
TODO_BETA: Name is currently unused but acts as a reminder that an
optional onchain representation of the service is necessary
session_id:
type: string
description: A unique pseudoranom ID for this session
title: >-
NOTE: session_id can be derived from the above values using on-chain
but is included in the header for convenience
session_start_block_height:
type: string
format: int64
title: The height at which this session started
session_end_block_height:
type: string
format: int64
description: >-
Note that`session_end_block_height` is a derivative of (`start` +
`num_blocks_per_session`)
as goverened by on-chain params at the time of the session start.
It is stored as an additional field to simplofy business logic in case
the number of blocks_per_session changes during the session.
The height at which this session ended, this is the last block of the
session
description: >-
SessionHeader is a lightweight header for a session that can be passed
around.
It is the minimal amount of data required to hydrate & retrieve all data
relevant to the session.
Tools
yamllint

[error] 27174-27174: duplication of key "poktroll.proof.Params" in mapping

(key-duplicates)


[error] 27228-27228: duplication of key "poktroll.proof.Proof" in mapping

(key-duplicates)


[error] 27296-27296: duplication of key "poktroll.session.SessionHeader" in mapping

(key-duplicates)

@okdas okdas requested a review from Olshansk August 2, 2024 02:01
app/upgrades.go Outdated Show resolved Hide resolved
app/upgrades.go Outdated Show resolved Hide resolved
ProofKeeper proofmodulekeeper.Keeper
TokenomicsKeeper tokenomicsmodulekeeper.Keeper
SharedKeeper sharedmodulekeeper.Keeper
// this line is used by starport scaffolding # stargate/app/keeperDeclaration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Point #2 from this conversation: #702 (comment)

Need to add the line back and add a comment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes - I was sure I already moved the line. Thanks for catching this.

app/upgrades.go Outdated
"github.com/pokt-network/poktroll/app/upgrades"
)

// allUpgrades includes all upgrades that have been created, but not necessarily submitted on-chain
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub releases might not always be listed in this (allUpgrades) slice.

#PUC why?

Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few small comments. Should be good to merge in tomorrow.

@@ -0,0 +1,80 @@
package keepers

import (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per comment #1 in #702 (comment). Can you please:

  1. Explain why we have diverged from cosmos standards
  2. Link to this PR in the comment for reference
  3. Mention ignite might stop working.

Copy link
Member Author

@okdas okdas Aug 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Done.
  2. Done.
  3. Ignite scaffolding already needs manual adjustments due to our proto/package structure. This change doesn't break functionality; we just need to move the scaffolded keeper to a separate package.

tools/scripts/upgrades/increase-block-size.sh Show resolved Hide resolved
tools/scripts/upgrades/increase-block-size.sh Outdated Show resolved Hide resolved
tools/scripts/upgrades/increase-block-size.sh Outdated Show resolved Hide resolved
app/upgrades/historical.go Outdated Show resolved Hide resolved
@@ -116,6 +116,78 @@
},
"expiration": "2500-01-01T00:00:00Z"
},
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? (Given the other PR that went in)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No and I don't see it in GitHub changes. Do you?
Screenshot 2024-08-02 at 11 58 14 AM

tools/scripts/upgrades/cosmovisor-start-node.sh Outdated Show resolved Hide resolved
@okdas okdas requested a review from Olshansk August 2, 2024 20:21

echo -e "\nPre-requisites:"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: Is it not easier to just do 1 echo statement?

https://stackoverflow.com/a/10969990/768439

Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one last comment (PTAL) but LGTM otherwise!

@okdas okdas mentioned this pull request Aug 2, 2024
14 tasks
Copy link

github-actions bot commented Aug 2, 2024

The CI will now also run the e2e tests on devnet, which increases the time it takes to complete all CI checks.

You may need to run make trigger_ci to submit an empty commit that'll trigger the tests.

GCP workloads (requires changing the namespace to 702)
Grafana network dashboard for devnet-issue-{issue-id}

@github-actions github-actions bot added devnet push-image CI related - pushes images to ghcr.io labels Aug 2, 2024
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

@@ -0,0 +1,54 @@
#!/bin/bash
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add error handling for the shebang line.

Consider adding set -e to exit the script if any command fails.

#!/bin/bash
+ set -e
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
#!/bin/bash
#!/bin/bash
set -e

Comment on lines +17 to +18
POKTROLLD_OLD_BINARY_PATH=$HOME/pocket/pokrtoll-for-releases/poktrolld
POKTROLLD_NEW_BINARY_PATH=$HOME/pocket/poktroll/poktrolld
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use absolute paths for binaries.

Using absolute paths for binaries can prevent issues related to relative paths.

export POKTROLLD_OLD_BINARY_PATH=$HOME/pocket/pokrtoll-for-releases/poktrolld
export POKTROLLD_NEW_BINARY_PATH=$HOME/pocket/poktroll/poktrolld
+ export POKTROLLD_OLD_BINARY_PATH=/absolute/path/to/pokrtoll-for-releases/poktrolld
+ export POKTROLLD_NEW_BINARY_PATH=/absolute/path/to/poktroll/poktrolld

Committable suggestion was skipped due to low confidence.


read -p "Do you want to continue? (y/n): " answer
if [[ $answer != "y" ]]; then
echo "Script execution cancelled."
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix misspelling in user confirmation step.

"Cancelled" is a misspelling of "canceled".

echo "Script execution cancelled."
+ echo "Script execution canceled."
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
echo "Script execution cancelled."
echo "Script execution canceled."
Tools
GitHub Check: misspell

[warning] 32-32: [misspell] tools/scripts/upgrades/cosmovisor-start-node.sh#L32
"cancelled" is a misspelling of "canceled"

export DAEMON_HOME=$HOME/.poktroll # `localnet_regenesis` creates new genesis in this directory by default.

# Cleans up old upgrade binary and home dir.
rm -rf $DAEMON_HOME
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add error handling for directory removal.

Consider adding error handling for the rm -rf command to ensure the directory is removed successfully.

rm -rf $DAEMON_HOME
+ if [ $? -ne 0 ]; then
+   echo "Failed to remove $DAEMON_HOME"
+   exit 1
+ fi
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
rm -rf $DAEMON_HOME
rm -rf $DAEMON_HOME
if [ $? -ne 0 ]; then
echo "Failed to remove $DAEMON_HOME"
exit 1
fi

rm -rf $DAEMON_HOME

# Runs regenesis.
make localnet_regenesis
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add error handling for the make command.

Consider adding error handling for the make localnet_regenesis command to ensure it completes successfully.

make localnet_regenesis
+ if [ $? -ne 0 ]; then
+   echo "Failed to run localnet_regenesis"
+   exit 1
+ fi
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
make localnet_regenesis
make localnet_regenesis
if [ $? -ne 0 ]; then
echo "Failed to run localnet_regenesis"
exit 1
fi

Comment on lines +50 to +52
mkdir -p $DAEMON_HOME/cosmovisor/genesis/bin/ $DAEMON_HOME/cosmovisor/upgrades/$POKTROLLD_UPGRADE_PLAN_NAME/bin/
cp -r $POKTROLLD_OLD_BINARY_PATH $DAEMON_HOME/cosmovisor/genesis/bin/poktrolld
cp -r $POKTROLLD_NEW_BINARY_PATH $DAEMON_HOME/cosmovisor/upgrades/$POKTROLLD_UPGRADE_PLAN_NAME/bin/poktrolld
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add error handling for directory creation and file copying.

Consider adding error handling for the mkdir and cp commands to ensure they complete successfully.

mkdir -p $DAEMON_HOME/cosmovisor/genesis/bin/ $DAEMON_HOME/cosmovisor/upgrades/$POKTROLLD_UPGRADE_PLAN_NAME/bin/
+ if [ $? -ne 0 ]; then
+   echo "Failed to create directories"
+   exit 1
+ fi
cp -r $POKTROLLD_OLD_BINARY_PATH $DAEMON_HOME/cosmovisor/genesis/bin/poktrolld
+ if [ $? -ne 0 ]; then
+   echo "Failed to copy old binary"
+   exit 1
+ fi
cp -r $POKTROLLD_NEW_BINARY_PATH $DAEMON_HOME/cosmovisor/upgrades/$POKTROLLD_UPGRADE_PLAN_NAME/bin/poktrolld
+ if [ $? -ne 0 ]; then
+   echo "Failed to copy new binary"
+   exit 1
+ fi
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
mkdir -p $DAEMON_HOME/cosmovisor/genesis/bin/ $DAEMON_HOME/cosmovisor/upgrades/$POKTROLLD_UPGRADE_PLAN_NAME/bin/
cp -r $POKTROLLD_OLD_BINARY_PATH $DAEMON_HOME/cosmovisor/genesis/bin/poktrolld
cp -r $POKTROLLD_NEW_BINARY_PATH $DAEMON_HOME/cosmovisor/upgrades/$POKTROLLD_UPGRADE_PLAN_NAME/bin/poktrolld
mkdir -p $DAEMON_HOME/cosmovisor/genesis/bin/ $DAEMON_HOME/cosmovisor/upgrades/$POKTROLLD_UPGRADE_PLAN_NAME/bin/
if [ $? -ne 0 ]; then
echo "Failed to create directories"
exit 1
fi
cp -r $POKTROLLD_OLD_BINARY_PATH $DAEMON_HOME/cosmovisor/genesis/bin/poktrolld
if [ $? -ne 0 ]; then
echo "Failed to copy old binary"
exit 1
fi
cp -r $POKTROLLD_NEW_BINARY_PATH $DAEMON_HOME/cosmovisor/upgrades/$POKTROLLD_UPGRADE_PLAN_NAME/bin/poktrolld
if [ $? -ne 0 ]; then
echo "Failed to copy new binary"
exit 1
fi

Copy link

github-actions bot commented Aug 2, 2024

The CI will now also run the e2e tests on devnet, which increases the time it takes to complete all CI checks.

You may need to run make trigger_ci to submit an empty commit that'll trigger the tests.

GCP workloads (requires changing the namespace to 702)
Grafana network dashboard for devnet-issue-702

@okdas
Copy link
Member Author

okdas commented Aug 2, 2024

While tests are failing now, they were green on the previous commit. No changes to the go code in the last commit.
E2E tests are also green.

@okdas okdas merged commit 08ce5b6 into main Aug 2, 2024
11 of 12 checks passed
okdas added a commit that referenced this pull request Aug 19, 2024
## Summary

Follow-up PR to #702 to document protocol upgrades procedure.

## Issue

- #702 
- #625 

## Type of change

Select one or more:

- [ ] New feature, functionality or library
- [ ] Bug fix
- [ ] Code health or cleanup
- [x] Documentation
- [ ] Other (specify)

## Testing

**Documentation changes** (only if making doc changes)
- [ ] `make docusaurus_start`; only needed if you make doc changes

**Local Testing** (only if making code changes)
- [ ] **Unit Tests**: `make go_develop_and_test`
- [ ] **LocalNet E2E Tests**: `make test_e2e`
- See [quickstart
guide](https://dev.poktroll.com/developer_guide/quickstart) for
instructions

**PR Testing** (only if making code changes)
- [ ] **DevNet E2E Tests**: Add the `devnet-test-e2e` label to the PR.
- **THIS IS VERY EXPENSIVE**, so only do it after all the reviews are
complete.
- Optionally run `make trigger_ci` if you want to re-trigger tests
without any code changes
- If tests fail, try re-running failed tests only using the GitHub UI as
shown
[here](https://github.com/pokt-network/poktroll/assets/1892194/607984e9-0615-4569-9452-4c730190c1d2)


## Sanity Checklist

- [ ] I have tested my changes using the available tooling
- [ ] I have commented my code
- [ ] I have performed a self-review of my own code; both comments &
source code
- [ ] I create and reference any new tickets, if applicable
- [ ] I have left TODOs throughout the codebase, if applicable


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit


- **New Features**
- Introduced a new category for "Upgrades" in the documentation to
enhance navigation related to protocol upgrades.
- Launched comprehensive documentation on the governance mechanisms for
protocol parameter adjustments, detailing the DAO's role and examples of
changes.
- Added a structured page for past and upcoming protocol upgrades,
improving user understanding of upgrade statuses.
- Released a document outlining the protocol upgrade process, specifying
steps and criteria for implementation.

- **Documentation Updates**
- Improved organization and clarity throughout the documentation by
refining content structure and descriptions related to protocol
upgrades.
- Updated sidebar positions and references to ensure accurate navigation
and resource linking for "Full Node" documentation.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Daniel Olshansky <[email protected]>
okdas added a commit that referenced this pull request Nov 14, 2024
okdas added a commit that referenced this pull request Nov 14, 2024
## Summary

Follow-up PR to #702 to document protocol upgrades procedure.

## Issue

- #702 
- #625 

## Type of change

Select one or more:

- [ ] New feature, functionality or library
- [ ] Bug fix
- [ ] Code health or cleanup
- [x] Documentation
- [ ] Other (specify)

## Testing

**Documentation changes** (only if making doc changes)
- [ ] `make docusaurus_start`; only needed if you make doc changes

**Local Testing** (only if making code changes)
- [ ] **Unit Tests**: `make go_develop_and_test`
- [ ] **LocalNet E2E Tests**: `make test_e2e`
- See [quickstart
guide](https://dev.poktroll.com/developer_guide/quickstart) for
instructions

**PR Testing** (only if making code changes)
- [ ] **DevNet E2E Tests**: Add the `devnet-test-e2e` label to the PR.
- **THIS IS VERY EXPENSIVE**, so only do it after all the reviews are
complete.
- Optionally run `make trigger_ci` if you want to re-trigger tests
without any code changes
- If tests fail, try re-running failed tests only using the GitHub UI as
shown
[here](https://github.com/pokt-network/poktroll/assets/1892194/607984e9-0615-4569-9452-4c730190c1d2)


## Sanity Checklist

- [ ] I have tested my changes using the available tooling
- [ ] I have commented my code
- [ ] I have performed a self-review of my own code; both comments &
source code
- [ ] I create and reference any new tickets, if applicable
- [ ] I have left TODOs throughout the codebase, if applicable


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit


- **New Features**
- Introduced a new category for "Upgrades" in the documentation to
enhance navigation related to protocol upgrades.
- Launched comprehensive documentation on the governance mechanisms for
protocol parameter adjustments, detailing the DAO's role and examples of
changes.
- Added a structured page for past and upcoming protocol upgrades,
improving user understanding of upgrade statuses.
- Released a document outlining the protocol upgrade process, specifying
steps and criteria for implementation.

- **Documentation Updates**
- Improved organization and clarity throughout the documentation by
refining content structure and descriptions related to protocol
upgrades.
- Updated sidebar positions and references to ensure accurate navigation
and resource linking for "Full Node" documentation.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Daniel Olshansky <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devnet-test-e2e governance Governance related changes on-chain On-chain business logic protocol General core protocol related changes push-image CI related - pushes images to ghcr.io tooling Tooling - CLI, scripts, helpers, off-chain, etc...
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

2 participants