Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Performance] Reduce RelayMiner memory consumption under load #739

Merged
merged 16 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -421,6 +421,12 @@ test_load_relays_stress_localnet: test_e2e_env warn_message_local_stress_test ##
-tags=load,test -run LoadRelays --log-level=debug --timeout=30m \
--manifest ./load-testing/loadtest_manifest_localnet.yaml

.PHONY: test_load_relays_stress_localnet_single_supplier
test_load_relays_stress_localnet_single_supplier: test_e2e_env warn_message_local_stress_test ## Run the stress test for E2E relays on LocalNet using exclusively one supplier.
go test -v -count=1 ./load-testing/tests/... \
-tags=load,test -run TestLoadRelaysSingleSupplier --log-level=debug --timeout=30m \
--manifest ./load-testing/loadtest_manifest_localnet_single_supplier.yaml

.PHONY: test_verbose
test_verbose: check_go_version ## Run all go tests verbosely
go test -count=1 -v -race -tags test ./...
Expand Down
3 changes: 3 additions & 0 deletions api/poktroll/application/params.pulsar.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docusaurus/docs/develop/developer_guide/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -433,7 +433,7 @@ The following is an example config to get you started:
```bash
cat <<EOF >> shannon_relayminer_config.yaml
default_signing_key_names: [ "shannon_supplier" ]
smt_store_path: smt_stores
smt_store_path: $HOME/.poktroll/smt
metrics:
enabled: true
addr: :9999 # you may need to change the metrics server port due to port conflicts.
Expand Down
23 changes: 15 additions & 8 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,18 @@ module github.com/pokt-network/poktroll
go 1.22.2

replace (
// fix upstream GHSA-h395-qcrw-5vmq vulnerability.
okdas marked this conversation as resolved.
Show resolved Hide resolved
github.com/gin-gonic/gin => github.com/gin-gonic/gin v1.7.0
// DEVELOPER_TIP: Uncomment to use a local copy of shannon-sdk for development purposes.
// github.com/pokt-network/shannon-sdk => ../shannon-sdk

// DEVELOPER_TIP: Uncomment to use a local copy of smt for development purposes.
// github.com/pokt-network/smt => ../smt
// github.com/pokt-network/smt/kvstore/badger => ../smt/kvstore/badger
// github.com/pokt-network/smt/kvstore/pebble => ../smt/kvstore/pebble

// fix upstream GHSA-h395-qcrw-5vmq vulnerability.
github.com/gin-gonic/gin => github.com/gin-gonic/gin v1.7.0
// TODO_IN_THIS_PR: bump and remove

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[linter-name (fail-on-found)] reported by reviewdog 🐶
// TODO_IN_THIS_PR: bump and remove

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@okdas Let's push to finish the SMT PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Olshansk requested another review there

github.com/pokt-network/smt => github.com/pokt-network/smt v0.0.0-20240819232427-be8f5d598321

// replace broken goleveldb
github.com/syndtr/goleveldb => github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7
Expand Down Expand Up @@ -57,8 +61,8 @@ require (
// repo is the first obvious idea, but has to be carefully considered, automated, and is not
// a hard blocker.
github.com/pokt-network/shannon-sdk v0.0.0-20240814144717-dfa95b525d46
// TODO_IN_THIS_PR: bump after https://github.com/pokt-network/smt/pull/52 is in

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[linter-name (fail-on-found)] reported by reviewdog 🐶
// TODO_IN_THIS_PR: bump after pokt-network/smt#52 is in

github.com/pokt-network/smt v0.12.0
github.com/pokt-network/smt/kvstore/badger v0.0.0-20240109205447-868237978c0b
github.com/prometheus/client_golang v1.19.0
github.com/regen-network/gocuke v1.1.0
github.com/rs/zerolog v1.32.0
Expand All @@ -79,7 +83,11 @@ require (
gopkg.in/yaml.v2 v2.4.0
)

require golang.org/x/text v0.16.0
require (
// TODO_IN_THIS_PR: bump to the main branch commit after https://github.com/pokt-network/smt/pull/52 is in

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[linter-name (fail-on-found)] reported by reviewdog 🐶
// TODO_IN_THIS_PR: bump to the main branch commit after pokt-network/smt#52 is in

github.com/pokt-network/smt/kvstore/pebble v0.0.0-20240819232427-be8f5d598321
golang.org/x/text v0.16.0
)

require (
buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.34.2-20240508200655-46a4cf4ba109.2 // indirect
Expand Down Expand Up @@ -119,9 +127,10 @@ require (
github.com/chzyer/readline v1.5.1 // indirect
github.com/cockroachdb/apd/v2 v2.0.2 // indirect
github.com/cockroachdb/apd/v3 v3.2.1 // indirect
github.com/cockroachdb/errors v1.11.1 // indirect
github.com/cockroachdb/errors v1.11.3 // indirect
github.com/cockroachdb/fifo v0.0.0-20240606204812-0bbfbd93a7ce // indirect
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b // indirect
github.com/cockroachdb/pebble v1.1.0 // indirect
github.com/cockroachdb/pebble v1.1.2 // indirect
github.com/cockroachdb/redact v1.1.5 // indirect
github.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 // indirect
github.com/cometbft/cometbft-db v0.9.1 // indirect
Expand All @@ -145,7 +154,6 @@ require (
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 // indirect
github.com/desertbit/timer v0.0.0-20180107155436-c41aec40b27f // indirect
github.com/dgraph-io/badger/v2 v2.2007.4 // indirect
github.com/dgraph-io/badger/v4 v4.2.1-0.20231013074411-fb1b00959581 // indirect
github.com/dgraph-io/ristretto v0.1.1 // indirect
github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect
github.com/distribution/reference v0.6.0 // indirect
Expand Down Expand Up @@ -179,7 +187,6 @@ require (
github.com/golang/snappy v0.0.4 // indirect
github.com/google/btree v1.1.2 // indirect
github.com/google/cel-go v0.20.1 // indirect
github.com/google/flatbuffers v1.12.1 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/go-containerregistry v0.20.0 // indirect
github.com/google/orderedcode v0.0.1 // indirect
Expand Down
22 changes: 10 additions & 12 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -349,12 +349,14 @@ github.com/cockroachdb/apd/v3 v3.2.1/go.mod h1:klXJcjp+FffLTHlhIG69tezTDvdP065na
github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8=
github.com/cockroachdb/datadriven v1.0.3-0.20230413201302-be42291fc80f h1:otljaYPt5hWxV3MUfO5dFPFiOXg9CyG5/kCfayTqsJ4=
github.com/cockroachdb/datadriven v1.0.3-0.20230413201302-be42291fc80f/go.mod h1:a9RdTaap04u637JoCzcUoIcDmvwSUtcUFtT/C3kJlTU=
github.com/cockroachdb/errors v1.11.1 h1:xSEW75zKaKCWzR3OfxXUxgrk/NtT4G1MiOv5lWZazG8=
github.com/cockroachdb/errors v1.11.1/go.mod h1:8MUxA3Gi6b25tYlFEBGLf+D8aISL+M4MIpiWMSNRfxw=
github.com/cockroachdb/errors v1.11.3 h1:5bA+k2Y6r+oz/6Z/RFlNeVCesGARKuC6YymtcDrbC/I=
github.com/cockroachdb/errors v1.11.3/go.mod h1:m4UIW4CDjx+R5cybPsNrRbreomiFqt8o1h1wUVazSd8=
github.com/cockroachdb/fifo v0.0.0-20240606204812-0bbfbd93a7ce h1:giXvy4KSc/6g/esnpM7Geqxka4WSqI1SZc7sMJFd3y4=
github.com/cockroachdb/fifo v0.0.0-20240606204812-0bbfbd93a7ce/go.mod h1:9/y3cnZ5GKakj/H4y9r9GTjCvAFta7KLgSHPJJYc52M=
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b h1:r6VH0faHjZeQy818SGhaone5OnYfxFR/+AzdY3sf5aE=
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b/go.mod h1:Vz9DsVWQQhf3vs21MhPMZpMGSht7O/2vFW2xusFUVOs=
github.com/cockroachdb/pebble v1.1.0 h1:pcFh8CdCIt2kmEpK0OIatq67Ln9uGDYY3d5XnE0LJG4=
github.com/cockroachdb/pebble v1.1.0/go.mod h1:sEHm5NOXxyiAoKWhoFxT8xMgd/f3RA6qUqQ1BXKrh2E=
github.com/cockroachdb/pebble v1.1.2 h1:CUh2IPtR4swHlEj48Rhfzw6l/d0qA31fItcIszQVIsA=
github.com/cockroachdb/pebble v1.1.2/go.mod h1:4exszw1r40423ZsmkG/09AFEG83I0uDgfujJdbL6kYU=
github.com/cockroachdb/redact v1.1.5 h1:u1PMllDkdFfPWaNGMyLD1+so+aq3uUItthCFqzwPJ30=
github.com/cockroachdb/redact v1.1.5/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg=
github.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 h1:zuQyyAKVxetITBuuhv3BI9cMrmStnpT18zmgmTxunpo=
Expand Down Expand Up @@ -435,8 +437,6 @@ github.com/desertbit/timer v0.0.0-20180107155436-c41aec40b27f h1:U5y3Y5UE0w7amNe
github.com/desertbit/timer v0.0.0-20180107155436-c41aec40b27f/go.mod h1:xH/i4TFMt8koVQZ6WFms69WAsDWr2XsYL3Hkl7jkoLE=
github.com/dgraph-io/badger/v2 v2.2007.4 h1:TRWBQg8UrlUhaFdco01nO2uXwzKS7zd+HVdwV/GHc4o=
github.com/dgraph-io/badger/v2 v2.2007.4/go.mod h1:vSw/ax2qojzbN6eXHIx6KPKtCSHJN/Uz0X0VPruTIhk=
github.com/dgraph-io/badger/v4 v4.2.1-0.20231013074411-fb1b00959581 h1:yy45brf1ktmnkTCZlHynP1gRlVwZ9g19oz5D9wG81v4=
github.com/dgraph-io/badger/v4 v4.2.1-0.20231013074411-fb1b00959581/go.mod h1:T/uWAYxrXdaXw64ihI++9RMbKTCpKd/yE9+saARew7k=
github.com/dgraph-io/ristretto v0.0.3-0.20200630154024-f66de99634de/go.mod h1:KPxhHT9ZxKefz+PCeOGsrHpl1qZ7i70dGTu2u+Ahh6E=
github.com/dgraph-io/ristretto v0.1.1 h1:6CWw5tJNgpegArSHpNHJKldNeq03FQCwYvfMVWajOK8=
github.com/dgraph-io/ristretto v0.1.1/go.mod h1:S1GPSBCYCIhmVNfcth17y2zZtQT6wzkzgwUve0VDWWA=
Expand Down Expand Up @@ -631,8 +631,6 @@ github.com/google/btree v1.1.2 h1:xf4v41cLI2Z6FxbKm+8Bu+m8ifhj15JuZ9sa0jZCMUU=
github.com/google/btree v1.1.2/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4=
github.com/google/cel-go v0.20.1 h1:nDx9r8S3L4pE61eDdt8igGj8rf5kjYR3ILxWIpWNi84=
github.com/google/cel-go v0.20.1/go.mod h1:kWcIzTsPX0zmQ+H3TirHstLLf9ep5QTsZBN9u4dOYLg=
github.com/google/flatbuffers v1.12.1 h1:MVlul7pQNoDzWRLTw5imwYsl+usrS1TXG2H4jg6ImGw=
github.com/google/flatbuffers v1.12.1/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
Expand Down Expand Up @@ -996,10 +994,10 @@ github.com/pokt-network/ring-go v0.1.0 h1:hF7mDR4VVCIqqDAsrloP8azM9y1mprc99YgnTj
github.com/pokt-network/ring-go v0.1.0/go.mod h1:8NHPH7H3EwrPX3XHfpyRI6bz4gApkE3+fd0XZRbMWP0=
github.com/pokt-network/shannon-sdk v0.0.0-20240814144717-dfa95b525d46 h1:rkJa3LGPN+f/k2Vv9t09UWtgbpkMA0NLHRZDiGKPqo4=
github.com/pokt-network/shannon-sdk v0.0.0-20240814144717-dfa95b525d46/go.mod h1:Jql/lobckajY8BVHDRkQPf6zYraompTK1et78jKVi68=
github.com/pokt-network/smt v0.12.0 h1:uqru/0ykC4LnBoMacakobNOd1iRK69PlohqjMtLmYNA=
github.com/pokt-network/smt v0.12.0/go.mod h1:S4Ho4OPkK2v2vUCHNtA49XDjqUC/OFYpBbynRVYmxvA=
github.com/pokt-network/smt/kvstore/badger v0.0.0-20240109205447-868237978c0b h1:TjfgV3vgW0zW47Br/OgUXD4M8iyR74EYanbFfN4ed8o=
github.com/pokt-network/smt/kvstore/badger v0.0.0-20240109205447-868237978c0b/go.mod h1:GbzcG5ebj8twKmBL1VzdPM4NS44okwYXBfQaVXT+6yU=
github.com/pokt-network/smt v0.0.0-20240819232427-be8f5d598321 h1:Ru+yl6E6+chh/wAqkVcAI6OtR17s83p6x4t/6Hur+jk=
github.com/pokt-network/smt v0.0.0-20240819232427-be8f5d598321/go.mod h1:S4Ho4OPkK2v2vUCHNtA49XDjqUC/OFYpBbynRVYmxvA=
github.com/pokt-network/smt/kvstore/pebble v0.0.0-20240819232427-be8f5d598321 h1:jMNlDjbmzEqW5/o9a05LbF+GjDQNVFTZQkoIbFyOX5E=
github.com/pokt-network/smt/kvstore/pebble v0.0.0-20240819232427-be8f5d598321/go.mod h1:CZlY7W+7LU5hMFoTb9PmyL4wu/i41t3+gVMvDePxAmo=
github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs=
Expand Down
51 changes: 51 additions & 0 deletions load-testing/loadtest_manifest_localnet_single_supplier.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# NB: The number of pre-provisioned **LocalNet** actors are managed in
# 'localnet_config.yaml' by the respective actors `count` property.

is_ephemeral_chain: true # This should be `true` for LocalNet as it is an ephemeral network

# The service ID to use for the load test.
service_id: anvil

# The address of the account that will be used to fund the the application,
# gateway and supplier accounts so that they can stake on the network.
funding_account_address: pokt1eeeksh2tvkh7wzmfrljnhw4wrhs55lcuvmekkw # address for pnf account
okdas marked this conversation as resolved.
Show resolved Hide resolved

# List of pre-provisioned suppliers used for load testing.
# These suppliers will be progressively staked during the load test, according
# to the test plan that is being executed.
#
# When running on **LocalNet**, the number of the suppliers defined below should match
# the `relayminers.count` in your `localnet_config.yaml`.
suppliers:
# The supplier address that is available in the load test's environment keyring,
# used to identify the supplier and sign relays and transactions with.
# It must be the address corresponding to the provided signing_key_name in the
# `relay_miner_config.yaml` file.

# RelayMiner 1; http://localhost:10350/r/relayminer1/overview
- address: pokt19a3t4yunp0dlpfjrp7qwnzwlrzd5fzs2gjaaaj
# The advertised URL used by the supplier when it submits a stake message on-chain.
exposed_url: http://relayminer1:8545

# List of pre-provisioned gateways used for load testing.
# These gateways will be progressively staked and delegated to during the load test.
#
# When running on LocalNet, the number of the gateways defined below should match
# the `gateways.count` in your `localnet_config.yaml`.
gateways:
# The gateway address that is available in the load test's environment keyring,
# used to identify the gateway and sign relays and transactions with.
# It must be the address corresponding to the provided signing_key_name in the
# `appgate_server_config.yaml` file.

# Gateway 1; http://localhost:10350/r/gateway1/overview
- address: pokt15vzxjqklzjtlz7lahe8z2dfe9nm5vxwwmscne4
exposed_url: http://localhost:42079 # The gateway url that the user sends relays to (e.g. curl)

# Gateway 2; http://localhost:10350/r/gateway2/overview
- address: pokt15w3fhfyc0lttv7r585e2ncpf6t2kl9uh8rsnyz
exposed_url: http://localhost:42080

# Gateway 3; http://localhost:10350/r/gateway3/overview
- address: pokt1zhmkkd0rh788mc9prfq0m2h88t9ge0j83gnxya
exposed_url: http://localhost:42081
6 changes: 3 additions & 3 deletions load-testing/tests/relays_stress.feature
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ Feature: Loading gateway server with relays
| supplier | 1 |
And more actors are staked as follows:
| actor | actor inc amount | blocks per inc | max actors |
| application | 4 | 4 | 12 |
| gateway | 1 | 4 | 3 |
| supplier | 1 | 4 | 3 |
| application | 4 | 10 | 12 |
| gateway | 1 | 10 | 3 |
| supplier | 1 | 10 | 3 |
Comment on lines +13 to +15
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was under the impression that the "blocks per inc" needed to be a multiple of the blocks per session param to maintain constant rates of change across various metrics as the test scales actors. I also thought that there was a check for this somewhere in the load test helpers, around the "plans".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bryanchriswhite Are you sure about this?

  1. If so -> can you help @okdas look into the code and find the reasoning.
  2. If not -> my intuition is that it keeps things simpler (to reason about) but doesn't need to be enforced.

In real life, we'll be staking / unstaking irrespective of the blocker per session so it doesn't make sense for the framework to have this limitation.

@okdas W/e the resolution ends up being, seems like the helper in the code needs a #PUC.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bryanchriswhite interesting. This check is the reason I adjusted the number - as I was getting an error. Given our current blocks per session is 10 (bumped from 4 a couple of weeks ago), what do you think should be the best value?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@okdas that's the check I was referring to. 👍 I did not realize that blocks per session was 10; this all makes sense now.

When a load of concurrent relay requests are sent from the applications
Then the correct pairs count of claim and proof messages should be committed on-chain
17 changes: 17 additions & 0 deletions load-testing/tests/relays_stress_single_suppier.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
Feature: Loading gateway server with relays

Scenario: Incrementing the number of relays and actors
Given localnet is running
And a rate of "1" relay requests per second is sent per application
And the following initial actors are staked:
| actor | count |
| application | 4 |
| gateway | 1 |
| supplier | 1 |
And more actors are staked as follows:
| actor | actor inc amount | blocks per inc | max actors |
| application | 4 | 10 | 12 |
| gateway | 1 | 10 | 3 |
| supplier | 1 | 10 | 1 |
bryanchriswhite marked this conversation as resolved.
Show resolved Hide resolved
When a load of concurrent relay requests are sent from the applications
Then the correct pairs count of claim and proof messages should be committed on-chain
4 changes: 4 additions & 0 deletions load-testing/tests/relays_stress_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,10 @@ func TestLoadRelays(t *testing.T) {
gocuke.NewRunner(t, &relaysSuite{}).Path(filepath.Join(".", "relays_stress.feature")).Run()
}

func TestLoadRelaysSingleSupplier(t *testing.T) {
gocuke.NewRunner(t, &relaysSuite{}).Path(filepath.Join(".", "relays_stress_single_suppier.feature")).Run()
}

func (s *relaysSuite) LocalnetIsRunning() {
s.ctx, s.cancelCtx = context.WithCancel(context.Background())

Expand Down
2 changes: 1 addition & 1 deletion localnet/kubernetes/values-relayminer-common.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
config:
smt_store_path: smt_stores
smt_store_path: /root/.poktroll/smt
metrics:
enabled: true
addr: :9090
Expand Down
2 changes: 1 addition & 1 deletion localnet/poktrolld/config/relayminer_config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
signing_key_name: supplier1
smt_store_path: smt_stores
smt_store_path: /root/.poktroll/smt
metrics:
enabled: true
addr: :9090
Expand Down
4 changes: 2 additions & 2 deletions pkg/client/supplier/client_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import (
"cosmossdk.io/depinject"
"github.com/golang/mock/gomock"
"github.com/pokt-network/smt"
"github.com/pokt-network/smt/kvstore/badger"
"github.com/pokt-network/smt/kvstore/pebble"
"github.com/stretchr/testify/require"

"github.com/pokt-network/poktroll/pkg/client/keyring"
Expand Down Expand Up @@ -175,7 +175,7 @@ func TestSupplierClient_SubmitProof(t *testing.T) {
},
}

kvStore, err := badger.NewKVStore("")
kvStore, err := pebble.NewKVStore("")
require.NoError(t, err)

// Generating an ephemeral tree & spec just so we can submit
Expand Down
20 changes: 12 additions & 8 deletions pkg/relayer/session/sessiontree.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (

cosmostypes "github.com/cosmos/cosmos-sdk/types"
"github.com/pokt-network/smt"
"github.com/pokt-network/smt/kvstore/badger"
"github.com/pokt-network/smt/kvstore/pebble"

"github.com/pokt-network/poktroll/pkg/crypto/protocol"
"github.com/pokt-network/poktroll/pkg/relayer"
Expand Down Expand Up @@ -51,7 +51,7 @@ type sessionTree struct {
proofBz []byte

// treeStore is the KVStore used to store the SMST.
treeStore badger.BadgerKVStore
treeStore pebble.PebbleKVStore

// storePath is the path to the KVStore used to store the SMST.
// It is created from the storePrefix and the session.sessionId.
Expand All @@ -72,14 +72,18 @@ func NewSessionTree(
) (relayer.SessionTree, error) {
// Join the storePrefix and the session.sessionId and supplier's operator address to
// create a unique storePath.
storePath := filepath.Join(storesDirectory, sessionHeader.SessionId, "_", supplierOperatorAddress.String())

// TODO_IMPROVE: instead of creating a new KV store for each session, it will be more beneficial to
okdas marked this conversation as resolved.
Show resolved Hide resolved
// use one key store. KV databases are often optimized for writing into one database. They keys can
// use supplier address and session id as prefix. The current approach might not be RAM/IO efficient.
storePath := filepath.Join(storesDirectory, supplierOperatorAddress.String(), sessionHeader.SessionId)

// Make sure storePath does not exist when creating a new SessionTree
if _, err := os.Stat(storePath); err != nil && !os.IsNotExist(err) {
return nil, ErrSessionTreeStorePathExists.Wrapf("storePath: %q", storePath)
}

treeStore, err := badger.NewKVStore(storePath)
treeStore, err := pebble.NewKVStore(storePath)
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -157,7 +161,7 @@ func (st *sessionTree) ProveClosest(path []byte) (proof *smt.SparseMerkleClosest
}

// Restore the KVStore from disk since it has been closed after the claim has been generated.
st.treeStore, err = badger.NewKVStore(st.storePath)
st.treeStore, err = pebble.NewKVStore(st.storePath)
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -240,9 +244,9 @@ func (st *sessionTree) Delete() error {

st.isClaiming = false

if err := st.treeStore.ClearAll(); err != nil {
return err
}
// NB: We used to call `st.treeStore.ClearAll()` here.
// This was intentionally removed to lower the IO load.
// When the database is closed, it is deleted it from disk right away.

if err := st.treeStore.Stop(); err != nil {
return err
Expand Down
4 changes: 2 additions & 2 deletions tests/integration/tokenomics/relay_mining_difficulty_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"testing"

"github.com/pokt-network/smt"
"github.com/pokt-network/smt/kvstore/badger"
"github.com/pokt-network/smt/kvstore/pebble"
"github.com/stretchr/testify/require"

"github.com/pokt-network/poktroll/cmd/poktrolld/cmd"
Expand Down Expand Up @@ -175,7 +175,7 @@ func prepareSMST(
// Generating an ephemeral tree & spec just so we can submit
// a proof of the right size.
// TODO_TECHDEBT(#446): Centralize the configuration for the SMT spec.
kvStore, err := badger.NewKVStore("")
kvStore, err := pebble.NewKVStore("")
require.NoError(t, err)

// NB: A signed mined relay is a MinedRelay type with the appropriate
Expand Down
3 changes: 3 additions & 0 deletions x/application/types/params.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading