Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Reconciler Configuration #23

Merged
merged 1 commit into from
May 18, 2020
Merged

Improve Reconciler Configuration #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 25 additions & 24 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,30 +94,31 @@ Usage:
rosetta-cli check [flags]

Flags:
--block-concurrency uint concurrency to use while fetching blocks (default 8)
--bootstrap-balances string Absolute path to a file used to bootstrap balances before starting syncing.
Populating this value after beginning syncing will return an error.
--data-dir string folder used to store logs and any data used to perform validation
--end int block index to stop syncing (default -1)
--exempt-accounts string Absolute path to a file listing all accounts to exempt from balance
tracking and reconciliation. Look at the examples directory for an example of
how to structure this file.
--halt-on-reconciliation-error Determines if block processing should halt on a reconciliation
error. It can be beneficial to collect all reconciliation errors or silence
reconciliation errors during development. (default true)
-h, --help help for check
--interesting-accounts string Absolute path to a file listing all accounts to check on each block. Look
at the examples directory for an example of how to structure this file.
--log-balance-changes log balance changes
--log-blocks log processed blocks
--log-reconciliations log balance reconciliations
--log-transactions log processed transactions
--lookup-balance-by-block When set to true, balances are looked up at the block where a balance
change occurred instead of at the current block. Blockchains that do not support
historical balance lookup should set this to false. (default true)
--reconciler-concurrency uint concurrency to use while fetching accounts during reconciliation (default 8)
--start int block index to start syncing (default -1)
--transaction-concurrency uint concurrency to use while fetching transactions (if required) (default 16)
--active-reconciliation-concurrency uint concurrency to use while fetching accounts during active reconciliation (default 8)
--block-concurrency uint concurrency to use while fetching blocks (default 8)
--bootstrap-balances string Absolute path to a file used to bootstrap balances before starting syncing.
Populating this value after beginning syncing will return an error.
--data-dir string folder used to store logs and any data used to perform validation
--end int block index to stop syncing (default -1)
--exempt-accounts string Absolute path to a file listing all accounts to exempt from balance
tracking and reconciliation. Look at the examples directory for an example of
how to structure this file.
--halt-on-reconciliation-error Determines if block processing should halt on a reconciliation
error. It can be beneficial to collect all reconciliation errors or silence
reconciliation errors during development. (default true)
-h, --help help for check
--inactive-reconciliation-concurrency uint concurrency to use while fetching accounts during inactive reconciliation (default 4)
--interesting-accounts string Absolute path to a file listing all accounts to check on each block. Look
at the examples directory for an example of how to structure this file.
--log-balance-changes log balance changes
--log-blocks log processed blocks
--log-reconciliations log balance reconciliations
--log-transactions log processed transactions
--lookup-balance-by-block When set to true, balances are looked up at the block where a balance
change occurred instead of at the current block. Blockchains that do not support
historical balance lookup should set this to false. (default true)
--start int block index to start syncing (default -1)
--transaction-concurrency uint concurrency to use while fetching transactions (if required) (default 16)

Global Flags:
--server-url string base URL for a Rosetta server (default "http://localhost:8080")
Expand Down
25 changes: 18 additions & 7 deletions cmd/check.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,9 +114,13 @@ of what one of these files looks like.`,
// while fetching transactions (if required).
TransactionConcurrency uint64

// ReconcilerConcurrency is the concurrency to use
// while fetching accounts during reconciliation.
ReconcilerConcurrency uint64
// ActiveReconciliationConcurrency is the concurrency to use
// while fetching accounts during active reconciliation.
ActiveReconciliationConcurrency uint64

// InactiveReconciliationConcurrency is the concurrency to use
// while fetching accounts during inactive reconciliation.
InactiveReconciliationConcurrency uint64

// LogBlocks determines if blocks are
// logged.
Expand Down Expand Up @@ -207,10 +211,16 @@ func init() {
"concurrency to use while fetching transactions (if required)",
)
checkCmd.Flags().Uint64Var(
&ReconcilerConcurrency,
"reconciler-concurrency",
&ActiveReconciliationConcurrency,
"active-reconciliation-concurrency",
8,
"concurrency to use while fetching accounts during reconciliation",
"concurrency to use while fetching accounts during active reconciliation",
)
checkCmd.Flags().Uint64Var(
&InactiveReconciliationConcurrency,
"inactive-reconciliation-concurrency",
4,
"concurrency to use while fetching accounts during inactive reconciliation",
)
checkCmd.Flags().BoolVar(
&LogBlocks,
Expand Down Expand Up @@ -381,7 +391,8 @@ func runCheckCmd(cmd *cobra.Command, args []string) {
reconcilerHelper,
reconcilerHandler,
fetcher,
reconciler.WithReconcilerConcurrency(int(ReconcilerConcurrency)),
reconciler.WithActiveConcurrency(int(ActiveReconciliationConcurrency)),
reconciler.WithInactiveConcurrency(int(InactiveReconciliationConcurrency)),
reconciler.WithLookupBalanceByBlock(LookupBalanceByBlock),
reconciler.WithInterestingAccounts(interestingAccounts),
reconciler.WithSeenAccounts(seenAccounts),
Expand Down
2 changes: 1 addition & 1 deletion cmd/root.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,6 @@ var versionCmd = &cobra.Command{
Use: "version",
Short: "Print rosetta-cli version",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("v0.2.2")
fmt.Println("v0.2.3")
},
}
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.13

require (
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 // indirect
github.com/coinbase/rosetta-sdk-go v0.2.1-0.20200513162126-2278020ff1c2
github.com/coinbase/rosetta-sdk-go v0.2.1-0.20200518143022-cea5733d9710
github.com/dgraph-io/badger v1.6.1
github.com/golang/protobuf v1.4.2 // indirect
github.com/mattn/goveralls v0.0.5 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ github.com/coinbase/rosetta-sdk-go v0.2.1-0.20200513022350-4a871d015a27 h1:HfYzT
github.com/coinbase/rosetta-sdk-go v0.2.1-0.20200513022350-4a871d015a27/go.mod h1:XKM7urGHLqGQJi9kM97N+GpMLJuCAGYXy2wOm3KzxEE=
github.com/coinbase/rosetta-sdk-go v0.2.1-0.20200513162126-2278020ff1c2 h1:ibaPMZAs6dlh+lmexnEnL3DcaZ+r9NUknPQ/9FgZ7MQ=
github.com/coinbase/rosetta-sdk-go v0.2.1-0.20200513162126-2278020ff1c2/go.mod h1:XKM7urGHLqGQJi9kM97N+GpMLJuCAGYXy2wOm3KzxEE=
github.com/coinbase/rosetta-sdk-go v0.2.1-0.20200518143022-cea5733d9710 h1:XlYar0V5oPA764ZL0oOtNWB5+cgzdLw2cyQiAuVu+4Q=
github.com/coinbase/rosetta-sdk-go v0.2.1-0.20200518143022-cea5733d9710/go.mod h1:XKM7urGHLqGQJi9kM97N+GpMLJuCAGYXy2wOm3KzxEE=
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
Expand Down