Skip to content

Commit

Permalink
Merge pull request #70 from coinbase/patrick/configuration-file
Browse files Browse the repository at this point in the history 
Add Configuration File Support
  • Loading branch information
@patrick-ogrady
patrick-ogrady authored Jul 23, 2020
2 parents 33358ed + 997c73c commit 00983dd
Show file tree
Hide file tree
Showing 16 changed files with 965 additions and 338 deletions.
4 changes: 2 additions & 2 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ ADDLICENSE_CMD=go run github.com/google/addlicense
ADDLICENCE_SCRIPT=${ADDLICENSE_CMD} -c "Coinbase, Inc." -l "apache" -v
GOLINES_CMD=go run github.com/segmentio/golines
GOVERALLS_CMD=go run github.com/mattn/goveralls
TEST_SCRIPT=go test -v ./internal/...
TEST_SCRIPT=go test -v ./internal/... ./configuration/...

deps:
go get ./...
Expand Down Expand Up @@ -39,7 +39,7 @@ check-license:
${ADDLICENCE_SCRIPT} -check .

shorten-lines:
${GOLINES_CMD} -w --shorten-comments internal cmd
${GOLINES_CMD} -w --shorten-comments internal cmd configuration

salus:
docker run --rm -t -v ${PWD}:/home/repo coinbase/salus
Expand Down
158 changes: 95 additions & 63 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,17 +41,25 @@ Usage:
rosetta-cli [command]
Available Commands:
check Check the correctness of a Rosetta Data API Server
create:configuration Generate a static configuration file for the Asserter
help Help about any command
version Print rosetta-cli version
view:account View an account balance
view:block View a block
view:network View network status
check:construction Check the correctness of a Rosetta Construction API Implementation
check:data Check the correctness of a Rosetta Data API Implementation
configuration:create Create a default configuration file at the provided path
configuration:validate Validate the correctness of a configuration file at the provided path
help Help about any command
utils:asserter-configuration Generate a static configuration file for the Asserter
version Print rosetta-cli version
view:account View an account balance
view:block View a block
view:network View network status
Flags:
-h, --help help for rosetta-cli
--server-url string base URL for a Rosetta server (default "http://localhost:8080")
--configuration-file string Configuration file that provides connection and test settings.
If you would like to generate a starter configuration file (populated
with the defaults), run rosetta-cli configuration:create.
Any fields not populated in the configuration file will be populated with
default values.
-h, --help help for rosetta-cli
Use "rosetta-cli [command] --help" for more information about a command.
```
Expand All @@ -67,18 +75,23 @@ Flags:
-h, --help help for version
Global Flags:
--server-url string base URL for a Rosetta server (default "http://localhost:8080")
--configuration-file string Configuration file that provides connection and test settings.
If you would like to generate a starter configuration file (populated
with the defaults), run rosetta-cli configuration:create.
Any fields not populated in the configuration file will be populated with
default values.
```

### check
### check:data
```
Check all server responses are properly constructed, that
there are no duplicate blocks and transactions, that blocks can be processed
from genesis to the current block (re-orgs handled automatically), and that
computed balance changes are equal to balance changes reported by the node.
When re-running this command, it will start where it left off if you specify
some --data-dir. Otherwise, it will create a new temporary directory and start
some data directory. Otherwise, it will create a new temporary directory and start
again from the genesis block. If you want to discard some number of blocks
populate the --start flag with some block index. Starting from a given index
can be useful to debug a small range of blocks for issues but it is highly
Expand All @@ -87,87 +100,65 @@ are performed.
By default, account balances are looked up at specific heights (instead of
only at the current block). If your node does not support this functionality
set --lookup-balance-by-block to false. This will make reconciliation much
set historical balance disabled to true. This will make reconciliation much
less efficient but it will still work.
If check fails due to an INACTIVE reconciliation error (balance changed without
any corresponding operation), the cli will automatically try to find the block
missing an operation. If --lookup-balance-by-block is not enabled, this automatic
missing an operation. If historical balance disabled is true, this automatic
debugging tool does not work.
To debug an INACTIVE account reconciliation error without --lookup-balance-by-block, set the
--interesting-accounts flag to the absolute path of a JSON file containing
To debug an INACTIVE account reconciliation error without historical balance lookup,
set the interesting accunts to the path of a JSON file containing
accounts that will be actively checked for balance changes at each block. This
will return an error at the block where a balance change occurred with no
corresponding operations.
If your blockchain has a genesis allocation of funds and you set
--lookup-balance-by-block to false, you must provide an
historical balance disabled to true, you must provide an
absolute path to a JSON file containing initial balances with the
--bootstrap-balances flag. You can look at the examples folder for an example
bootstrap balance config. You can look at the examples folder for an example
of what one of these files looks like.
Usage:
rosetta-cli check [flags]
rosetta-cli check:data [flags]
Flags:
--active-reconciliation-concurrency uint concurrency to use while fetching accounts during active reconciliation (default 8)
--block-concurrency uint concurrency to use while fetching blocks (default 8)
--bootstrap-balances string Absolute path to a file used to bootstrap balances before starting syncing.
Populating this value after beginning syncing will return an error.
--data-dir string folder used to store logs and any data used to perform validation
--end int block index to stop syncing (default -1)
--exempt-accounts string Absolute path to a file listing all accounts to exempt from balance
tracking and reconciliation. Look at the examples directory for an example of
how to structure this file.
--halt-on-reconciliation-error Determines if block processing should halt on a reconciliation
error. It can be beneficial to collect all reconciliation errors or silence
reconciliation errors during development. (default true)
-h, --help help for check
--inactive-reconciliation-concurrency uint concurrency to use while fetching accounts during inactive reconciliation (default 4)
--inactive-reconciliation-frequency uint the number of blocks to wait between inactive reconiliations on each account (default 250)
--interesting-accounts string Absolute path to a file listing all accounts to check on each block. Look
at the examples directory for an example of how to structure this file.
--log-balance-changes log balance changes
--log-blocks log processed blocks
--log-reconciliations log balance reconciliations
--log-transactions log processed transactions
--lookup-balance-by-block When set to true, balances are looked up at the block where a balance
change occurred instead of at the current block. Blockchains that do not support
historical balance lookup should set this to false. (default true)
--start int block index to start syncing (default -1)
--transaction-concurrency uint concurrency to use while fetching transactions (if required) (default 16)
--end int block index to stop syncing (default -1)
-h, --help help for check:data
--start int block index to start syncing (default -1)
Global Flags:
--server-url string base URL for a Rosetta server (default "http://localhost:8080")
--configuration-file string Configuration file that provides connection and test settings.
If you would like to generate a starter configuration file (populated
with the defaults), run rosetta-cli configuration:create.
Any fields not populated in the configuration file will be populated with
default values.
```

#### Status Codes
If there are no issues found while running `check`, it will exit with a `0` status code.
If there are any issues, it will exit with a `1` status code. It can be useful
to run this command as an integration test for any changes to your implementation.

### create:configuration
### configuration:create
```
In production deployments, it is useful to initialize the response
Asserter (https://github.com/coinbase/rosetta-sdk-go/tree/master/asserter) using
a static configuration instead of intializing a configuration dynamically
from the node. This allows a client to error on new types/statuses that may
have been added in an update instead of silently erroring.
To use this command, simply provide an absolute path as the argument for where
the configuration file should be saved (in JSON). Populate the optional
--server-url flag with the url of the server to generate the configuration
from.
Check the correctness of a Rosetta Construction API Implementation
Usage:
rosetta-cli create:configuration [flags]
rosetta-cli check:construction [flags]
Flags:
-h, --help help for create:configuration
-h, --help help for check:construction
Global Flags:
--server-url string base URL for a Rosetta server (default "http://localhost:8080")
--configuration-file string Configuration file that provides connection and test settings.
If you would like to generate a starter configuration file (populated
with the defaults), run rosetta-cli configuration:create.
Any fields not populated in the configuration file will be populated with
default values.
```

### view:network
Expand All @@ -186,7 +177,12 @@ Flags:
-h, --help help for view:network
Global Flags:
--server-url string base URL for a Rosetta server (default "http://localhost:8080")
--configuration-file string Configuration file that provides connection and test settings.
If you would like to generate a starter configuration file (populated
with the defaults), run rosetta-cli configuration:create.
Any fields not populated in the configuration file will be populated with
default values.
```

### view:account
Expand All @@ -207,7 +203,12 @@ Flags:
-h, --help help for view:account
Global Flags:
--server-url string base URL for a Rosetta server (default "http://localhost:8080")
--configuration-file string Configuration file that provides connection and test settings.
If you would like to generate a starter configuration file (populated
with the defaults), run rosetta-cli configuration:create.
Any fields not populated in the configuration file will be populated with
default values.
```

### view:block
Expand All @@ -229,7 +230,38 @@ Flags:
-h, --help help for view:block
Global Flags:
--server-url string base URL for a Rosetta server (default "http://localhost:8080")
--configuration-file string Configuration file that provides connection and test settings.
If you would like to generate a starter configuration file (populated
with the defaults), run rosetta-cli configuration:create.
Any fields not populated in the configuration file will be populated with
default values.
```

### utils:asserter-configuration
```
In production deployments, it is useful to initialize the response
Asserter (https://github.com/coinbase/rosetta-sdk-go/tree/master/asserter) using
a static configuration instead of intializing a configuration dynamically
from the node. This allows a client to error on new types/statuses that may
have been added in an update instead of silently erroring.
To use this command, simply provide an absolute path as the argument for where
the configuration file should be saved (in JSON).
Usage:
rosetta-cli utils:asserter-configuration [flags]
Flags:
-h, --help help for utils:asserter-configuration
Global Flags:
--configuration-file string Configuration file that provides connection and test settings.
If you would like to generate a starter configuration file (populated
with the defaults), run rosetta-cli configuration:create.
Any fields not populated in the configuration file will be populated with
default values.
```

## Development
Expand Down
34 changes: 34 additions & 0 deletions cmd/check_construction.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Copyright 2020 Coinbase, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package cmd

import (
"log"

"github.com/spf13/cobra"
)

var (
checkConstructionCmd = &cobra.Command{
Use: "check:construction",
Short: "Check the correctness of a Rosetta Construction API Implementation",
Run: runCheckConstructionCmd,
}
)

func runCheckConstructionCmd(cmd *cobra.Command, args []string) {
// ensureDataDirectoryExists()
log.Fatal("not implemented!")
}
Loading

0 comments on commit 00983dd

Please sign in to comment.