diff --git a/README.md b/README.md index 7747a272228b..f4130b3d4675 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ Also, read our [Sharding Reference Implementation Doc](https://github.com/prysma # Table of Contents - [Installation](#installation) -- [Sharding Instructions](#sharding-instructions) +- [Instructions](#instructions) - [Testing](#testing) - [Contributing](#contributing) - [License](#license) @@ -44,7 +44,7 @@ bazel version Bazel manages all of the dependencies for you (including go and necessary compilers) so you are all set to build prysm. -# Sharding Instructions +# Instructions To get started with running the project, follow the instructions to initialize your own private Ethereum blockchain and geth node, as they will be required to run before you can begin running our system @@ -72,8 +72,10 @@ The `alloc` portion specifies account addresses with prefunded ETH when the Ethe Then, you can build and init a new instance of a local, Ethereum blockchain as follows: - $ geth init /path/to/genesis.json -datadir /path/to/your/datadir - $ geth --nodiscover console --datadir /path/to/your/datadir --networkid 12345 +``` +geth init /path/to/genesis.json -datadir /path/to/your/datadir +geth --nodiscover console --datadir /path/to/your/datadir --networkid 12345 +```` It is **important** to note that the `--networkid` flag must match the `chainId` property in the genesis file. @@ -86,46 +88,59 @@ Then, the geth console can start up and you can start a miner as follows: Now, save the passphrase you used in the geth node into a text file called password.txt. Then, once you have this private geth node running on your local network, we will need to generate test, pending transactions that can then be processed into collations by proposers. For this, we have created an in-house transaction generator CLI tool. -# Sharding Minimal Protocol +# Running Ethereum 2.0 -**NOTE**: This section is in flux: will be deprecated in favor of a beacon chain) +**NOTE**: This section is in flux, much of this will likely change as the beacon chain spec evolves. Build our system first ``` -$ bazel build //client/... +bazel build //... ``` -## Becoming a Attester +## Step 1: Deploy a Validator Registation Contract + +Deploy the Validator Registration Contract into the chain of the running geth node by following the instructions [here](https://github.com/prysmaticlabs/prysm/blob/master/contracts/validator-registration-contract/deployVRC/README.md). + +## Step 2: Running a Beacon Node + +Make sure a geth node is running as a separate process according to the instructions from the previous section. Then, you can run a full beacon node as follows: + +``` +bazel run //beacon-chain --\ + --web3provider ws://127.0.0.1:8546 \ + --datadir /path/to/your/datadir \ + --rpc-port 5000 +``` +This will spin up a full beacon node that connects to your running geth node, opens up an RPC connection for sharding clients to connect to it, and begins listening for p2p events. -Make sure a geth node is running as a separate process. Then, to deposit ETH and join as a attester in the Sharding Manager Contract, run the following command: +To try out the beacon node in development by simulating incoming blocks, run the same command above but enable the `--simulator` and a debug level, log verbosity with `--verbosity debug` to see everything happening underneath the hood. ``` -bazel run //client -- \ - --actor "attester" \ - --deposit \ - --datadir /path/to/your/datadir \ - --password /path/to/your/password.txt \ - --networkid 12345 +bazel run //beacon-chain --\ + --web3provider ws://127.0.0.1:8546 \ + --datadir /path/to/your/datadir \ + --rpc-port 5000 \ + --simulator \ + --verbosity debug ``` -This will extract 1000ETH from your account balance and insert you into the SMC's attesters. Then, the program will listen for incoming block headers and notify you when you have been selected as to vote on proposals for a certain shard in a given period. Once you are selected, your sharding node will download collation information to check for data availability on vote on proposals that have been submitted via the `addHeader` function on the SMC. +Now, deposit ETH to become a validator in the contract using instructions [here](https://github.com/prysmaticlabs/prysm/blob/master/beacon-chain/VALIDATOR_REGISTER.md) + +## Step 3: Running a Sharding Client -Concurrently, you will need to run another service that is tasked with processing transactions into collations and submitting them to the SMC via the `addHeader` function. +Once your beacon node is up, you'll need to attach a sharding client as a separate process. This client is in charge of running attester/proposer responsibilities and handling shards (shards to be designed in phase 2). This client will listen for incoming beacon blocks and crystallized states and determine when its time to perform attester/proposer responsibilities accordingly. -## Running a Collation Proposal Node +Run as follows: ``` -bazel run //client -- \ - --actor "proposer" \ - --datadir /path/to/your/datadir \ - --password /path/to/your/password.txt \ - --shardid 0 \ - --networkid 12345 +bazel run //client --\ + --beacon-rpc-provider http://localhost:4000 \ + --verbosity debug ``` -This node is tasked with processing pending transactions into blobs within collations by serializing data into collation bodies. It is responsible for submitting proposals on shard 0 (collation headers) to the SMC via the `addHeader` function. +Then, the beacon node will update this client with new blocks + crystallized states in order for the client to act as an attester or proposer. ## Running via Docker @@ -154,28 +169,18 @@ targets for the container images such that they can be pulled from GCR or dockerhub. -# Making Changes - -## Rebuilding the Sharding Manager Contract Bindings - -The Sharding Manager Contract is built in Solidity and deployed to a running geth node upon launch of the sharding node if it does not exist in the network at a specified address. If there are any changes to the SMC's code, the Golang bindigs must be rebuilt with the following command. - - go generate github.com/prysmaticlabs/prysm/client/contracts - # OR - cd client/contracts && go generate - # Testing To run the unit tests of our system do: ``` -$ bazel test //... +bazel test //... ``` To run our linter, make sure you have [gometalinter](https://github.com/alecthomas/gometalinter) installed and then run ``` -$ gometalinter ./... +gometalinter ./... ``` # Contributing diff --git a/client/internal/BUILD.bazel b/client/internal/BUILD.bazel index c824a1ee0b31..e1a88934f4b3 100644 --- a/client/internal/BUILD.bazel +++ b/client/internal/BUILD.bazel @@ -3,20 +3,12 @@ load("@io_bazel_rules_go//go:def.bzl", "go_library") go_library( name = "go_default_library", testonly = True, - srcs = [ - "beacon_service_mock.go", - "client_helper.go", - ], + srcs = ["beacon_service_mock.go"], importpath = "github.com/prysmaticlabs/prysm/client/internal", visibility = ["//client:__subpackages__"], deps = [ "//proto/beacon/p2p/v1:go_default_library", "//proto/beacon/rpc/v1:go_default_library", - "@com_github_ethereum_go_ethereum//accounts/abi/bind:go_default_library", - "@com_github_ethereum_go_ethereum//accounts/abi/bind/backends:go_default_library", - "@com_github_ethereum_go_ethereum//common:go_default_library", - "@com_github_ethereum_go_ethereum//core/types:go_default_library", - "@com_github_ethereum_go_ethereum//crypto:go_default_library", "@com_github_golang_mock//gomock:go_default_library", "@io_bazel_rules_go//proto/wkt:empty_go_proto", "@org_golang_google_grpc//:go_default_library", diff --git a/client/internal/README.md b/client/internal/README.md index 0fcbccc022cc..2d340da7b8ca 100644 --- a/client/internal/README.md +++ b/client/internal/README.md @@ -1,3 +1,3 @@ -# Sharding Internal Package +# Client Internal Package -This package should be used for test helpers and non-production code only! +This package should be used for test helpers and non-production code only! Mocks generated with gomock and omckgen will go in this package. diff --git a/client/internal/client_helper.go b/client/internal/client_helper.go deleted file mode 100644 index 4eb60d8a98f0..000000000000 --- a/client/internal/client_helper.go +++ /dev/null @@ -1,42 +0,0 @@ -// Package internal provides a MockClient for testing. -package internal - -import ( - "context" - "math/big" - "testing" - - "github.com/ethereum/go-ethereum/accounts/abi/bind" - "github.com/ethereum/go-ethereum/accounts/abi/bind/backends" - "github.com/ethereum/go-ethereum/common" - gethTypes "github.com/ethereum/go-ethereum/core/types" - "github.com/ethereum/go-ethereum/crypto" -) - -var ( - key, _ = crypto.HexToECDSA("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291") -) - -// MockClient for testing proposer. -type MockClient struct { - T *testing.T - Backend *backends.SimulatedBackend - BlockNumber int64 -} - -// TransactionReceipt returns the transaction receipt from the mock backend. -func (m *MockClient) TransactionReceipt(hash common.Hash) (*gethTypes.Receipt, error) { - return m.Backend.TransactionReceipt(context.Background(), hash) -} - -// CreateTXOpts returns transaction opts with the value assigned. -func (m *MockClient) CreateTXOpts(value *big.Int) (*bind.TransactOpts, error) { - txOpts := TransactOpts() - txOpts.Value = value - return txOpts, nil -} - -// TransactOpts Creates a new transaction options. -func TransactOpts() *bind.TransactOpts { - return bind.NewKeyedTransactor(key) -} diff --git a/client/main.go b/client/main.go index 6314a9a2e6b7..da949716996d 100644 --- a/client/main.go +++ b/client/main.go @@ -1,4 +1,3 @@ -// Package client defines the required functionalities for the sharding client. package main import ( @@ -15,6 +14,13 @@ import ( ) func startNode(ctx *cli.Context) error { + verbosity := ctx.GlobalString(cmd.VerbosityFlag.Name) + level, err := logrus.ParseLevel(verbosity) + if err != nil { + return err + } + logrus.SetLevel(level) + shardingNode, err := node.NewShardInstance(ctx) if err != nil { return err @@ -55,7 +61,17 @@ VERSION: app.Usage = `launches a sharding client that interacts with a beacon chain, starts proposer services, shardp2p connections, and more ` app.Action = startNode - app.Flags = []cli.Flag{types.ActorFlag, types.BeaconRPCProviderFlag, cmd.VerbosityFlag, cmd.DataDirFlag, debug.PProfFlag, debug.PProfAddrFlag, debug.PProfPortFlag, debug.MemProfileRateFlag, debug.CPUProfileFlag, debug.TraceFlag} + app.Flags = []cli.Flag{ + types.BeaconRPCProviderFlag, + cmd.VerbosityFlag, + cmd.DataDirFlag, + debug.PProfFlag, + debug.PProfAddrFlag, + debug.PProfPortFlag, + debug.MemProfileRateFlag, + debug.CPUProfileFlag, + debug.TraceFlag, + } app.Before = func(ctx *cli.Context) error { runtime.GOMAXPROCS(runtime.NumCPU()) diff --git a/client/node/BUILD.bazel b/client/node/BUILD.bazel index 5fdaf0782c80..a1d43c8805fa 100644 --- a/client/node/BUILD.bazel +++ b/client/node/BUILD.bazel @@ -8,7 +8,6 @@ go_library( deps = [ "//client/attester:go_default_library", "//client/beacon:go_default_library", - "//client/params:go_default_library", "//client/proposer:go_default_library", "//client/rpcclient:go_default_library", "//client/txpool:go_default_library", diff --git a/client/node/node.go b/client/node/node.go index 49374a957414..c56529d93e65 100644 --- a/client/node/node.go +++ b/client/node/node.go @@ -1,7 +1,5 @@ -// Package node defines a backend for a sharding-enabled, Ethereum blockchain. -// It defines a struct which handles the lifecycle of services in the -// sharding system, providing a bridge to the main Ethereum blockchain, -// as well as instantiating peer-to-peer networking for shards. +// Package node defines a sharding client which connects to a +// full beacon node as part of the Ethereum 2.0 specification. package node import ( @@ -14,7 +12,6 @@ import ( "github.com/prysmaticlabs/prysm/client/attester" "github.com/prysmaticlabs/prysm/client/beacon" - "github.com/prysmaticlabs/prysm/client/params" "github.com/prysmaticlabs/prysm/client/proposer" "github.com/prysmaticlabs/prysm/client/rpcclient" "github.com/prysmaticlabs/prysm/client/txpool" @@ -32,21 +29,17 @@ var log = logrus.WithField("prefix", "node") const shardChainDBName = "shardchaindata" -// ShardEthereum is a service that is registered and started when geth is launched. -// it contains APIs and fields that handle the different components of the sharded -// Ethereum network. +// ShardEthereum defines an instance of a sharding client that manages +// the entire lifecycle of services attached to it participating in +// Ethereum 2.0. type ShardEthereum struct { - shardConfig *params.Config // Holds necessary information to configure shards. - - // Lifecycle and service stores. - services *shared.ServiceRegistry + services *shared.ServiceRegistry // Lifecycle and service store. lock sync.RWMutex stop chan struct{} // Channel to wait for termination notifications. db *database.DB } -// NewShardInstance creates a new sharding-enabled Ethereum instance. This is called in the main -// geth sharding entrypoint. +// NewShardInstance creates a new, Ethereum 2.0 sharding client. func NewShardInstance(ctx *cli.Context) (*ShardEthereum, error) { registry := shared.NewServiceRegistry() shardEthereum := &ShardEthereum{ @@ -54,9 +47,6 @@ func NewShardInstance(ctx *cli.Context) (*ShardEthereum, error) { stop: make(chan struct{}), } - // Configure shardConfig by loading the default. - shardEthereum.shardConfig = params.DefaultConfig() - if err := shardEthereum.startDB(ctx); err != nil { return nil, err } @@ -65,8 +55,7 @@ func NewShardInstance(ctx *cli.Context) (*ShardEthereum, error) { return nil, err } - actorFlag := ctx.GlobalString(types.ActorFlag.Name) - if err := shardEthereum.registerTXPool(actorFlag); err != nil { + if err := shardEthereum.registerTXPool(); err != nil { return nil, err } @@ -78,18 +67,22 @@ func NewShardInstance(ctx *cli.Context) (*ShardEthereum, error) { return nil, err } - if err := shardEthereum.registerActorService(actorFlag); err != nil { + if err := shardEthereum.registerAttesterService(); err != nil { + return nil, err + } + + if err := shardEthereum.registerProposerService(); err != nil { return nil, err } return shardEthereum, nil } -// Start the ShardEthereum service and kicks off the p2p and actor's main loop. +// Start every service in the sharding client. func (s *ShardEthereum) Start() { s.lock.Lock() - log.Info("Starting sharding node") + log.Info("Starting sharding client") s.services.StartAll() @@ -110,7 +103,7 @@ func (s *ShardEthereum) Start() { } } debug.Exit() // Ensure trace and CPU profile data are flushed. - panic("Panic closing the sharding node") + panic("Panic closing the sharding client") }() // Wait for stop channel to be closed. @@ -124,7 +117,7 @@ func (s *ShardEthereum) Close() { s.db.Close() s.services.StopAll() - log.Info("Stopping sharding node") + log.Info("Stopping sharding client") close(s.stop) } @@ -156,10 +149,7 @@ func (s *ShardEthereum) registerP2P() error { // event feed. For our first releases, this can just relay test/fake transaction data // the proposer can serialize into collation blobs. // TODO: design this txpool system for our first release. -func (s *ShardEthereum) registerTXPool(actor string) error { - if actor != "proposer" { - return nil - } +func (s *ShardEthereum) registerTXPool() error { var shardp2p *p2p.Server if err := s.services.FetchService(&shardp2p); err != nil { return err @@ -182,22 +172,26 @@ func (s *ShardEthereum) registerBeaconService() error { return s.services.RegisterService(b) } -// registerActorService registers the actor according to CLI flags. Either attester/proposer. -func (s *ShardEthereum) registerActorService(actor string) error { +// registerAttesterService that listens to assignments from the beacon service. +func (s *ShardEthereum) registerAttesterService() error { var beaconService *beacon.Service if err := s.services.FetchService(&beaconService); err != nil { return err } - switch actor { - case "attester": - att := attester.NewAttester(context.TODO(), beaconService) - return s.services.RegisterService(att) - case "proposer": - prop := proposer.NewProposer(context.TODO(), beaconService) - return s.services.RegisterService(prop) + att := attester.NewAttester(context.TODO(), beaconService) + return s.services.RegisterService(att) +} + +// registerProposerService that listens to assignments from the beacon service. +func (s *ShardEthereum) registerProposerService() error { + var beaconService *beacon.Service + if err := s.services.FetchService(&beaconService); err != nil { + return err } - return nil + + prop := proposer.NewProposer(context.TODO(), beaconService) + return s.services.RegisterService(prop) } // registerRPCClientService registers a new RPC client that connects to a beacon node. diff --git a/client/rpcclient/service.go b/client/rpcclient/service.go index 1831deadd9d7..33da68b21974 100644 --- a/client/rpcclient/service.go +++ b/client/rpcclient/service.go @@ -1,8 +1,10 @@ -// Package rpcclient defines the services for the RPC connections of the client. +// Package rpcclient defines a gRPC connection to a beacon node. package rpcclient import ( "context" + "fmt" + "net/url" pb "github.com/prysmaticlabs/prysm/proto/beacon/rpc/v1" "github.com/sirupsen/logrus" @@ -53,7 +55,11 @@ func (s *Service) Start() { server = grpc.WithInsecure() log.Warn("You are using an insecure gRPC connection! Please provide a certificate and key to use a secure connection.") } - conn, err := grpc.Dial(s.endpoint, server) + providerURL, err := url.Parse(s.endpoint) + if err != nil { + log.Fatalf("Unable to parse beacon RPC provider endpoint url: %v", err) + } + conn, err := grpc.Dial(fmt.Sprintf("[%s]:%s", providerURL.Hostname(), providerURL.Port()), server) if err != nil { log.Errorf("Could not dial endpoint: %s, %v", s.endpoint, err) return diff --git a/client/types/flags.go b/client/types/flags.go index cf67c5d99186..9a019ee9ceed 100644 --- a/client/types/flags.go +++ b/client/types/flags.go @@ -5,11 +5,6 @@ import ( ) var ( - // ActorFlag defines the role of the sharding client. Either proposer, attester, or simulator. - ActorFlag = cli.StringFlag{ - Name: "actor", - Usage: `use the --actor attester or --actor proposer to start a attester or proposer service in the sharding node`, - } // BeaconRPCProviderFlag defines a beacon node RPC endpoint. BeaconRPCProviderFlag = cli.StringFlag{ Name: "beacon-rpc-provider", diff --git a/client/types/interfaces.go b/client/types/interfaces.go index e881b907bc83..bea0c682267e 100644 --- a/client/types/interfaces.go +++ b/client/types/interfaces.go @@ -3,17 +3,8 @@ package types import ( "github.com/ethereum/go-ethereum/common" pb "github.com/prysmaticlabs/prysm/proto/beacon/rpc/v1" - "github.com/prysmaticlabs/prysm/shared" ) -// Node defines a a sharding-enabled Ethereum instance that provides -// full control and shared access of necessary components and services -// for a sharded Ethereum blockchain. -type Node interface { - Start() - Close() -} - // BeaconClient defines a service that interacts with a beacon node via RPC to determine // attestation/proposal responsibilities. type BeaconClient interface { @@ -21,12 +12,6 @@ type BeaconClient interface { ProposerAssignment() <-chan bool } -// Actor refers to either a attester, proposer in the sharding spec. -type Actor interface { - shared.Service - // TODO: will actors have actor-specific methods? To be decided. -} - // CollationFetcher defines functionality for a struct that is able to extract // respond with collation information to the caller. Shard implements this interface. type CollationFetcher interface { diff --git a/contracts/validator-registration-contract/deployVRC/README.md b/contracts/validator-registration-contract/deployVRC/README.md index cd09146f2f92..0f130b4f60be 100644 --- a/contracts/validator-registration-contract/deployVRC/README.md +++ b/contracts/validator-registration-contract/deployVRC/README.md @@ -1,4 +1,4 @@ -## Utility to Deploy Validator Registraction Contract +## Utility to Deploy Validator Registration Contract This is a utility to help users deploy validator registration contract for running their own beacon chain node in a local containerized set up. To run the utility, it assumes there is a running geth node as a separate process attached to proof-of-work main chain. The utility will deploy the validator registration contract and print out the contract address. Users will pass the contract address to the beacon chain node to monitor when they have been conducted to become an active validator. @@ -20,7 +20,9 @@ This is a utility to help users deploy validator registration contract for runni **--version, -v** print the version ### Example + To use private key with default RPC: + ``` bazel run //deployVRC --privKey yourPrivateKey ``` @@ -32,14 +34,18 @@ bazel run //deployVRC --\ --UTCPath /path/to/your/keystore/UTCJSON \ --passwordFile /path/to/your/password.txt ``` + To use UTC JSON with RPC: + ``` bazel run //deployVRC --\ --httpPath http://localhost:8545/ \ --UTCPath /path/to/your/keystore/UTCJSON \ --passwordFile /path/to/your/password.txt ``` + or + ``` bazel run //deployVRC --\ --UTCPath /path/to/your/keystore/UTCJSON \ @@ -47,6 +53,7 @@ bazel run //deployVRC --\ ``` ### Output + ``` INFO[0001] New contract deployed at 0x5275C2220C574330E230bFB7e4a0b96f60a18f02 ```