[Merged by Bors] - Implement standard keystore API

book/src/api-vc-auth-header.md

@@ -6,13 +6,13 @@ The validator client HTTP server requires that all requests have the following
 HTTP header:
 
 - Name: `Authorization`
-- Value: `Basic <api-token>`
+- Value: `Bearer <api-token>`
 
 Where `<api-token>` is a string that can be obtained from the validator client
 host. Here is an example `Authorization` header:
 
 ```
-Authorization Basic api-token-0x03eace4c98e8f77477bb99efb74f9af10d800bd3318f92c33b719a4644254d4123
+Authorization: Bearer api-token-0x03eace4c98e8f77477bb99efb74f9af10d800bd3318f92c33b719a4644254d4123
 ```
 
 ## Obtaining the API token
@@ -35,12 +35,27 @@ to the file containing the api token.
 Sep 28 19:17:52.615 INFO HTTP API started                        api_token_file: "$HOME/prater/validators/api-token.txt", listen_address: 127.0.0.1:5062
 ```
 
+The _path_ to the API token may also be fetched from the HTTP API itself (this endpoint is the only
+one accessible without the token):
+
+```bash
+curl http://localhost:5062/lighthouse/auth
+```
+
+Response:
+
+```json
+{
+    "token_path": "/home/karlm/.lighthouse/prater/validators/api-token.txt"
+}
+```
+
 ## Example
 
 Here is an example `curl` command using the API token in the `Authorization` header:
 
 ```bash
-curl localhost:5062/lighthouse/version -H "Authorization: Basic api-token-0x03eace4c98e8f77477bb99efb74f9af10d800bd3318f92c33b719a4644254d4123"
+curl localhost:5062/lighthouse/version -H "Authorization: Bearer api-token-0x03eace4c98e8f77477bb99efb74f9af10d800bd3318f92c33b719a4644254d4123"
 ```
 
 The server should respond with its version:

book/src/api-vc-endpoints.md

@@ -4,15 +4,19 @@
 
 HTTP Path | Description |
 | --- | -- |
-[`GET /lighthouse/version`](#get-lighthouseversion) | Get the Lighthouse software version
-[`GET /lighthouse/health`](#get-lighthousehealth) | Get information about the host machine
-[`GET /lighthouse/spec`](#get-lighthousespec) | Get the Eth2 specification used by the validator
-[`GET /lighthouse/validators`](#get-lighthousevalidators) | List all validators
-[`GET /lighthouse/validators/:voting_pubkey`](#get-lighthousevalidatorsvoting_pubkey) | Get a specific validator
-[`PATCH /lighthouse/validators/:voting_pubkey`](#patch-lighthousevalidatorsvoting_pubkey) | Update a specific validator
+[`GET /lighthouse/version`](#get-lighthouseversion) | Get the Lighthouse software version.
+[`GET /lighthouse/health`](#get-lighthousehealth) | Get information about the host machine.
+[`GET /lighthouse/spec`](#get-lighthousespec) | Get the Eth2 specification used by the validator.
+[`GET /lighthouse/auth`](#get-lighthouseauth) | Get the location of the authorization token.
+[`GET /lighthouse/validators`](#get-lighthousevalidators) | List all validators.
+[`GET /lighthouse/validators/:voting_pubkey`](#get-lighthousevalidatorsvoting_pubkey) | Get a specific validator.
+[`PATCH /lighthouse/validators/:voting_pubkey`](#patch-lighthousevalidatorsvoting_pubkey) | Update a specific validator.
 [`POST /lighthouse/validators`](#post-lighthousevalidators) | Create a new validator and mnemonic.
 [`POST /lighthouse/validators/keystore`](#post-lighthousevalidatorskeystore) | Import a keystore.
 [`POST /lighthouse/validators/mnemonic`](#post-lighthousevalidatorsmnemonic) | Create a new validator from an existing mnemonic.
+[`POST /lighthouse/validators/web3signer`](#post-lighthousevalidatorsweb3signer) | Add web3signer validators.
+
+In addition to the above endpoints Lighthouse also supports all of the [standard keymanager APIs](https://ethereum.github.io/keymanager-APIs/).
 
 ## `GET /lighthouse/version`
 
@@ -153,6 +157,37 @@ Typical Responses | 200
 }
 ```
 
+## `GET /lighthouse/auth`
+
+Fetch the filesystem path of the [authorization token](./api-vc-auth-header.md).
+Unlike the other endpoints this may be called _without_ providing an authorization token.
+
+This API is intended to be called from the same machine as the validator client, so that the token
+file may be read by a local user with access rights.
+
+### HTTP Specification
+
+| Property | Specification |
+| --- |--- |
+Path | `/lighthouse/auth`
+Method | GET
+Required Headers | -
+Typical Responses | 200
+
+### Example Path
+
+```
+localhost:5062/lighthouse/auth
+```
+
+### Example Response Body
+
+```json
+{
+    "token_path": "/home/karlm/.lighthouse/prater/validators/api-token.txt"
+}
+```
+
 ## `GET /lighthouse/validators`
 
 Lists all validators managed by this validator client.

book/src/api-vc.md

@@ -1,9 +1,12 @@
 # Validator Client API
 
-Lighthouse implements a HTTP/JSON API for the validator client. Since there is
-no Eth2 standard validator client API, Lighthouse has defined its own.
+Lighthouse implements a JSON HTTP API for the validator client which enables programmatic management
+of validators and keys.
 
-A full list of endpoints can be found in [Endpoints](./api-vc-endpoints.md).
+The API includes all of the endpoints from the [standard keymanager
+API](https://ethereum.github.io/keymanager-APIs/) that is implemented by other clients and remote
+signers. It also includes some Lighthouse-specific endpoints which are described in
+[Endpoints](./api-vc-endpoints.md).
 
 > Note: All requests to the HTTP server must supply an
 > [`Authorization`](./api-vc-auth-header.md) header. All responses contain a

common/account_utils/src/lib.rs

@@ -85,15 +85,23 @@ pub fn write_file_via_temporary(
     Ok(())
 }
 
-/// Generates a random alphanumeric password of length `DEFAULT_PASSWORD_LEN`.
+/// Generates a random alphanumeric password of length `DEFAULT_PASSWORD_LEN` as `PlainText`.
 pub fn random_password() -> PlainText {
+    random_password_raw_string().into_bytes().into()
+}
+
+/// Generates a random alphanumeric password of length `DEFAULT_PASSWORD_LEN` as `ZeroizeString`.
+pub fn random_password_string() -> ZeroizeString {
+    random_password_raw_string().into()
+}
+
+/// Common implementation for `random_password` and `random_password_string`.
+fn random_password_raw_string() -> String {
     rand::thread_rng()
         .sample_iter(&Alphanumeric)
         .take(DEFAULT_PASSWORD_LEN)
         .map(char::from)
-        .collect::<String>()
-        .into_bytes()
-        .into()
+        .collect()
 }
 
 /// Remove any number of newline or carriage returns from the end of a vector of bytes.

common/account_utils/src/validator_definitions.rs

@@ -46,9 +46,6 @@ pub enum Error {
 }
 
 /// Defines how the validator client should attempt to sign messages for this validator.
-///
-/// Presently there is only a single variant, however we expect more variants to arise (e.g.,
-/// remote signing).
 #[derive(Clone, PartialEq, Serialize, Deserialize)]
 #[serde(tag = "type")]
 pub enum SigningDefinition {
@@ -78,6 +75,12 @@ pub enum SigningDefinition {
     },
 }
 
+impl SigningDefinition {
+    pub fn is_local_keystore(&self) -> bool {
+        matches!(self, SigningDefinition::LocalKeystore { .. })
+    }
+}
+
 /// A validator that may be initialized by this validator client.
 ///
 /// Presently there is only a single variant, however we expect more variants to arise (e.g.,
@@ -293,6 +296,11 @@ impl ValidatorDefinitions {
         Ok(())
     }
 
+    /// Retain only the definitions matching the given predicate.
+    pub fn retain(&mut self, f: impl FnMut(&ValidatorDefinition) -> bool) {
+        self.0.retain(f);
+    }
+
     /// Adds a new `ValidatorDefinition` to `self`.
     pub fn push(&mut self, def: ValidatorDefinition) {
         self.0.push(def)

common/eth2/Cargo.toml

@@ -25,6 +25,7 @@ eth2_ssz_derive = "0.3.0"
 futures-util = "0.3.8"
 futures = "0.3.8"
 store = { path = "../../beacon_node/store", optional = true }
+slashing_protection = { path = "../../validator_client/slashing_protection", optional = true }
 
 [target.'cfg(target_os = "linux")'.dependencies]
 # TODO: update psutil once fix is merged: https://github.com/rust-psutil/rust-psutil/pull/93
@@ -35,4 +36,4 @@ procinfo = { version = "0.4.2", optional = true }
 
 [features]
 default = ["lighthouse"]
-lighthouse = ["proto_array", "psutil", "procinfo", "store"]
+lighthouse = ["proto_array", "psutil", "procinfo", "store", "slashing_protection"]

common/eth2/src/lib.rs

@@ -28,6 +28,7 @@ use serde::{de::DeserializeOwned, Serialize};
 use std::convert::TryFrom;
 use std::fmt;
 use std::iter::Iterator;
+use std::path::PathBuf;
 use std::time::Duration;
 
 pub const V1: EndpointVersion = EndpointVersion(1);
@@ -59,6 +60,12 @@ pub enum Error {
     InvalidServerSentEvent(String),
     /// The server returned an invalid SSZ response.
     InvalidSsz(ssz::DecodeError),
+    /// An I/O error occurred while loading an API token from disk.
+    TokenReadError(PathBuf, std::io::Error),
+    /// The client has been configured without a server pubkey, but requires one for this request.
+    NoServerPubkey,
+    /// The client has been configured without an API token, but requires one for this request.
+    NoToken,
 }
 
 impl From<reqwest::Error> for Error {
@@ -82,6 +89,8 @@ impl Error {
             Error::InvalidJson(_) => None,
             Error::InvalidServerSentEvent(_) => None,
             Error::InvalidSsz(_) => None,
+            Error::TokenReadError(..) => None,
+            Error::NoServerPubkey | Error::NoToken => None,
         }
     }
 }