Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: sync committee proof server #100

Merged
merged 4 commits into from
Jul 17, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions ethereum/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions ethereum/core/Cargo.toml
Original file line number Diff line number Diff line change
@@ -11,6 +11,7 @@ anyhow = { workspace = true }
bls12_381 = { workspace = true, features = ["experimental"] }
hex = { workspace = true }
getset = { workspace = true }
serde = { workspace = true, features = ["derive"] }
sha2 = { workspace = true }
thiserror = { workspace = true }
tiny-keccak = { workspace = true, features = ["keccak"] }
2 changes: 1 addition & 1 deletion ethereum/core/src/crypto/sig.rs
Original file line number Diff line number Diff line change
@@ -274,7 +274,7 @@ pub const SYNC_AGGREGATE_BYTES_LEN: usize = SYNC_COMMITTEE_SIZE / 8 + SIG_LEN;
/// index who signed the message as bits and the actual signature.
///
/// From [the Altaïr specifications](https://github.com/ethereum/consensus-specs/blob/81f3ea8322aff6b9fb15132d050f8f98b16bdba4/specs/altair/beacon-chain.md#syncaggregate).
#[derive(Debug, Clone, Getters)]
#[derive(Debug, Clone, Eq, PartialEq, Getters)]
#[getset(get = "pub")]
pub struct SyncAggregate {
sync_committee_bits: [u8; SYNC_COMMITTEE_SIZE],
3 changes: 2 additions & 1 deletion ethereum/core/src/types/store.rs
Original file line number Diff line number Diff line change
@@ -180,7 +180,8 @@ impl LightClientStore {
.sync_aggregate()
.sync_committee_bits()
.iter()
.sum::<u8>()
.map(|&bit| u64::from(bit))
.sum::<u64>()
< 1
{
return Err(ConsensusError::InsufficientSigners);
2 changes: 1 addition & 1 deletion ethereum/core/src/types/update.rs
Original file line number Diff line number Diff line change
@@ -30,7 +30,7 @@ pub const UPDATE_BASE_BYTES_LEN: usize = LIGHT_CLIENT_HEADER_BASE_BYTES_LEN * 2
/// A data structure containing the necessary data for a light client to update its state from the Beacon chain.
///
/// From [the Altaïr specifications](https://github.com/ethereum/consensus-specs/blob/81f3ea8322aff6b9fb15132d050f8f98b16bdba4/specs/altair/light-client/sync-protocol.md#lightclientupdate).
#[derive(Debug, Clone, Getters)]
#[derive(Debug, Clone, Eq, PartialEq, Getters)]
#[getset(get = "pub")]
pub struct Update {
attested_header: LightClientHeader,
2 changes: 1 addition & 1 deletion ethereum/docs/src/run/setup_client.md
Original file line number Diff line number Diff line change
@@ -10,5 +10,5 @@ With our deployment machine properly configured, we can run the client.
```bash
git clone [email protected]:lurk-lab/zk-light-clients.git && \
cd zk-light-clients/ethereum && \
RUST_LOG="debug" cargo +nightly-2024-05-31 run -p light-client --release --bin client -- -c <CHECKPOINT_PROVIDER_ADDRESS> -b <BEACON_NODE_ADDRESS>
RUST_LOG="debug" cargo +nightly-2024-05-31 run -p light-client --release --bin client -- -c <CHECKPOINT_PROVIDER_ADDRESS> -b <BEACON_NODE_ADDRESS> -p <PROOF_SERVER_ADDRESS>
```
43 changes: 42 additions & 1 deletion ethereum/docs/src/run/setup_proof_server.md
Original file line number Diff line number Diff line change
@@ -1 +1,42 @@
# Deploy the Proof Server
# Deploy the Proof Server

We have two components to deploy for the Proof Server to work as intended. The primary and the secondary server.
There is no particular order in which they should be deployed, but here we will deploy the secondary and then
the primary.

For best results, the primary and secondary servers should be deployed to **different server instances**, so that
proof generation can happen in parallel if necessary.

## Requirements

Make sure to finish the [initial configuration](./configuration.md) first.

## Environment variables

- `RUSTFLAGS="-C target-cpu=native -C opt-level=3"`:
- `-C target-cpu=native`: This will ensure that the binary is optimized
for the CPU it is running on. This is very important
for [plonky3](https://github.com/plonky3/plonky3?tab=readme-ov-file#cpu-features) performance.
- `-C opt-level=3`: This turns on the maximum level of compiler optimizations.
- This can also be configured in `~/.cargo/config.toml` instead by adding:
```toml
[target.'cfg(all())']
rustflags = ["-C", "target-cpu=native", "-C", "opt-level=3"]
```

Make sure to launch the proof servers with `cargo +nightly-2024-05-31`.

> **Note**
>
> One can also set the `RUST_LOG` environment variable to `debug` to get more information
> about the execution of the server.

## Deploy the primary server

Finally, once the primary server is configured in the same fashion, run it:

```bash
git clone [email protected]:lurk-lab/zk-light-clients.git && \
cd zk-light-clients/ethereum/light-client && \
SHARD_BATCH_SIZE=0 RUSTFLAGS="-C target-cpu=native -C opt-level=3" cargo +nightly-2024-05-31 run --release --bin server_primary -- -a <NETWORK_ADDESS> --snd-addr <SECONDARY_SERVER_ADDRESS>
```
Binary file modified ethereum/ethereum-programs/artifacts/committee-change-program
Binary file not shown.
4 changes: 1 addition & 3 deletions ethereum/light-client/Cargo.toml
Original file line number Diff line number Diff line change
@@ -14,6 +14,7 @@ hex = { workspace = true }
log = { workspace = true }
reqwest = { workspace = true, features = ["json"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true, features = ["full"] }
# LC crates
@@ -22,9 +23,6 @@ ethereum-programs = { path = "../ethereum-programs" }
# Sphinx crates
sphinx-sdk = { workspace = true }

[dev-dependencies]
serde_json = { workspace = true }

[[bin]]
name = "client"
path = "src/bin/client.rs"
93 changes: 59 additions & 34 deletions ethereum/light-client/src/bin/client.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
// Copyright (c) Yatima, Inc.
// SPDX-License-Identifier: Apache-2.0

use anyhow::Result;
use clap::Parser;
use ethereum_lc_core::merkle::Merkleized;
use ethereum_lc::client::Client;
use ethereum_lc::proofs::ProvingMode;
use ethereum_lc_core::types::store::LightClientStore;
use ethereum_lc_core::types::utils::calc_sync_period;
use log::info;
use std::sync::Arc;

/// The maximum number of light client updates that can be requested.
///
@@ -26,22 +29,52 @@ struct Cli {
/// It is recommended to use https://www.lightclientdata.org
#[arg(short, long)]
beacon_node_address: String,

/// The address of the proof server
#[arg(short, long)]
proof_server_address: String,
}

pub struct ClientState {
client: Client,
store: LightClientStore,
}

#[tokio::main]
async fn main() {
let Cli {
checkpoint_provider_address,
beacon_node_address,
proof_server_address,
..
} = Cli::parse();

// Initialize the logger.
env_logger::init();

let checkpoint_provider_address = Arc::new(checkpoint_provider_address);
let beacon_node_address = Arc::new(beacon_node_address);
let proof_server_address = Arc::new(proof_server_address);

let _state = initialize_light_client(
checkpoint_provider_address,
beacon_node_address,
proof_server_address,
)
.await;
}

async fn initialize_light_client(
checkpoint_provider_address: Arc<String>,
beacon_node_address: Arc<String>,
proof_server_address: Arc<String>,
) -> Result<ClientState> {
// Instantiate client.
let client =
ethereum_lc::client::Client::new(&checkpoint_provider_address, &beacon_node_address);
let client = Client::new(
&checkpoint_provider_address,
&beacon_node_address,
&proof_server_address,
);

info!("Fetching latest state checkpoint and bootstrap data...");

@@ -79,15 +112,18 @@ async fn main() {
.try_into()
.expect("Failed to convert checkpoint bytes to Bytes32");

let mut store = LightClientStore::initialize(trusted_block_root, &bootstrap)
let store = LightClientStore::initialize(trusted_block_root, &bootstrap)
.expect("Could not initialize the store based on bootstrap data");

let mut client_state = ClientState { client, store };

info!("Fetching updates...");

// Fetch updates
let sync_period = calc_sync_period(bootstrap.header().beacon().slot());

let update_response = client
let update_response = client_state
.client
.get_update_data(sync_period, MAX_REQUEST_LIGHT_CLIENT_UPDATES)
.await
.expect("Failed to fetch update data");
@@ -109,35 +145,24 @@ async fn main() {
info!("Sync period changed, updating store...");
}

store
let proof = client_state
.client
.prove_committee_change(ProvingMode::STARK, &client_state.store, update.update())
.await
.expect("Failed to prove committee change");

client_state
.client
.verify_committee_change(proof.clone())
.await
.expect("Failed to prove committee change");

// TODO this is redundant, to simplify
client_state
.store
.process_light_client_update(update.update())
.expect("Failed to process update");

assert_eq!(
store
.next_sync_committee()
.clone()
.unwrap()
.hash_tree_root()
.unwrap(),
update
.update()
.next_sync_committee()
.hash_tree_root()
.unwrap()
);

if calc_sync_period(bootstrap.header().beacon().slot())
!= calc_sync_period(update.update().attested_header().beacon().slot())
{
assert_eq!(
store.finalized_header().hash_tree_root().unwrap(),
update.update().finalized_header().hash_tree_root().unwrap()
);
assert_eq!(
store.optimistic_header().hash_tree_root().unwrap(),
update.update().finalized_header().hash_tree_root().unwrap()
)
}
.unwrap()
}

Ok(client_state)
}
76 changes: 74 additions & 2 deletions ethereum/light-client/src/bin/server_primary.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,77 @@
// Copyright (c) Yatima, Inc.
// SPDX-License-Identifier: Apache-2.0

#[allow(clippy::missing_const_for_fn)]
fn main() {}
use anyhow::{Error, Result};
use clap::Parser;
use ethereum_lc::proofs::committee_change::CommitteeChangeProver;
use ethereum_lc::proofs::Prover;
use ethereum_lc::types::network::Request;
use ethereum_lc::utils::{read_bytes, write_bytes};
use log::{error, info};
use std::sync::Arc;
use tokio::net::TcpListener;
use tokio::task::spawn_blocking;

#[derive(Parser)]
struct Cli {
/// Address of this server. E.g. 127.0.0.1:1234
#[arg(short, long)]
addr: String,

/// Address of the secondary server. E.g. 127.0.0.1:4321
#[arg(long)]
snd_addr: String,
}

#[tokio::main]
async fn main() -> Result<()> {
let Cli { addr, .. } = Cli::parse();

env_logger::init();

let listener = TcpListener::bind(addr).await?;
info!("Server is running on {}", listener.local_addr()?);

let committee_prover = Arc::new(CommitteeChangeProver::new());

loop {
let (mut client_stream, _) = listener.accept().await?;
info!("Received a connection");

let committee_prover = committee_prover.clone();

tokio::spawn(async move {
info!("Awaiting request");
let request_bytes = read_bytes(&mut client_stream).await?;
info!("Request received");

info!("Deserializing request");
match Request::from_bytes(&request_bytes) {
Ok(request) => match request {
Request::ProveCommitteeChange(boxed) => {
info!("Start proving");
let proof_handle = spawn_blocking(move || {
let (proving_mode, inputs) = boxed.as_ref();
committee_prover.prove(inputs.clone(), proving_mode.clone())
});
let proof = proof_handle.await??;
info!("Proof generated. Serializing");
let proof_bytes = proof.to_bytes()?;
info!("Sending proof");
write_bytes(&mut client_stream, &proof_bytes).await?;
info!("Proof sent");
}
Request::VerifyCommitteeChange(proof) => {
write_bytes(
&mut client_stream,
&[u8::from(committee_prover.verify(&proof).is_ok())],
)
.await?;
}
},
Err(err) => error!("Failed to deserialize request object: {err}"),
}
Ok::<(), Error>(())
});
}
}
Loading