Skip to content

Commit

Permalink
Merge pull request #3185 from ethereum/fc-is_data_available
Browse files Browse the repository at this point in the history
Move `is_data_available` check to fork-choice `on_block`
  • Loading branch information
djrtwo authored Jan 6, 2023
2 parents 93ebf44 + 4ba2266 commit ed1c6be
Show file tree
Hide file tree
Showing 7 changed files with 139 additions and 57 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Features are researched and developed in parallel, and then consolidated into se
| Code Name or Topic | Specs | Notes |
| - | - | - |
| Capella (tentative) | <ul><li>Core</li><ul><li>[Beacon chain changes](specs/capella/beacon-chain.md)</li><li>[Capella fork](specs/capella/fork.md)</li></ul><li>Additions</li><ul><li>[Validator additions](specs/capella/validator.md)</li><li>[P2P networking](specs/capella/p2p-interface.md)</li></ul></ul> |
| EIP4844 (tentative) | <ul><li>Core</li><ul><li>[Beacon Chain changes](specs/eip4844/beacon-chain.md)</li><li>[EIP-4844 fork](specs/eip4844/fork.md)</li><li>[Polynomial commitments](specs/eip4844/polynomial-commitments.md)</li></ul><li>Additions</li><ul><li>[Honest validator guide changes](specs/eip4844/validator.md)</li><li>[P2P networking](specs/eip4844/p2p-interface.md)</li></ul></ul> |
| EIP4844 (tentative) | <ul><li>Core</li><ul><li>[Beacon Chain changes](specs/eip4844/beacon-chain.md)</li><li>[EIP-4844 fork](specs/eip4844/fork.md)</li><li>[Polynomial commitments](specs/eip4844/polynomial-commitments.md)</li><li>[Fork choice changes](specs/eip4844/fork-choice.md)</li></ul><li>Additions</li><ul><li>[Honest validator guide changes](specs/eip4844/validator.md)</li><li>[P2P networking](specs/eip4844/p2p-interface.md)</li></ul></ul> |
| Sharding (outdated) | <ul><li>Core</li><ul><li>[Beacon Chain changes](specs/sharding/beacon-chain.md)</li></ul><li>Additions</li><ul><li>[P2P networking](specs/sharding/p2p-interface.md)</li></ul></ul> |
| Custody Game (outdated) | <ul><li>Core</li><ul><li>[Beacon Chain changes](specs/custody_game/beacon-chain.md)</li></ul><li>Additions</li><ul><li>[Honest validator guide changes](specs/custody_game/validator.md)</li></ul></ul> | Dependent on sharding |
| Data Availability Sampling (outdated) | <ul><li>Core</li><ul><li>[Core types and functions](specs/das/das-core.md)</li><li>[Fork choice changes](specs/das/fork-choice.md)</li></ul><li>Additions</li><ul><li>[P2P Networking](specs/das/p2p-interface.md)</li><li>[Sampling process](specs/das/sampling.md)</li></ul></ul> | <ul><li> Dependent on sharding</li><li>[Technical explainer](https://hackmd.io/@HWeNw8hNRimMm2m2GH56Cw/B1YJPGkpD)</li></ul> |
Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -992,6 +992,7 @@ def finalize_options(self):
self.md_doc_paths += """
specs/eip4844/beacon-chain.md
specs/eip4844/fork.md
specs/eip4844/fork-choice.md
specs/eip4844/polynomial-commitments.md
specs/eip4844/p2p-interface.md
specs/eip4844/validator.md
Expand Down
45 changes: 0 additions & 45 deletions specs/eip4844/beacon-chain.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,6 @@
- [`ExecutionPayloadHeader`](#executionpayloadheader)
- [Helper functions](#helper-functions)
- [Misc](#misc)
- [`validate_blobs_sidecar`](#validate_blobs_sidecar)
- [`is_data_available`](#is_data_available)
- [`kzg_commitment_to_versioned_hash`](#kzg_commitment_to_versioned_hash)
- [`tx_peek_blob_versioned_hashes`](#tx_peek_blob_versioned_hashes)
- [`verify_kzg_commitments_against_transactions`](#verify_kzg_commitments_against_transactions)
Expand Down Expand Up @@ -145,46 +143,6 @@ class ExecutionPayloadHeader(Container):

### Misc

#### `validate_blobs_sidecar`

```python
def validate_blobs_sidecar(slot: Slot,
beacon_block_root: Root,
expected_kzg_commitments: Sequence[KZGCommitment],
blobs_sidecar: BlobsSidecar) -> None:
assert slot == blobs_sidecar.beacon_block_slot
assert beacon_block_root == blobs_sidecar.beacon_block_root
blobs = blobs_sidecar.blobs
kzg_aggregated_proof = blobs_sidecar.kzg_aggregated_proof
assert len(expected_kzg_commitments) == len(blobs)

assert verify_aggregate_kzg_proof(blobs, expected_kzg_commitments, kzg_aggregated_proof)
```

#### `is_data_available`

The implementation of `is_data_available` will become more sophisticated during later sharding upgrades.
Initially, it requires every verifying actor to retrieve the matching `BlobsSidecar`,
and validate the sidecar with `validate_blobs_sidecar`.

The block MUST NOT be considered valid until a valid `BlobsSidecar` has been downloaded. Blocks that have been previously validated as available SHOULD be considered available even if the associated `BlobsSidecar` has subsequently been pruned.

```python
def is_data_available(slot: Slot, beacon_block_root: Root, blob_kzg_commitments: Sequence[KZGCommitment]) -> bool:
# `retrieve_blobs_sidecar` is implementation and context dependent, raises an exception if not available.
# Note: the p2p network does not guarantee sidecar retrieval outside of `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS`
sidecar = retrieve_blobs_sidecar(slot, beacon_block_root)

# For testing, `retrieve_blobs_sidecar` returns "TEST".
# TODO: Remove it once we have a way to inject `BlobsSidecar` into tests.
if isinstance(sidecar, str):
return True

validate_blobs_sidecar(slot, beacon_block_root, blob_kzg_commitments, sidecar)
return True
```


#### `kzg_commitment_to_versioned_hash`

```python
Expand Down Expand Up @@ -240,9 +198,6 @@ def process_block(state: BeaconState, block: BeaconBlock) -> None:
process_operations(state, block.body)
process_sync_aggregate(state, block.body.sync_aggregate)
process_blob_kzg_commitments(state, block.body) # [New in EIP-4844]

# New in EIP-4844
assert is_data_available(block.slot, hash_tree_root(block), block.body.blob_kzg_commitments)
```

#### Execution payload
Expand Down
137 changes: 137 additions & 0 deletions specs/eip4844/fork-choice.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# EIP-4844 -- Fork Choice

## Table of contents
<!-- TOC -->
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

- [Introduction](#introduction)
- [Containers](#containers)
- [`BlobsSidecar`](#blobssidecar)
- [Helpers](#helpers)
- [`validate_blobs_sidecar`](#validate_blobs_sidecar)
- [`is_data_available`](#is_data_available)
- [Updated fork-choice handlers](#updated-fork-choice-handlers)
- [`on_block`](#on_block)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
<!-- /TOC -->

## Introduction

This is the modification of the fork choice accompanying the EIP-4844 upgrade.

## Containers

### `BlobsSidecar`

```python
class BlobsSidecar(Container):
beacon_block_root: Root
beacon_block_slot: Slot
blobs: List[Blob, MAX_BLOBS_PER_BLOCK]
kzg_aggregated_proof: KZGProof
```

## Helpers

#### `validate_blobs_sidecar`

```python
def validate_blobs_sidecar(slot: Slot,
beacon_block_root: Root,
expected_kzg_commitments: Sequence[KZGCommitment],
blobs_sidecar: BlobsSidecar) -> None:
assert slot == blobs_sidecar.beacon_block_slot
assert beacon_block_root == blobs_sidecar.beacon_block_root
blobs = blobs_sidecar.blobs
kzg_aggregated_proof = blobs_sidecar.kzg_aggregated_proof
assert len(expected_kzg_commitments) == len(blobs)

assert verify_aggregate_kzg_proof(blobs, expected_kzg_commitments, kzg_aggregated_proof)
```

#### `is_data_available`

The implementation of `is_data_available` will become more sophisticated during later scaling upgrades.
Initially, verification requires every verifying actor to retrieve the matching `BlobsSidecar`,
and validate the sidecar with `validate_blobs_sidecar`.

The block MUST NOT be considered valid until a valid `BlobsSidecar` has been downloaded. Blocks that have been previously validated as available SHOULD be considered available even if the associated `BlobsSidecar` has subsequently been pruned.

```python
def is_data_available(slot: Slot, beacon_block_root: Root, blob_kzg_commitments: Sequence[KZGCommitment]) -> bool:
# `retrieve_blobs_sidecar` is implementation and context dependent, raises an exception if not available.
# Note: the p2p network does not guarantee sidecar retrieval outside of `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS`
sidecar = retrieve_blobs_sidecar(slot, beacon_block_root)

# For testing, `retrieve_blobs_sidecar` returns "TEST".
# TODO: Remove it once we have a way to inject `BlobsSidecar` into tests.
if isinstance(sidecar, str):
return True

validate_blobs_sidecar(slot, beacon_block_root, blob_kzg_commitments, sidecar)
return True
```

## Updated fork-choice handlers

### `on_block`

*Note*: The only modification is the addition of the verification of transition block conditions.

```python
def on_block(store: Store, signed_block: SignedBeaconBlock) -> None:
"""
Run ``on_block`` upon receiving a new block.
"""
block = signed_block.message
# Parent block must be known
assert block.parent_root in store.block_states
# Make a copy of the state to avoid mutability issues
pre_state = copy(store.block_states[block.parent_root])
# Blocks cannot be in the future. If they are, their consideration must be delayed until they are in the past.
assert get_current_slot(store) >= block.slot

# Check that block is later than the finalized epoch slot (optimization to reduce calls to get_ancestor)
finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch)
assert block.slot > finalized_slot
# Check block is a descendant of the finalized block at the checkpoint finalized slot
assert get_ancestor(store, block.parent_root, finalized_slot) == store.finalized_checkpoint.root

# [New in EIP-4844]
# Check if blob data is available
# If not, this block MAY be queued and subsequently considered when blob data becomes available
assert is_data_available(block.slot, hash_tree_root(block), block.body.blob_kzg_commitments)

# Check the block is valid and compute the post-state
state = pre_state.copy()
state_transition(state, signed_block, True)

# Check the merge transition
if is_merge_transition_block(pre_state, block.body):
validate_merge_block(block)

# Add new block to the store
store.blocks[hash_tree_root(block)] = block
# Add new state for this block to the store
store.block_states[hash_tree_root(block)] = state

# Add proposer score boost if the block is timely
time_into_slot = (store.time - store.genesis_time) % SECONDS_PER_SLOT
is_before_attesting_interval = time_into_slot < SECONDS_PER_SLOT // INTERVALS_PER_SLOT
if get_current_slot(store) == block.slot and is_before_attesting_interval:
store.proposer_boost_root = hash_tree_root(block)

# Update justified checkpoint
if state.current_justified_checkpoint.epoch > store.justified_checkpoint.epoch:
if state.current_justified_checkpoint.epoch > store.best_justified_checkpoint.epoch:
store.best_justified_checkpoint = state.current_justified_checkpoint
if should_update_justified_checkpoint(store, state.current_justified_checkpoint):
store.justified_checkpoint = state.current_justified_checkpoint

# Update finalized checkpoint
if state.finalized_checkpoint.epoch > store.finalized_checkpoint.epoch:
store.finalized_checkpoint = state.finalized_checkpoint
store.justified_checkpoint = state.current_justified_checkpoint
```
11 changes: 0 additions & 11 deletions specs/eip4844/p2p-interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ The specification of these changes continues in the same format as the network s

- [Configuration](#configuration)
- [Containers](#containers)
- [`BlobsSidecar`](#blobssidecar)
- [`SignedBeaconBlockAndBlobsSidecar`](#signedbeaconblockandblobssidecar)
- [The gossip domain: gossipsub](#the-gossip-domain-gossipsub)
- [Topics and messages](#topics-and-messages)
Expand Down Expand Up @@ -41,16 +40,6 @@ The specification of these changes continues in the same format as the network s

## Containers

### `BlobsSidecar`

```python
class BlobsSidecar(Container):
beacon_block_root: Root
beacon_block_slot: Slot
blobs: List[Blob, MAX_BLOBS_PER_BLOCK]
kzg_aggregated_proof: KZGProof
```

### `SignedBeaconBlockAndBlobsSidecar`

```python
Expand Down

0 comments on commit ed1c6be

Please sign in to comment.