Skip to content

Commit

Permalink
Merge branch 'main' into bouncy-castle-force-jdk18
Browse files Browse the repository at this point in the history
  • Loading branch information
fab-10 authored Oct 12, 2023
2 parents a77b09a + 1c293fe commit 31db6df
Show file tree
Hide file tree
Showing 157 changed files with 6,071 additions and 4,726 deletions.
35 changes: 34 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,37 @@
# Changelog

## Next release
## Next Release

### Breaking Changes

### Additions and Improvements

### Bug Fixes

### Download Links

## 23.10.0
### Layered Transaction Pool: the new default transaction pool implementation
With this release the previously experimental Layered txpool is marked stable and enabled by default, so please read the following instructions if you used to tune txpool behaviour,
otherwise you can simply go with the default and enjoy the improved performance of the new txpool.

#### Upgrading to Layered Transaction Pool
If you do not specify any txpool option, then you can skip this section.
If you have tuned the txpool using one of these options: `tx-pool-retention-hours`, `tx-pool-limit-by-account-percentage` or `tx-pool-max-size`,
then you need to update your configuration as described below:
- `tx-pool-retention-hours`: simply remove it, since it is not applicable in the Layered txpool, old transactions will eventually expire when the memory cache is full.
- `tx-pool-limit-by-account-percentage`: replace it with `tx-pool-max-future-by-sender`, which specify the max number of sequential transactions of single sender are kept in the txpool, by default it is 200.
- `tx-pool-max-size`: the Layered txpool is not limited by a max number of transactions, but by the estimated memory size the transactions occupy, so you need to remove this option, and to tune the max amount of memory<sup>*</sup> use the new option `tx-pool-layer-max-capacity` as described below.

You can still opt-out of the Layered txpool, setting `tx-pool=legacy` in config file or via cli argument, but be warned that the Legacy implementation will be deprecated for removal soon, so start testing the new implementation.

#### Configuring the Layered Transaction Pool
By default, the txpool is tuned for mainnet usage, but if you are using private networks or want to otherwise tune it, these are the new options:
- `tx-pool-max-future-by-sender`: specify the max number of sequential transactions of a single sender are kept in the txpool, by default it is 200, increase it to allow a single sender to fit more transactions in a single block. For private networks, this can safely be set in the hundreds or thousands if you want to ensure future transactions (with large nonce gaps) remain in the pool.
- `tx-pool-layer-max-capacity`: set the max amount of memory<sup>*</sup> in bytes, a single memory limited layer can occupy, by default is 12.5MB, keep in mind that there are 2 memory limited layers, so the expected memory consumption is twice the value specified by this option, so 25MB by default. Increase this value if you have spare RAM and the eviction rate is high for your network.
- `tx-pool-max-prioritized`: set the max number of transactions allowed in the first layer, that only contains transactions that are candidate for inclusion in the next block creation task. It makes sense to limit the value to the max number of transactions that fit in a block in your network, by default is 2000.

<sup>*</sup>: the memory used by the txpool is an estimation, we are working to make it always more accurate.

### Breaking Changes
- Removed support for Kotti network (ETC) [#5816](https://github.com/hyperledger/besu/pull/5816)
Expand All @@ -21,7 +52,9 @@
- Don't put control characters, escaped or otherwise, in t8n stacktraces [#5910](https://github.com/hyperledger/besu/pull/5910)

### Download Links
https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/23.10.0/besu-23.10.0.tar.gz / sha256: 3c75f3792bfdb0892705b378f0b8bfc14ef6cecf1d8afe711d8d8687ed6687cf

https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/23.10.0/besu-23.10.0.zip / sha256: d5dafff4c3cbf104bf75b34a9f108dcdd7b08d2759de75ec65cd997f38f52866

## 23.7.3

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
import org.hyperledger.besu.plugin.services.StorageService;
import org.hyperledger.besu.plugin.services.TransactionSelectionService;
import org.hyperledger.besu.plugin.services.storage.rocksdb.RocksDBPlugin;
import org.hyperledger.besu.plugin.services.txselection.TransactionSelectorFactory;
import org.hyperledger.besu.plugin.services.txselection.PluginTransactionSelectorFactory;
import org.hyperledger.besu.plugin.services.txvalidator.PluginTransactionValidatorFactory;
import org.hyperledger.besu.services.BesuConfigurationImpl;
import org.hyperledger.besu.services.BesuEventsImpl;
Expand Down Expand Up @@ -185,7 +185,7 @@ public void startNode(final BesuNode node) {

final int maxPeers = 25;

final Optional<TransactionSelectorFactory> transactionSelectorFactory =
final Optional<PluginTransactionSelectorFactory> transactionSelectorFactory =
getTransactionSelectorFactory(besuPluginContext);

final PluginTransactionValidatorFactory pluginTransactionValidatorFactory =
Expand Down Expand Up @@ -323,7 +323,7 @@ public String getConsoleContents() {
throw new RuntimeException("Console contents can only be captured in process execution");
}

private Optional<TransactionSelectorFactory> getTransactionSelectorFactory(
private Optional<PluginTransactionSelectorFactory> getTransactionSelectorFactory(
final BesuPluginContextImpl besuPluginContext) {
final Optional<TransactionSelectionService> txSelectionService =
besuPluginContext.getService(TransactionSelectionService.class);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -546,6 +546,7 @@ public BesuNode createExecutionEngineGenesisNode(final String name, final String
.bootnodeEligible(false)
.miningEnabled()
.jsonRpcEnabled()
.jsonRpcTxPool()
.engineRpcEnabled(true)
.jsonRpcDebug()
.build());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.util.Arrays;
import java.util.stream.Collectors;

Expand Down Expand Up @@ -82,9 +83,18 @@ public void test() throws IOException {
testsContext.mapper.readValue(testCaseFileURI.toURL(), JsonRpcTestCase.class);

final String rpcMethod = String.valueOf(testCase.getRequest().get("method"));

OkHttpClient client = testsContext.httpClient;
if (System.getenv("BESU_DEBUG_CHILD_PROCESS_PORT") != null) {
// if running in debug mode, set a longer timeout
client =
testsContext
.httpClient
.newBuilder()
.readTimeout(900, java.util.concurrent.TimeUnit.SECONDS)
.build();
}
final Call testRequest =
testsContext.httpClient.newCall(
client.newCall(
new Request.Builder()
.url(getRpcUrl(rpcMethod))
.post(RequestBody.create(testCase.getRequest().toString(), MEDIA_TYPE_JSON))
Expand All @@ -93,6 +103,7 @@ public void test() throws IOException {

assertThat(response.code()).isEqualTo(testCase.getStatusCode());
final ObjectNode actualBody = JsonUtil.objectNodeFromString(response.body().string());
evaluateResponse(actualBody, testRequest, testCase, testCaseFileURI.toURL());
final ObjectNode expectedBody =
JsonUtil.objectNodeFromString(testCase.getResponse().toString());
assertThat(actualBody)
Expand All @@ -101,6 +112,12 @@ public void test() throws IOException {
.isEqualTo(expectedBody);
}

protected void evaluateResponse(
final ObjectNode responseBody,
final Call testRequest,
final JsonRpcTestCase testCase,
final URL url) {}

private String getRpcUrl(final String rpcMethod) {
if (rpcMethod.contains("eth_") || rpcMethod.contains("engine_")) {
return testsContext.besuNode.engineRpcUrl().get();
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
/*
* Copyright contributors to Hyperledger Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.tests.acceptance.jsonrpc;

import static org.assertj.core.api.Assertions.assertThat;

import org.hyperledger.besu.tests.acceptance.dsl.rpc.JsonRpcTestCase;

import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;

import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.JsonNodeType;
import com.fasterxml.jackson.databind.node.ObjectNode;
import okhttp3.Call;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;

@RunWith(Parameterized.class)
public class ExecutionEngineCancunBlockBuildingAcceptanceTest extends AbstractJsonRpcTest {
private static final String GENESIS_FILE = "/jsonrpc/engine/cancun/genesis.json";
private static final String TEST_CASE_PATH = "/jsonrpc/engine/cancun/test-cases/block-production";

private static JsonRpcTestsContext testsContext;

public ExecutionEngineCancunBlockBuildingAcceptanceTest(
final String ignored, final URI testCaseFileURI) {
super(ignored, testsContext, testCaseFileURI);
}

@BeforeClass
public static void init() throws IOException {
testsContext = new JsonRpcTestsContext(GENESIS_FILE);
}

@Parameterized.Parameters(name = "{0}")
public static Iterable<Object[]> testCases() throws URISyntaxException {
return testCases(TEST_CASE_PATH);
}

@Override
protected void evaluateResponse(
final ObjectNode responseBody,
final Call testRequest,
final JsonRpcTestCase testCase,
final URL url) {
if (url.toString().endsWith("10_cancun_build_on_genesis.json")) {
// if we just asked the node to build, give it some time to build
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
if (url.toString().endsWith("12_cancun_get_built_block.json")) {

// final ObjectNode rpcResponse = JsonUtil.objectNodeFromString(response.body().string());
final ObjectNode result = (ObjectNode) responseBody.get("result");
final ObjectNode execPayload = (ObjectNode) result.get("executionPayload");
final ObjectNode blobsBundle = (ObjectNode) result.get("blobsBundle");
assertThat(execPayload.get("transactions").getNodeType()).isEqualTo(JsonNodeType.ARRAY);
final ArrayNode transactions = (ArrayNode) execPayload.get("transactions");
// actually, you need to decode the transactions and count how many unique
// versioned hashes are referenced amongst them.
assertThat(blobsBundle.get("commitments").getNodeType()).isEqualTo(JsonNodeType.ARRAY);
final ArrayNode commitments = (ArrayNode) blobsBundle.get("commitments");
assertThat(blobsBundle.get("blobs").getNodeType()).isEqualTo(JsonNodeType.ARRAY);
final ArrayNode blobs = (ArrayNode) blobsBundle.get("blobs");
final ArrayNode proofs = (ArrayNode) blobsBundle.get("proofs");
assertThat(2).isEqualTo(transactions.size());
assertThat(6).isEqualTo(commitments.size());
assertThat(6).isEqualTo(blobs.size());
assertThat(6).isEqualTo(proofs.size());
}
}

@AfterClass
public static void tearDown() {
testsContext.cluster.close();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,10 @@

import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Ignore;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;

@RunWith(Parameterized.class)
@Ignore("EIP-6110 is not yet implemented")
public class ExecutionEngineEip6110AcceptanceTest extends AbstractJsonRpcTest {
private static final String GENESIS_FILE = "/jsonrpc/engine/eip6110/genesis.json";
private static final String TEST_CASE_PATH = "/jsonrpc/engine/eip6110/test-cases/";
Expand Down
Loading

0 comments on commit 31db6df

Please sign in to comment.