Skip to content

Commit

Permalink
readme: faq on how to use as a lib (#12553)
Browse files Browse the repository at this point in the history
  • Loading branch information
@AskAlexSharov
AskAlexSharov authored Nov 1, 2024
1 parent eb1f470 commit 177f1d6
Show file tree
Hide file tree
Showing 2 changed files with 39 additions and 38 deletions.
76 changes: 39 additions & 37 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ by default.
- [Config Files TOML](#config-files-toml)
- [Beacon Chain (Consensus Layer)](#beacon-chain-consensus-layer)
- [Caplin](#caplin)
- [Caplin's Usage](#caplins-usage)
- [Caplin's Usage](#caplins-usage)
- [Multiple Instances / One Machine](#multiple-instances--one-machine)
- [Dev Chain](#dev-chain)
- [Key features](#key-features)
Expand All @@ -39,24 +39,24 @@ by default.
- [JSON-RPC daemon](#json-rpc-daemon)
- [Grafana dashboard](#grafana-dashboard)
- [FAQ](#faq)
- [How much RAM do I need](#how-much-ram-do-i-need)
- [Use as library](#use-as-library)
- [Default Ports and Firewalls](#default-ports-and-firewalls)
- [`erigon` ports](#erigon-ports)
- [`caplin` ports](#caplin-ports)
- [`beaconAPI` ports](#beaconapi-ports)
- [`shared` ports](#shared-ports)
- [`other` ports](#other-ports)
- [Hetzner expecting strict firewall rules](#hetzner-expecting-strict-firewall-rules)
- [`erigon` ports](#erigon-ports)
- [`caplin` ports](#caplin-ports)
- [`beaconAPI` ports](#beaconapi-ports)
- [`shared` ports](#shared-ports)
- [`other` ports](#other-ports)
- [Hetzner expecting strict firewall rules](#hetzner-expecting-strict-firewall-rules)
- [Run as a separate user - `systemd` example](#run-as-a-separate-user---systemd-example)
- [How to get diagnostic for bug report?](#how-to-get-diagnostic-for-bug-report)
- [How to run local devnet?](#how-to-run-local-devnet)
- [Grab diagnostic for bug report](#grab-diagnostic-for-bug-report)
- [Run local devnet](#run-local-devnet)
- [Docker permissions error](#docker-permissions-error)
- [How to run public RPC api](#how-to-run-public-rpc-api)
- [Public RPC](#public-rpc)
- [RaspberyPI](#raspberypi)
- [Run all components by docker-compose](#run-all-components-by-docker-compose)
- [Optional: Setup dedicated user](#optional-setup-dedicated-user)
- [Environment Variables](#environment-variables)
- [Run](#run)
- [Optional: Setup dedicated user](#optional-setup-dedicated-user)
- [Environment Variables](#environment-variables)
- [Run](#run)
- [How to change db pagesize](#how-to-change-db-pagesize)
- [Erigon3 perf tricks](#erigon3-perf-tricks)
- [Windows](#windows)
Expand Down Expand Up @@ -203,19 +203,18 @@ du -hsc /erigon/snapshots/*

### Erigon3 changes from Erigon2

- Sync from scratch doesn't require re-exec all history. Latest state and it's history are in snapshots - can download.
- ExecutionStage - now including many E2 stages: stage_hash_state, stage_trie, stage_log_index, stage_history_index,
stage_trace_index
- Initial sync does download LatestState and it's history - no re-exec from 0 anymore.
- ExecutionStage included many E2 stages: stage_hash_state, stage_trie, log_index, history_index, trace_index
- E3 can execute 1 historical transaction - without executing it's block - because history/indices have
transaction-granularity, instead of block-granularity.
- E3 doesn't store Logs (aka Receipts) - it always re-executing historical txn (but it's cheaper then in E2 - see point
above).
- `--sync.loop.block.limit` is enabled by default. (Default: `5_000`.
Set `--sync.loop.block.limit=10_000 --batchSize=2g` to increase sync speed on good hardware).
- datadir/chaindata is small now - to prevent it's grow: we recommend set `--batchSize <= 2G`. And it's fine to
`rm -rf chaindata`
- Restart doesn't loose much partial progress: `--sync.loop.block.limit=5_000` enabled by default
- `chaindata` is less than `15gb`. It's ok to `rm -rf chaindata`. To prevent it's grow: recommend `--batchSize <= 1G`
- can symlink/mount latest state to fast drive and history to cheap drive
- Archive Node is default. Full Node: `--prune.mode=full`, Minimal Node (EIP-4444): `--prune.mode=minimal`
- `--internalcl` is enabled by default. to disable use `--externalcl`
- `--prune` flags changed: default `--prune.mode=archive`, FullNode: `--prune.mode=full`, MinimalNode (EIP-4444):
`--prune.mode=minimal`.

### Logging

Expand Down Expand Up @@ -404,7 +403,7 @@ DB. That reduces write amplification and DB inserts are orders of magnitude quic
### JSON-RPC daemon

Most of Erigon's components (txpool, rpcdaemon, snapshots downloader, sentry, ...) can work inside Erigon and as
independent process on same Server (or another Server). Example
independent process on same Server (or another Server). Example:

```sh
make erigon rpcdaemon
Expand All @@ -413,8 +412,10 @@ make erigon rpcdaemon
./build/bin/rpcdaemon --datadir=/my --http.api=eth,erigon,web3,net,debug,trace,txpool --ws
```

Supported JSON-RPC calls ([eth](./cmd/rpcdaemon/commands/eth_api.go), [debug](./cmd/rpcdaemon/commands/debug_api.go)
, [net](./cmd/rpcdaemon/commands/net_api.go), [web3](./cmd/rpcdaemon/commands/web3_api.go)):
- Supported JSON-RPC
calls: [eth](./cmd/rpcdaemon/commands/eth_api.go), [debug](./cmd/rpcdaemon/commands/debug_api.go), [net](./cmd/rpcdaemon/commands/net_api.go), [web3](./cmd/rpcdaemon/commands/web3_api.go)
- increase throughput by: `--rpc.batch.concurrency`, `--rpc.batch.limit`, `--db.read.concurrency`
- increase throughput by disabling: `--http.compression`, `--ws.compression`

<code>🔬 See [RPC-Daemon docs](./cmd/rpcdaemon/README.md)</code>

Expand All @@ -425,13 +426,14 @@ Supported JSON-RPC calls ([eth](./cmd/rpcdaemon/commands/eth_api.go), [debug](./
FAQ
================

### How much RAM do I need
### Use as library

- Baseline (ext4 SSD): 16Gb RAM sync takes 6 days, 32Gb - 5 days, 64Gb - 4 days
- +1 day on "zfs compression=off". +2 days on "zfs compression=on" (2x compression ratio). +3 days on btrfs.
- -1 day on NVMe

Detailed explanation: [./docs/programmers_guide/db_faq.md](./docs/programmers_guide/db_faq.md)
```
# please use git branch name (or commit hash). don't use git tags
go mod edit -replace github.com/erigontech/erigon-lib=github.com/erigontech/erigon/erigon-lib@5498f854e44df5c8f0804ff4f0747c0dec3caad5
go get github.com/erigontech/erigon@main
go mod tidy
```

### Default Ports and Firewalls

Expand Down Expand Up @@ -527,15 +529,15 @@ or `/opt/erigon` as the installation path, for example:
make DIST=/opt/erigon install
```
### How to get diagnostic for bug report?
### Grab diagnostic for bug report
- Get stack trace: `kill -SIGUSR1 <pid>`, get trace and stop: `kill -6 <pid>`
- Get CPU profiling: add `--pprof` flag and run
`go tool pprof -png http://127.0.0.1:6060/debug/pprof/profile\?seconds\=20 > cpu.png`
- Get RAM profiling: add `--pprof` flag and run
`go tool pprof -inuse_space -png http://127.0.0.1:6060/debug/pprof/heap > mem.png`
### How to run local devnet?
### Run local devnet
<code> 🔬 Detailed explanation is [here](/DEV_CHAIN.md).</code>
Expand All @@ -547,12 +549,12 @@ UID/GID (1000).
More details
in [post](https://www.fullstaq.com/knowledge-hub/blogs/docker-and-the-host-filesystem-owner-matching-problem)
### How to run public RPC api
### Public RPC
- `--txpool.nolocals=true`
- don't add `admin` in `--http.api` list
- to increase throughput may need
increase/decrease: `--db.read.concurrency`, `--rpc.batch.concurrency`, `--rpc.batch.limit`
- `--http.corsdomain="*"` is bad-practice: set exact hostname or IP
- protect from DOS by reducing: `--rpc.batch.concurrency`, `--rpc.batch.limit`
### RaspberyPI
Expand Down Expand Up @@ -648,7 +650,7 @@ non-root user for security reasons. For more information about how to do this, r
### Erigon3 perf tricks
- `--sync.loop.block.limit=10_000 --batchSize=2g` - likely will help for sync speed.
- on BorMainnet may help: `--sync.loop.block.limit=10_000`
- on cloud-drives (good throughput, bad latency) - can enable OS's brain to pre-fetch: `SNAPSHOT_MADV_RND=false`
- can lock latest state in RAM - to prevent from eviction (node may face high historical RPC traffic without impacting
Chain-Tip perf):
Expand Down
1 change: 0 additions & 1 deletion tests/init_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,6 @@ func sortedMapKeys(m reflect.Value) []string {
}

func runTestFunc(runTest interface{}, t *testing.T, name string, m reflect.Value, key string) {
t.Parallel()
reflect.ValueOf(runTest).Call([]reflect.Value{
reflect.ValueOf(t),
reflect.ValueOf(name),
Expand Down

0 comments on commit 177f1d6

Please sign in to comment.