Skip to content

Commit

Permalink
Merge pull request #1165 from input-output-hk/kderme/fix-docs
Browse files Browse the repository at this point in the history
Improve configuration docs
  • Loading branch information
erikd authored Jun 28, 2022
2 parents c32408f + 1eaeb4a commit b994337
Showing 1 changed file with 24 additions and 22 deletions.
46 changes: 24 additions & 22 deletions doc/configuration.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,37 @@
# Configuration

The initial design of db-sync was an one fit all approach, where it would serve as a general purpose
backend for cardano applications, including light wallets, explorers etc. Over the time many new
feautures have been added, like historic rewards with Shelley, scripts with Allegra, multiassets
with Mary, Plutus scripts and reeemers with Alonzo, pool metadata with the integration of SMASH etc.
Many applications use only a small fraction of these feautures, but db-sync needs to use the
resources that all these features require. It is reasonable to introduce flags and options that
turn off some of these features, expecially the most expensive ones in terms of performance. These
configuration options require proper documentation, which is presented below
The initial design of db-sync was a one size fits all approach. It served as a general-purpose
backend for Cardano applications, including light wallets, explorers, etc. Over time, many new
features have been added, including historic rewards with Shelley, scripts with Allegra, multiassets
with Mary, Plutus scripts and redeemers with Alonzo, stake pool metadata with the integration of
SMASH, etc.

While db-sync needs to use the resources that all these features require, many applications use only
a small fraction of these features. Therefore, it is reasonable to introduce flags and options that
turn off some of these features, especially the most expensive ones in terms of performance. These
configuration options require proper documentation, which is presented below.

### --disable-ledger

One of the features of db-sync that uses the most resources maintaining the ledger state and
replaying all the ledger rules. This is the only way to get historic rewards and other things that
are not included in the blocks, like historic stake distribution, ada pots, epoch params etc. The
flag `--disable-ledger` provides the option to turn off these feauture and significantly reduces
usage of memory (by up to 10GB on mainnet) and sync time.
One of the db-sync features that uses the most resources is that it maintains a ledger state and
replays all the ledger rules. This is the only way to get historic reward details and other data
that is not included in the blocks (ie. historic stake distribution, ada pots, epoch parameters,
etc). The flag --disable-ledger provides the option to turn off these features and significantly
reduce memory usage (by up to 10GB on mainnet) and sync time. Another benefit of this option is
that there are no rollbacks on startup, which tend to take quite some time, since there are no
ledger snapshots maintained on disk.

When this flag is enabled, some feautures are missing and some db tables are left empty:
When this flag is enabled, some features are missing and some DB tables are left empty:
- `redeemer.fee` is left null
- `reward` table is left empty
- `epoch_stake` table is left empty
- `ada_pots` table is left empty
- `epoch_param` table is left empty

Released snapshots are compatible with this options. There could be some small inconsistencies
though, since the snapshots are created using the option. So the above data may exist up to the
slot/epoch of the snapshot creation and missing afterwards. The simplest way to fix this, is to
delete the existing data. Since this is still an experimental feauture, db-sync doesn't do anything
automatically, since we don't know how users want to use the feature.
Released snapshots are compatible with these options. Since the snapshots are created without the
option, there still can be some minor inconsistencies. The above data may exist up to the slot/epoch
of the snapshot creation and can be missing afterward. To fix this, when db-sync is initiated with
this flag, it will automatically remove all these data.

Warning: This will irreversibly delete data from existing snapshots.

Expand All @@ -44,12 +47,11 @@ delete from epoch_param;

### --disable-cache : Experimental

This disables the application level caches of db-sync. It reduces memory usage by a bit, but
increases sync time. Usually this flag is not worth it, unless there are some pretty big memory
This flag disables the application level caches of db-sync. It slightly reduces memory usage but
increases the syncing time. This flag is worth using only when experiencing significant memory
issues.

### --disable-epoch : Experimental

With this option the epoch table is left empty. Mostly left for historical reasons, since it
provides a negligible improvement in sync time.

0 comments on commit b994337

Please sign in to comment.