[SHIPPED]🚀 Hyperspace Patch Release 19/01/2023

[maciejwitowski]

We are going to do a pre-hackathon release to address the issues currently experienced on the Hyperspace.

Scope
Code:

• enable txid hash map by default if service ethAPIs #10067
• Tipset CID flakiness #10061
• config: Fevm.EnableEthRPC #10072
• fix: events: put the _signed_ message through the pipeline. #10079
• itests: Add tests for EthGetLogs filter combinations #10052

Docs:

• Instructions for block explorers and other partners to reconstruct the tipset CIDs that are missing from past tipsets.
• backfills tx id maps
• instructions for enabling eth api support (which enables tx id map)

Not confirmed scope additions

Decided to skip in this patch

• lotus send from an eth address #10049
• add a lotus-shed cmd to enable node operators to back porting the ID indexes maps that they are missing. #10075

[maciejwitowski]

@jennijuju do we need this #10049? What's the impact of not shipping it now?

[raulk]

✅ #10061 is fixed by #10069 and tested by #10071.

[raulk]

• Need to merge the simplified Eth endpoint setup as well, which activates the transaction hash index.
• Need to add the tool to backfill missing Eth transaction hashes, and add instructions for operators.

[raulk]

Instructions for block explorers and other partners to reconstruct the tipset CIDs that are missing from past tipsets.

For this, we have two options:

1. Use lotus chain set-head to rewind the chain and resync.
2. Create a tool, but because badger does not allow concurrent writes from different processes, there would be an interruption anyway.

Since the chain is short and light, I propose (1). Easier on us, known to them.

[raulk]

More for instructions:

• Enable the EnableEthRPC config flag: config: Fevm.EnableEthRPC #10072.

[jennijuju]

We likely no longer need #10075 for this tag given

• for all existing node operators and block explorers they will need to backfill the tipserCIDs that's missing. The way to do it is to resync via set-head.
• now with the new release, whoever that's serving ethapis will have the tx hash map enabled by default. during the resync, the tx hash will be populated as well.

@geoff-vball is verifying the above.

[maciejwitowski]

Adding #10052 to the scope.
Reasons:

• important fixes
• 1st review pass looks promising. Just a few minor fixes Raul will do without waiting for Ian
• Low risk of regression because of isolated changes +

[jennijuju]

Update - given the config changes and the resyncing needed, we believe it's easier just for the node operators to start a fresh repo.

[jennijuju]

Thanks to the testing and detailed bug reports, this is a 🥩 patch as we fixed many bugs w.r.t filecoin message cid <> eth tx hash, event log storing and so on! With this patch, we believe most of the node service providers, api service provider and explorer’s needs should be successfully fulfilled!

Here is the tag: v1.20.0-hyperpsace-0119.
Things we improved/fixed:

• config: Fevm.EnableEthRPC #10072
  • For node operators who want to support Eth APIs, set EnableEthRPC to true in your node config.toml, or you can export LOTUS_FEVM_ENABLEETHRPC=true as your env var.
  • If Eth APIs RPC are supported, events & eth tx hash map will be stored in two sqlite db by default.
• enable txid hash map by default if service ethAPIs #10067: always store eth tx id <> filecoin message cid to support eth api methods
• Tipset CID flakiness #10061: some Tipset CID was not correctly stored into the block store
• fix: events: put the _signed_ message through the pipeline. #10079

📣 We HIGHLY RECOMMEND all chain service providers, i:e: node, api, block explorer & etc, sync hyperspace from scratch to persist the most accurate historical chain states/information, i.e: tipset CID, Filecoin message CID and Eth tx id map, events&logs.

👩‍🏫 Here is how you can re-deploy your node (assuming your LOTUS_PATH is under ~/.lotus):
First, remove the old chainstore and event/txidmap databases

rm -rf ~/.lotus/datastore //remove your lotus chain datastore
rm -rf ~/.lotus/sqlite //remove your eth tx hash map or event database if exists 

Next you need to get the latest config file, which can be downloaded at https://gist.github.com/jennijuju/a0c9bd09a353ba8cd0255a09396c5915. The only default that’s adjusted is EnableEthRPC = true , tune the other configurations as you see fit.

Then you know the drill, we will build the latest lotus:

git fetch origin
git checkout v1.20.0-hyperpsace-0119
git submodule update --recursive
make clean && make hyperspacenet
sudo make install

Now you can start your node! As your nodes sync from the network, tipset CID will be automatically populated into your chain blockstore. You will also find two databases being created under ~/.lotus/sqlite : events.db & txhash.db . Once your node is fully synced, all events/logs triggered so far and all the Eth tx hashes &their Filecoin message CIDs will be stored there. If you want to opt-out events/logs, simply disable Fevm.Events in your config.toml.

That's it! Have fun buidl 💙 !