polygon integration

[buck54321]

I've sketched out the client bits and have started looking into a harness.

buck54321/dcrdex@8d990a1...buck54321:dcrdex:polygon

The harness is proving to be difficult. Typically, with Polygon, you'd run the POS component called Heimdall. I don't know if that's an option for us. but I'm trying to avoid it. Then you run the geth-like part, called bor. To run bor without Heimdall, they've provided some development flags. @chappjc found maticnetwork/bor#697. I've built the development branch and supplied a genesis file (which is just a copy of their testnet file with the network ID changed), and supplied the development flags, but the node is still looking for Heimdall, it appears.

IMPORTANT: 1) The requisite CLI flags are only found on their development branch. 2) The bor executable that needs to be in PATH is actually bor/cmd/cli, which compiles to an executable named cli. I just copied it to one of my PATH directories renamed to bor.

⭐ Anyway. If we can't figure it out, I propose that we just run another Ethereum simnet for the Polygon harness. We're really only interested in the RPC API, and Polygon should be compatible with Ethereum. On top of that, this would facilitate the development of an ethereum "base harness", which other EVM-compatible assets can leverage in their asset-integration process.

To Do

We can potentially split this work into 3 parts.

1. Harness
2. Server
3. Client

It'd be best to get the 1 done before 2 or 3. So I'd like to figure it out ASAP. Anyone object to using an Ethereum simnet?

---

[chappjc]

Wanted to try out v0.3.9-beta with the harness, but something is missing with keystore:

./harness.sh: line 128: /home/jon/dextest/polygon/alpha/node/keystore/UTC--2021-01-27T08-20-38.123221057Z--9ebba10a6136607688ca4f27fab70e23938cd027: No such file or directory
./harness.sh: line 133: /home/jon/dextest/polygon/alpha/node/keystore/UTC--2021-01-28T08-47-02.993754951Z--18d65fb8d60c1199bb1ad381be47aa692b482605: No such file or directory
Setting node key
./harness.sh: line 139: /home/jon/dextest/polygon/alpha/node/geth/nodekey: No such file or directory

Before that with the nodes root:

./harness.sh: line 50: /home/jon/dextest/polygon/alpha/node/polygon.toml: No such file or directory

That's with set -e and a newline added, so two line numbers higher.

[chappjc]

Actually, I've been wondering about eth too, why do we have a toml/conf file if everything seems to be given as cli args (or can be)?

[buck54321]

Actually, I've been wondering about eth too, why do we have a toml/conf file if everything seems to be given as cli args (or can be)?

Right!?.

[ukane-philemon]

I was able to start the server but got this error:

INFO [05-03|03:54:32.835] GRPC Server started                      addr=[::]:3131
INFO [05-03|03:54:32.837] Try reading as legacy genesis 
TRACE[05-03|03:54:32.843] Started watching keystore folder         path=/Users/philemon/dextest/polygon/alpha/node/keystore
DEBUG[05-03|03:54:32.843] FS scan times                            list="190.366µs" set="10.16µs" diff="7.166µs"
TRACE[05-03|03:54:32.844] Handled keystore changes                 time="628.758µs"
failed to unlock developer account: could not decrypt key with given password

[chappjc]

I'd like to try but it keeps saying keystore files are missing when I

Wanted to try out v0.3.9-beta with the harness, but something is missing with keystore:

./harness.sh: line 128: /home/jon/dextest/polygon/alpha/node/keystore/UTC--2021-01-27T08-20-38.123221057Z--9ebba10a6136607688ca4f27fab70e23938cd027: No such file or directory
./harness.sh: line 133: /home/jon/dextest/polygon/alpha/node/keystore/UTC--2021-01-28T08-47-02.993754951Z--18d65fb8d60c1199bb1ad381be47aa692b482605: No such file or directory
Setting node key
./harness.sh: line 139: /home/jon/dextest/polygon/alpha/node/geth/nodekey: No such file or directory

Before that with the nodes root:

./harness.sh: line 50: /home/jon/dextest/polygon/alpha/node/polygon.toml: No such file or directory

That's with set -e and a newline added, so two line numbers higher.

To test your branch @buck54321 is the create-harness.sh the same as with eth? Or did you forget to add/commit the file?

[ukane-philemon]

@chappjc, try with these changes here 65ac0bc
You'd have to fill the values in the config file via cmd, from my investigation I think bor cli overwrites non-empty cmd values with empty config file values if a config file is provided. In fact, bor cli prints a warning message about it, but it's not immediately clear that empty config values will overwrite non-empty cmd values(weird!).

[chappjc]

Nice @ukane-philemon , thanks!

[ukane-philemon]

Running bor cli successfully without looking for Hiemdell - polygon harness suggestions . But I'm unable to resolve these logs.

Also, we can't specify the --dev flag if we have a password-locked account, bor will try to use the password for the dev account and fail.

DEBUG[05-05|00:16:22.712] Couldn't add port mapping                proto=tcp extport=30303 intport=30303 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DEBUG[05-05|00:17:20.534] Current full block not old enough        number=0 hash=7b6650..2b1ca7 delay=90000

Logs

INFO [05-05|00:16:19.360] GRPC Server started                      addr=[::]:3131
INFO [05-05|00:16:19.362] Try reading as legacy genesis 
DEBUG[05-05|00:16:19.367] FS scan times                            list="164.116µs" set="7.782µs" diff="4.842µs"TRACE[05-05|00:16:19.368] Started watching keystore folder         path=/Users/philemon/dextest/polygon/alpha/node/keystoreTRACE[05-05|00:16:19.368] Handled keystore changes                 time="185.804µs"INFO [05-05|00:16:20.336] Unlocked account                         address=0x9eBBA10a6136607688Ca4F27fab70E23938cD027
DEBUG[05-05|00:16:20.336] Sanitizing Go's GC trigger               percent=100
INFO [05-05|00:16:20.336] Set global gas cap                       cap=50,000,000
INFO [05-05|00:16:20.337] Allocated trie memory caches             clean=153.00MiB dirty=256.00MiB
INFO [05-05|00:16:20.340] Allocated cache and file handles         database=/Users/philemon/dextest/polygon/alpha/node/bor/chaindata cache=512.00MiB handles=5120
DEBUG[05-05|00:16:20.420] Chain freezer table opened               database=/Users/philemon/dextest/polygon/alpha/node/geth-ancient table=matic-bor-receipts items=0 size=0.00BDEBUG[05-05|00:16:20.436] Chain freezer table opened               database=/Users/philemon/dextest/polygon/alpha/node/geth-ancient table=diffs              items=0 size=0.00BDEBUG[05-05|00:16:20.465] Chain freezer table opened               database=/Users/philemon/dextest/polygon/alpha/node/geth-ancient table=headers            items=0 size=0.00BDEBUG[05-05|00:16:20.494] Chain freezer table opened               database=/Users/philemon/dextest/polygon/alpha/node/geth-ancient table=hashes             items=0 size=0.00BDEBUG[05-05|00:16:20.513] Chain freezer table opened               database=/Users/philemon/dextest/polygon/alpha/node/geth-ancient table=bodies             items=0 size=0.00B
DEBUG[05-05|00:16:20.529] Chain freezer table opened               database=/Users/philemon/dextest/polygon/alpha/node/geth-ancient table=receipts           items=0 size=0.00B
INFO [05-05|00:16:20.529] Opened ancient database                  database=/Users/philemon/dextest/polygon/alpha/node/geth-ancient readonly=falseINFO [05-05|00:16:20.530] Writing custom genesis block 
DEBUG[05-05|00:16:20.530] Current full block hash unavailable 
INFO [05-05|00:16:20.535] Persisted trie from memory database      nodes=11 size=1.60KiB time="111.115µs" gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=0.00B
INFO [05-05|00:16:20.538] Initialised chain configuration          config="{ChainID: 137000 Homestead: 0 DAO: <nil> DAOSupport: false EIP150: 0 EIP155: 0 EIP158: 0 Byzantium: 0 Constantinople: 0 Petersburg: 0 Istanbul: 2722000, Muir Glacier: 2722000, Berlin: 13996000, London: 22640000, Arrow Glacier: <nil>, MergeFork: <nil>, Terminal TD: <nil>, Engine: bor}"
INFO [05-05|00:16:20.539] Gasprice oracle is ignoring threshold set threshold=2INFO [05-05|00:16:20.541] Initialising Ethereum protocol           network=137,000 dbversion=<nil>INFO [05-05|00:16:20.542] Loaded most recent local header          number=0 hash=7b6650..2b1ca7 td=1 age=4y5d12h
INFO [05-05|00:16:20.543] Loaded most recent local full block      number=0 hash=7b6650..2b1ca7 td=1 age=4y5d12h
INFO [05-05|00:16:20.543] Loaded most recent local fast block      number=0 hash=7b6650..2b1ca7 td=1 age=4y5d12hWARN [05-05|00:16:20.543] Failed to load snapshot, regenerating    err="missing or corrupted snapshot"
INFO [05-05|00:16:20.544] Rebuilding state snapshot 
DEBUG[05-05|00:16:20.544] Journalled generator progress            progress=empty
DEBUG[05-05|00:16:20.544] Start snapshot generation                root=1784d1..d91330
INFO [05-05|00:16:20.545] Resuming state snapshot generation       root=1784d1..d91330 accounts=0 slots=0 storage=0.00B elapsed="473.334µs"
TRACE[05-05|00:16:20.545] Detected outdated state range            kind=account prefix=0x61 last=0x err="wrong root: have 0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421 want 0x1784d1c465e9a4c39cc58b1df8d42f2669b00b1badd231a7c4679378b9d91330"DEBUG[05-05|00:16:20.545] Regenerated state range                  kind=account prefix=0x61 root=1784d1..d91330 last=0x count=8 created=8 updated=0 untouched=0 deleted=0DEBUG[05-05|00:16:20.545] Journalled generator progress            progress=done
INFO [05-05|00:16:20.545] Generated state snapshot                 accounts=8 slots=0 storage=457.00B elapsed=1.190ms
DEBUG[05-05|00:16:20.544] Reinjecting stale transactions           count=0
INFO [05-05|00:16:20.546] Regenerated local transaction journal    transactions=0 accounts=0
WARN [05-05|00:16:20.550] Error reading unclean shutdown markers   error="leveldb: not found"
INFO [05-05|00:16:20.550] Starting peer-to-peer node               instance=bor/v0.3.9-beta/darwin-amd64/go1.20.1
DEBUG[05-05|00:16:20.604] TCP listener up                          addr=[::]:30303
INFO [05-05|00:16:20.605] New local node record                    seq=1,683,242,180,604 id=8302ec0a28490eba ip=127.0.0.1 udp=0 tcp=30303
INFO [05-05|00:16:20.606] Started P2P networking                   self="enode://ae060d1456df3fc0228796a24e241d9d77b17b3e220ea632b221922db9eb53f094b693330349e9fad96daea4113337e508dd927daa2c955fd5f9692e34c9563a@127.0.0.1:30303?discport=0"
DEBUG[05-05|00:16:20.607] IPCs registered                          namespaces=admin,debug,web3,eth,txpool,personal,bor,miner,net
INFO [05-05|00:16:20.609] IPC endpoint opened                      url=/Users/philemon/dextest/polygon/alpha/node/bor.ipc
DEBUG[05-05|00:16:20.609] Allowed origin(s) for WS RPC interface [localhost] 
INFO [05-05|00:16:20.612] HTTP server started                      endpoint=127.0.0.1:38556 auth=false prefix=     cors=localhost vhosts=localhost
INFO [05-05|00:16:20.612] WebSocket enabled                        url=ws://127.0.0.1:38557
INFO [05-05|00:16:20.612] TxFetcher                                txArrivalWait=500ms
DEBUG[05-05|00:16:22.712] Couldn't add port mapping                proto=tcp extport=30303 intport=30303 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DEBUG[05-05|00:17:20.534] Current full block not old enough        number=0 hash=7b6650..2b1ca7 delay=90000

</details?