rosetta-cli check:data failures

[qiwu7]

getting duplicated transaction hash error while trying to sync from 50000 to 120010

Error: unable to process blocks: unable to process block: unable to store transaction hash: duplicate transaction hash: duplicate transaction bafy2bzacea5ljsf44bxpfjvekhrjy3tu2sivajg7zeuxk7wvqc6rrps2tmssi found in block 0171a0e4022016e66e66ffe5a99520283fa5fa3a963470163c31db7804f55163cf14e903d7a0:50003: unable to add block to storage 0171a0e4022016e66e66ffe5a99520283fa5fa3a963470163c31db7804f55163cf14e903d7a0:50003: unable to sync to 120010

[qiwu7]

If I query block 41281, I am getting 7393 duplicated transaction with this hsh bafy2bzacedrjokkpqbvaxza2v65km2lcsq5l6e3wx4j66r66ker5fs4p5orbq. It seems that we are including the same transaction multiple times. But if the 2 transactions are different but are having same transaction hash (message id) then it's a even worse issue. But please do some investigation and fix as needed.

{
    "network_identifier": {
        "blockchain": "Filecoin",
        "network": "{{testnet_network}}"
    },
    "block_identifier": {
        "index": 41281
    }
}

[raynaudoe]

I'll look into it as soon any of our nodes gets synced.
But from the look of it maybe it's related to something on traces, there's no way the proxy deliberately includes the same transaction more than once:
https://github.com/Zondax/rosetta-filecoin-proxy/blob/c301d038f6421073982e8ef993bacda8ce486d7e/rosetta/services/block.go#L203-L202

[arajasek]

41281 is the epoch when our first "upgrade" would be computed, which involved a lot of irregular transitions. These transitions don't actually live on chain as any messages, but we provided a way to include them in ExecutionTraces. Sounds like we need to modify the way we include them so that the tx hashes are not duplicated. This should be easy to do, it doesn't impact consensus.

[jleni]

Ok, I will mark the issue as an external dependency. This is something not related to the proxy itself.

[qiwu7]

@arajasek @jleni tipSet at 41281 is just an example. There are other blocks that contains dup transactions. For instance 50003 also includes duplicated transactions. I was not able to test every single block though.

[jleni]

Related to filecoin-project/lotus#4350

[raynaudoe]

I'll add some code to avoid including this "fake" transactions, hopefully checking if Msg.Nonce == math.MaxUint64 will be enough?
We also have to make sure that no one uses that nonce in a "real" transaction.

[jleni]

@raynaudoe Lotus changed this (Msg.Nonce == math.MaxUint64) in the PR. This will not work anymore, please check here filecoin-project/lotus#4350

[raynaudoe]

ok, I guess that filtering by address (builtin0.BurntFundsActorAddr and builtin.ReserveAddress) would be an option. I remember that both of those addresses had reconciliation issues on a previous rosetta-cli check run.