[Bug] Deduplication logic for Cosmos Event is faulty leading to missed event

[betterclever]

Bug report

Hi team!
I am an engineer at Persistence Labs where we maintain a self-hosted deployment of Graph Node for usage with Persistence Core-1 chain (a Cosmos SDK based L1).

I noticed that a few events were being missed in our subgraph which were similar but not exactly duplicates. Only first of them was being indexed, and the rest were being ignored and the handler actually never reached the subgraph.

On going deep, I realized this is happening due to a bug in the dedup logic for the events for the Cosmos Blockchain.

The dedup call happens here:

graph-node/graph/src/blockchain/block_stream.rs

Line 225 in 8c6edc1

trigger_data.dedup();

Rust stdlib dedup function uses PartialEq implementation for the dedup.

The PartialEq for BlockStream is defined here:

graph-node/chain/cosmos/src/trigger.rs

Lines 75 to 100 in 8c6edc1

	impl PartialEq for CosmosTrigger {
	fn eq(&self, other: &Self) -> bool {
	match (self, other) {
	(Self::Block(a_ptr), Self::Block(b_ptr)) => a_ptr == b_ptr,
	(
	Self::Event {
	event_data: a_event_data,
	origin: a_origin,
	},
	Self::Event {
	event_data: b_event_data,
	origin: b_origin,
	},
	) => {
	if let (Ok(a_event), Ok(b_event)) = (a_event_data.event(), b_event_data.event()) {
	a_event.event_type == b_event.event_type && a_origin == b_origin
	} else {
	false
	}
	}
	(Self::Transaction(a_ptr), Self::Transaction(b_ptr)) => a_ptr == b_ptr,
	(Self::Message(a_ptr), Self::Message(b_ptr)) => a_ptr == b_ptr,
	_ => false,
	}
	}
	}

Particularly, the line 90 has an issue where only the source and event name is checked for duplicate. There's a possibility that events with the same name are emitted multiple times in the same block in Cosmos in the same transaction with different attribute values.

Error Analysis and reproducible scenario

I have made a fork branch with a test-case where I downloaded a block from the PersistenceCore Core-1 Firehose and copied it's base64 data. I later processed the triggers exactly as in the current 0.33 release, and then ran the dedup operation to demonstrate that non-duplicate data is being removed leading to ignored events during indexing.

Link to test case in the fork:

graph-node/chain/cosmos/src/chain.rs

Line 470 in 01f3f43

fn test_decode_block() {

Run the test using:
cargo test --package graph-chain-cosmos --lib -- chain::test::test_decode_block --exact --nocapture

Fix

The fix is pretty straightforward. We need to update the PartialEq logic also include block data scenario. Only the name and source doesn't suffice.

Possible remediation for older releases where wrong data was indexed

As I can imagine, a lot of past indexed data must have been missed due to this issue. All those subgraphs would have to be re-indexed on a fixed Graph-node release binary to have data without any missing pieces.

Relevant log output

No response

IPFS hash

No response

Subgraph name or link to explorer

No response

Some information to help us out

• Tick this box if this bug is caused by a regression found in the latest release.
• Tick this box if this bug is specific to the hosted service.
• I have searched the issue tracker to make sure this issue is not a duplicate.

OS information

Linux

[betterclever]

As I see from the older raised bugs, this also is related and the root cause is exactly the same, however the error is much clearly highlighted in the current bug report with a reproducible scenario.

#4998

[wojciechowskip]

Hmm, so when it will be possible to re-index data? When can we expect new version of graph node to be released?

[p-diogo]

@wojciechowskip do you intend to re-index using the Hosted Service?

[wojciechowskip]

@wojciechowskip do you intend to re-index using the Hosted Service?

Yes, I am just waiting for the release to test it :)

[incrypto32]

@wojciechowskip its live on Hosted service.

[azf20]

Closing, thanks so much @betterclever !