Shadowgraph: Missing non-expired self parent

[OlegMazurov]

Description

From nightly performance tests:

2024-09-04 06:25:59.789 70730    WARN  STARTUP          <platformForkJoinThread-4> Shadowgraph: Missing non-expired self parent for EventDescriptorWrapper[eventDescriptor=EventDescriptor[hash=f878ffc19f173c403e74a73d79a6aa876401a1a4ff3e1c85ef9f3a11cb2156d02ed7152dec418566fa7152b2a32d080a, creatorNodeId=1, birthRound=1, generation=3720948], hash=f878ffc19f173c403e74a73d79a6aa876401a1a4ff3e1c85ef9f3a11cb2156d02ed7152dec418566fa7152b2a32d080a, creator=1]
    sp: EventDescriptorWrapper[eventDescriptor=EventDescriptor[hash=71830edcfab1fa3be98fa546632c8b0af4929cef2a11f8e8b91fc134ad63d4c33bfc0a01cfdbaf92015d87d93e2e8ddd, creatorNodeId=1, birthRound=1, generation=3720845], hash=71830edcfab1fa3be98fa546632c8b0af4929cef2a11f8e8b91fc134ad63d4c33bfc0a01cfdbaf92015d87d93e2e8ddd, creator=1]
    op0: EventDescriptorWrapper[eventDescriptor=EventDescriptor[hash=3944f2df75f120e664c8188cc19ac4e89448ffac416e5bcb42f7334951f8e814ffbedc540778e31806f9f86d4b513c0a, creatorNodeId=4, birthRound=1, generation=3720947], hash=3944f2df75f120e664c8188cc19ac4e89448ffac416e5bcb42f7334951f8e814ffbedc540778e31806f9f86d4b513c0a, creator=4]

The same issue was reproduced with the load generator running NftTransferLoadTest. It's likely the error above is the cause of receiving RECEIPT_NOT_FOUND after 180 sec for affected transactions.

Steps to reproduce

1. Nightly performance tests
2. Network load generator running NftTransferLoadTest

Additional context

No response

Hedera network

other

Version

0.54.0-develop.x5676419

Operating system

Linux

[litt3]

Shadowgraph: Missing non-expired self parent

This is only a warning, not an error. It's not a new warning.

There is no inherent problem if the shadowgraph is missing ancient-non-expired-parents for events. Mainly this warning log is still there for historical reasons, not because there is any real problem if this arises. It's not clear to me how this would be responsible for the RECEIPT_NOT_FOUND error.

[tomzhenghedera]

This is the first time the log message had been observed in the perf env, unless the message had been suppressed until v0.5 4 alpha.2, it's being trigged during the state saving(?) on v0.54 alpha.2 build. This looks like a regression to me even the underline root cause may be innocent(why it's just a WARN message). Also it's slow spamming the swirlds.log as messages being emitted around each state save.

[OlegMazurov]

See also #10116

[alex-kuzmin-hg]

Provided logs/details in #17396
Reproducible in large scale in v0.58.3 in 100M SmartContact load test