FVM Programs cache invalidation 🆕

[janezpodhostnik]

Why (Objective)

When programs cache gets invalidated each block it degrades execution performance and causes operational issues, with Execution node caching-up speed being only ~6 blocks/sec.

How will we measure success (Key Results)

• Programs cached across blocks > 0
• TPS improvement from 450 to ~600
• Execution node block catch-up rate increses from ~6 to ~30 blocks/sec

🐞 Bug Report

Programs cache is still invalidates on every block boundary on testnet and mainnet even after #6347. It works correctly on benchnet.

What is the severity of this bug?

Should have: The performance gains for execution are >7% (7% improvement is what we get on token transfer TPS measurements, but in that scenario only a few contracts are used every block, so that is an underestimate)

Another consideration is the catch-up speed for ENs. It used to be ~30 blocks per second and it is currently ~10 blocks (however not all of that is from this issue)

Reproduction steps

Not really reproduction steps, but you can see that the programs cached across blocks are 0 on this metric:

Specifications

• flow-go version: v0.37.14

Fix

There are two options:

1. force un-inlining of registers containing the Random Source History and the Metering settings. This would mean special casing for these registers.

2. Move the metering settings from the service account to a new account, and make changes in the FVM to use that account to read the metering settings.

Option 2. seems better.

Effort Estimate

~2 dev/weeks