No new epoch_info data right after the epoch boundary

[xray-robot]

Is your feature request related to a problem? Please describe.

Since cron is triggered every 15 minutes on epoch-info-cache-update, right after the epoch boundary the new epoch data is not available (in our case we lose epoch end_time information).

And in general many cached tables are not available at this time (15-30-45 minutes time span after the epoch boundary), perhaps we should invent some system of (db level?) triggers to either remove this waiting time or reduce it to acceptable values.

Describe the solution you'd like

1. Create some more precise triggers for caching functions
2. Add epoch_start_time, epoch_end_time to the tip endpoint (?)

The task is not urgent, but worthy of attention ☺️

[rdlrt]

Since cron is triggered every 15 minutes on epoch-info-cache-update, right after the epoch boundary the new epoch data is not available (in our case we lose epoch end_time information).

I think instead of changing the epoch-info-cache-update interval, we would like to move away from needing a cache table beyond block/tx indexes at start/end time of a particular epoch. This would mean epoch-info-cache is much slimmer, as fetching epoch_params/cost models could be done against live tables (while last tx could end up as empty until cache has finished), do you agree?

in general many cached tables are not available at this time

I think while the intervals can be adapted for epoch table, the cache information for most (account/pool/etc) is delayed on node and DB level, and only becomes available after calculation and insert is finished - for the cache to be able to trigger. For instance, in #195 - you'd see that we end up having to read the logs and wait for epoch stake to finish populating to be able to start preparing data for stake cache, which in turn is used for pool stake and so on. Also, from chain pov, the new epoch does not really start until there is a new block available, and while that block might be available - the associated info like treasury/reward/MIRs/etc for next epoch are populated few stability windows later within an epoch.

[rdlrt]

Closed in linked PR

[xray-robot]

I think instead of changing the epoch-info-cache-update interval, we would like to move away from needing a cache table beyond block/tx indexes at start/end time of a particular epoch. This would mean epoch-info-cache is much slimmer, as fetching epoch_params/cost models could be done against live tables (while last tx could end up as empty until cache has finished), do you agree?

It looks logical if the fetching can happen on live tables.

I think while the intervals can be adapted for epoch table, the cache information for most (account/pool/etc) is delayed on node and DB level, and only becomes available after calculation and insert is finished - for the cache to be able to trigger. For instance, in #195 - you'd see that we end up having to read the logs and wait for epoch stake to finish populating to be able to start preparing data for stake cache, which in turn is used for pool stake and so on. Also, from chain pov, the new epoch does not really start until there is a new block available, and while that block might be available - the associated info like treasury/reward/MIRs/etc for next epoch are populated few stability windows later within an epoch.

Yes, this all sounds logical, but maybe there should be some new endpoint that doesn't depend on inserting a stake, but indicates that an epoch has started (perhaps the data should be taken from live tables). For now, it's more of an indication that epoch_info has a gap with the network state in the half hour that db-sync stake processing takes.

Edited: I mean the current epoch_info is more related to txs/stake statistics data. Perhaps there should be an additional endpoint that shows more data about the epoch itself.

[xray-robot]

Whoops, sorry. I mixed tip and epoch_info in my head. Don't mind my ticket. 😅