[Merged by Bors] - atx syncer that persists results

[dshulyak]

part of: #5553

when requested we ask configured number of peers for epoch info (collection of atxs from that epoch). on a successful response we save known ids, and will ask again only in 30 minutes (configurable). also on restart we check persisted data, and potentially avoiding eager queries, if last query was made close to the epoch end.

concurrently with requesting epoch info updates, we will download atxs from peers. download is scheduled in batches, so that we can report progress. if peer advertised invalid atx id, we will evict such id after reaching max number of retries (20 in the pr).

to make error checking possible i extended errors emitted by p2p/server and fetcher.

[codecov]

Codecov Report

Attention: Patch coverage is 85.11327% with 46 lines in your changes are missing coverage. Please review.

Project coverage is 79.8%. Comparing base (3e59df3) to head (11d1a45).

Files	Patch %	Lines
syncer/atxsync/syncer.go	89.5%	12 Missing and 8 partials ⚠️
sql/atxsync/atxsync.go	80.5%	6 Missing and 7 partials ⚠️
fetch/mesh_data.go	68.0%	8 Missing ⚠️
checkpoint/recovery.go	60.0%	1 Missing and 1 partial ⚠️
log/zap.go	60.0%	1 Missing and 1 partial ⚠️
syncer/syncer.go	83.3%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##           develop   #5599    +/-   ##
========================================
  Coverage     79.7%   79.8%            
========================================
  Files          274     276     +2     
  Lines        27883   28180   +297     
========================================
+ Hits         22228   22490   +262     
- Misses        4108    4128    +20     
- Partials      1547    1562    +15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[poszu]

Am I missing something or requests will always be 0?

[dshulyak]

it is updated in code below

if errors.As(err, &batchError) {
	for hash, err := range batchError.Errors {
		if errors.Is(err, server.ErrPeerResponseFailed) {
			state[types.ATXID(hash)]++
		} else if errors.Is(err, pubsub.ErrValidationReject) {
			state[types.ATXID(hash)] = s.cfg.RequestsLimit
		}
	}
}

[dshulyak]

@ivan4th @poszu thanks for review, i addressed your comments

[dshulyak]

bors try

[spacemesh-bors]

try

Build failed:

• ci-status

[dshulyak]

need to understand what changes wrt retries

[dshulyak]

bors try

[spacemesh-bors]

try

Build succeeded:

• ci-status
• systest-status

[dshulyak]

bors try

[spacemesh-bors]

try

Build succeeded:

• ci-status
• systest-status

[dshulyak]

bors try

[spacemesh-bors]

try

Build succeeded:

• ci-status
• systest-status

[dshulyak]

bors merge

[spacemesh-bors]

Build failed:

• ci-status

[dshulyak]

bors cancel

#5652

[dshulyak]

bors merge

[dshulyak]

bors cancel

[spacemesh-bors]

Canceled.

[dshulyak]

bors merge

[spacemesh-bors]

Pull request successfully merged into develop.

Build succeeded:

• systest-status
• ci-status

[xearl4]

@dshulyak could you explain why atx syncer state is persisted into local.sql and not into state.sql? do i understand it correctly, that, if we want to copy a synced blockchain state.sql from one full node to another, we'd now also have to copy the atx sync state tables from local.sql?

[dshulyak]

no that information is meant only as an optimization to avoid asking peers for a data every time node boots. i would suggest not to copy that, just wait a bit longer when node boots up the first time. in future versions that data will be removed completely, and peers will checking atx consistency always in background.