-
Notifications
You must be signed in to change notification settings - Fork 214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Merged by Bors] - Hotfix for API queries for activations #5334
Conversation
Add a method to the mesh to return the valid, applied block for a given layer and return only this one
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## develop #5334 +/- ##
=========================================
+ Coverage 77.2% 77.7% +0.4%
=========================================
Files 262 265 +3
Lines 30681 30862 +181
=========================================
+ Hits 23716 23987 +271
+ Misses 5462 5362 -100
- Partials 1503 1513 +10 ☔ View full report in Codecov by Sentry. |
Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.18.0 to 1.18.1.
## Motivation Several changes listed as part of the 1.3.0 release have already been released in prior versions. ## Changes - Updated CHANGELOG to list changes at the correct versions - renamed 1.3.0 back to UNRELEASED, since more changes might be added before it is officially released ## Test Plan - n/a ## TODO - [x] Explain motivation or link existing issue(s) - [x] Test changes and document test plan - [x] Update documentation as needed - [x] Update [changelog](../CHANGELOG.md) as needed
Bumps [google-github-actions/setup-gcloud](https://github.com/google-github-actions/setup-gcloud) from 1 to 2.
We're seeing a lot of `I/O deadline` and general timeouts currently in the network. Increasing the timeouts to 25s from standard 10s could help with some of them.
## Motivation This is in anticipation for upcoming epochs, so that nodes can handle 300k+ active identities. ## Changes - Increase sizes for various wire types - Increase ATXHeader Cache to be able to hold the ATXs for Epoch 11 and Epoch 12 at the same time ## Test Plan n/a ## TODO <!-- This section should be removed when all items are complete --> - [x] Explain motivation or link existing issue(s) - [x] Test changes and document test plan - [x] Update documentation as needed - [x] Update [changelog](../CHANGELOG.md) as needed
## Motivation Reverts part of #5339 ## Changes `PutUvarint` from standard library and `github.com/multiformats/go-varint` do not behave exactly the same. With Go 1.20 to the stdlib implementation was fixed to detect short reads, but in contrast to `github.com/multiformats/go-varint` it will still not return an error when decoding non-minimal encodings of integers. ## Test Plan n/a ## TODO <!-- This section should be removed when all items are complete --> - [x] Explain motivation or link existing issue(s) - [x] Test changes and document test plan - [x] Update documentation as needed - [x] Update [changelog](../CHANGELOG.md) as needed
Bumps google.golang.org/grpc from 1.59.0 to 1.60.0.
Don't return activations from AccountMeshDataQuery Fixes #5006
This is also a hotfix. We should really return the full contents of the block with a state/status for each tx, but the gRPC model doesn't currently include this. Will handle this correctly in API v2.
api/grpcserver/mesh_service.go
Outdated
|
||
// Save activations too | ||
var activations []types.ATXID | ||
) (pbLayer *pb.Layer, err error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NIT: The named returns are not actually used (every return
statement explicitly states what values are returned). If the names here are removed you can use err
instead of err2
everywhere in this function 🙂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yea, oof, i like named returns so I added this, but our strict linting rules no longer allow this =\
type MeshAPIMockInstrumented struct { | ||
MeshAPIMock | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NIT: I know this extends MeshAPIMock
but would it make sense to instead use a gomock
Mock here instead? These handwritten mocks are really hard to maintain, because changing anything usually breaks several tests at once because they are shared between them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing this out. If we were doing this from scratch I'd agree, but the methods are all already mocked here:
go-spacemesh/api/grpcserver/grpcserver_test.go
Lines 247 to 321 in a05776e
type MeshAPIMock struct{} | |
// latest layer received. | |
func (m *MeshAPIMock) LatestLayer() types.LayerID { | |
return layerLatest | |
} | |
// latest layer approved/confirmed/applied to state | |
// The real logic here is a bit more complicated, as it depends whether the node | |
// is syncing or not. If it's not syncing, layers are applied to state as they're | |
// verified by Hare. If it's syncing, Hare is not run, and they are applied to | |
// state as they're confirmed by Tortoise and it advances pbase. This is all in | |
// flux right now so keep this simple for the purposes of testing. | |
func (m *MeshAPIMock) LatestLayerInState() types.LayerID { | |
return layerVerified | |
} | |
func (m *MeshAPIMock) ProcessedLayer() types.LayerID { | |
return layerVerified | |
} | |
func (m *MeshAPIMock) GetRewardsByCoinbase(types.Address) (rewards []*types.Reward, err error) { | |
return []*types.Reward{ | |
{ | |
Layer: layerFirst, | |
TotalReward: rewardAmount, | |
LayerReward: rewardAmount, | |
Coinbase: addr1, | |
SmesherID: rewardSmesherID, | |
}, | |
}, nil | |
} | |
func (m *MeshAPIMock) GetRewardsBySmesherId(types.NodeID) (rewards []*types.Reward, err error) { | |
return []*types.Reward{ | |
{ | |
Layer: layerFirst, | |
TotalReward: rewardAmount, | |
LayerReward: rewardAmount, | |
Coinbase: addr1, | |
SmesherID: rewardSmesherID, | |
}, | |
}, nil | |
} | |
func (m *MeshAPIMock) GetLayer(tid types.LayerID) (*types.Layer, error) { | |
if tid.After(layerCurrent) { | |
return nil, errors.New("requested layer later than current layer") | |
} else if tid.After(m.LatestLayer()) { | |
return nil, errors.New("haven't received that layer yet") | |
} | |
ballots := []*types.Ballot{ballot1} | |
blocks := []*types.Block{block1, block2, block3} | |
return types.NewExistingLayer(tid, ballots, blocks), nil | |
} | |
func (m *MeshAPIMock) GetLayerVerified(tid types.LayerID) (*types.Block, error) { | |
return block1, nil | |
} | |
func (m *MeshAPIMock) GetATXs( | |
context.Context, | |
[]types.ATXID, | |
) (map[types.ATXID]*types.VerifiedActivationTx, []types.ATXID) { | |
atxs := map[types.ATXID]*types.VerifiedActivationTx{ | |
globalAtx.ID(): globalAtx, | |
globalAtx2.ID(): globalAtx2, | |
} | |
return atxs, nil | |
} | |
func (m *MeshAPIMock) MeshHash(types.LayerID) (types.Hash32, error) { | |
return types.RandomHash(), nil | |
} |
MeshAPIMock
type on the auto-generated MockmeshAPI
type. let me know if you disagree.
Co-authored-by: Matthias Fasching <[email protected]>
Thank you @fasmat
bors merge |
Modify `MeshService.LayersQuery` to return ONLY the single, canonical block for each queried layer. Fixes performance issues for now by not querying any activations, proposals, etc. Modify `MeshService.AccountMeshDataQuery` not to query activations to prevent memory/CPU bomb ## Motivation Closes #5315 Closes #5006 ## Changes - Add method to mesh to return single, valid, applied block for a given layer - Modify `MeshService.LayersQuery` to use this new method - Additionally, remove query for activations (leave this empty for now) - Modify `MeshService.AccountMeshDataQuery` not to query activations ## Test Plan Tests updated ## TODO <!-- This section should be removed when all items are complete --> - [x] Explain motivation or link existing issue(s) - [x] Test changes and document test plan - [x] Update documentation as needed - [x] Update [changelog](../CHANGELOG.md) as needed Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matthias Fasching <[email protected]> Co-authored-by: Jędrzej Nowak <[email protected]>
Build failed:
|
To make troubleshooting systests easier
bors merge |
Modify `MeshService.LayersQuery` to return ONLY the single, canonical block for each queried layer. Fixes performance issues for now by not querying any activations, proposals, etc. Modify `MeshService.AccountMeshDataQuery` not to query activations to prevent memory/CPU bomb ## Motivation Closes #5315 Closes #5006 ## Changes - Add method to mesh to return single, valid, applied block for a given layer - Modify `MeshService.LayersQuery` to use this new method - Additionally, remove query for activations (leave this empty for now) - Modify `MeshService.AccountMeshDataQuery` not to query activations ## Test Plan Tests updated ## TODO <!-- This section should be removed when all items are complete --> - [x] Explain motivation or link existing issue(s) - [x] Test changes and document test plan - [x] Update documentation as needed - [x] Update [changelog](../CHANGELOG.md) as needed Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matthias Fasching <[email protected]> Co-authored-by: Jędrzej Nowak <[email protected]>
Build failed: |
Why this hotfix haven't merge into this release~~~,I think it is a problem, hang on long time when query a layer detail. |
bors merge |
Modify `MeshService.LayersQuery` to return ONLY the single, canonical block for each queried layer. Fixes performance issues for now by not querying any activations, proposals, etc. Modify `MeshService.AccountMeshDataQuery` not to query activations to prevent memory/CPU bomb ## Motivation Closes #5315 Closes #5006 ## Changes - Add method to mesh to return single, valid, applied block for a given layer - Modify `MeshService.LayersQuery` to use this new method - Additionally, remove query for activations (leave this empty for now) - Modify `MeshService.AccountMeshDataQuery` not to query activations ## Test Plan Tests updated ## TODO <!-- This section should be removed when all items are complete --> - [x] Explain motivation or link existing issue(s) - [x] Test changes and document test plan - [x] Update documentation as needed - [x] Update [changelog](../CHANGELOG.md) as needed Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matthias Fasching <[email protected]> Co-authored-by: Jędrzej Nowak <[email protected]> Co-authored-by: Kacper Sawicki <[email protected]>
Build failed: |
bors merge |
Modify `MeshService.LayersQuery` to return ONLY the single, canonical block for each queried layer. Fixes performance issues for now by not querying any activations, proposals, etc. Modify `MeshService.AccountMeshDataQuery` not to query activations to prevent memory/CPU bomb ## Motivation Closes #5315 Closes #5006 ## Changes - Add method to mesh to return single, valid, applied block for a given layer - Modify `MeshService.LayersQuery` to use this new method - Additionally, remove query for activations (leave this empty for now) - Modify `MeshService.AccountMeshDataQuery` not to query activations ## Test Plan Tests updated ## TODO <!-- This section should be removed when all items are complete --> - [x] Explain motivation or link existing issue(s) - [x] Test changes and document test plan - [x] Update documentation as needed - [x] Update [changelog](../CHANGELOG.md) as needed Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matthias Fasching <[email protected]> Co-authored-by: Jędrzej Nowak <[email protected]> Co-authored-by: Kacper Sawicki <[email protected]>
Pull request successfully merged into develop. Build succeeded: |
Modify `MeshService.LayersQuery` to return ONLY the single, canonical block for each queried layer. Fixes performance issues for now by not querying any activations, proposals, etc. Modify `MeshService.AccountMeshDataQuery` not to query activations to prevent memory/CPU bomb ## Motivation Closes #5315 Closes #5006 ## Changes - Add method to mesh to return single, valid, applied block for a given layer - Modify `MeshService.LayersQuery` to use this new method - Additionally, remove query for activations (leave this empty for now) - Modify `MeshService.AccountMeshDataQuery` not to query activations ## Test Plan Tests updated ## TODO <!-- This section should be removed when all items are complete --> - [x] Explain motivation or link existing issue(s) - [x] Test changes and document test plan - [x] Update documentation as needed - [x] Update [changelog](../CHANGELOG.md) as needed Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matthias Fasching <[email protected]> Co-authored-by: Jędrzej Nowak <[email protected]> Co-authored-by: Kacper Sawicki <[email protected]>
Backport #5334: Hotfix for API queries for activations
Modify
MeshService.LayersQuery
to return ONLY the single, canonical block for each queried layer. Fixes performance issues for now by not querying any activations, proposals, etc.Modify
MeshService.AccountMeshDataQuery
not to query activations to prevent memory/CPU bombMotivation
Closes #5315
Closes #5006
Changes
MeshService.LayersQuery
to use this new methodMeshService.AccountMeshDataQuery
not to query activationsTest Plan
Tests updated
TODO