Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Make simulator operations predetermined #2897

Closed
wants to merge 85 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
1c08b35
Initial pass
cwgoes Nov 14, 2018
492e9f0
Minor cleanup
cwgoes Nov 14, 2018
93242a2
untested core minting updates
rigelrozanski Nov 15, 2018
d9a53e5
fix initialMinter
rigelrozanski Nov 15, 2018
e61c357
Merge branch 'develop' into cwgoes/runtime-assertable-invariants
cwgoes Nov 15, 2018
dd4bc57
Update PENDING.md
cwgoes Nov 15, 2018
decf163
Add flag
cwgoes Nov 15, 2018
70d64ec
Fix flag, update docs
cwgoes Nov 15, 2018
7feb7aa
Update heights across state, add forZeroHeight handling to app.go
cwgoes Nov 15, 2018
079f7f4
Merge branch 'develop' into cwgoes/runtime-assertable-invariants
cwgoes Nov 15, 2018
f8ea009
Merge branch 'cwgoes/runtime-assertable-invariants' into cwgoes/expor…
cwgoes Nov 15, 2018
7c61478
Assert runtime invariants at application start, move export code
cwgoes Nov 15, 2018
bf2e525
Test fixes; 'make format'
cwgoes Nov 15, 2018
9c96e64
Bugfix height; return validators correctly
cwgoes Nov 15, 2018
481d801
Minor simulation fixes
cwgoes Nov 15, 2018
e6f05b2
Fix import-export tests, which cannot assert invariants
cwgoes Nov 15, 2018
58e026a
Simulation after import, include in CI
cwgoes Nov 15, 2018
e2a21f2
Avoid zero-validator case
cwgoes Nov 15, 2018
9a57abc
Fix height offset
cwgoes Nov 15, 2018
1cc058f
fix existing tests
rigelrozanski Nov 15, 2018
32e652f
working writing test
rigelrozanski Nov 16, 2018
2bd86c2
Merge branch 'develop' into cwgoes/export-for-zero-height
cwgoes Nov 16, 2018
7155483
Remove offset-by-one, must be some other issue
cwgoes Nov 16, 2018
834d472
'make format'
cwgoes Nov 16, 2018
7d6ff98
Add iteration functions
cwgoes Nov 16, 2018
c08f90c
Don't iterate over all accounts
cwgoes Nov 16, 2018
b9b979f
NextProvision test, some fixes
rigelrozanski Nov 16, 2018
f31a7e8
update mechanism to use average block time, PENDING, wip docs
rigelrozanski Nov 16, 2018
3dc8ab1
1 line compile fix
rigelrozanski Nov 16, 2018
5b0f3c9
finish updating docs
rigelrozanski Nov 16, 2018
5f70992
compile fix
rigelrozanski Nov 16, 2018
01a9246
address @cwgoes comments
rigelrozanski Nov 19, 2018
bb8c8ed
uint fixes
rigelrozanski Nov 19, 2018
fa469c2
benchmark using Int
rigelrozanski Nov 20, 2018
e1af6c6
use decimal
rigelrozanski Nov 20, 2018
58d34b7
Merge remote-tracking branch 'origin/develop' into rigel/blockly-mint
rigelrozanski Nov 20, 2018
8581dc4
Merge branch 'develop' into cwgoes/export-for-zero-height
cwgoes Nov 20, 2018
0f0ad9d
Address @rigelrozanski comments
cwgoes Nov 20, 2018
385cb68
Fix testcases
cwgoes Nov 20, 2018
06e7844
Add accum invariant
cwgoes Nov 20, 2018
645e023
minor invar update
rigelrozanski Nov 20, 2018
4d11313
doc update
rigelrozanski Nov 20, 2018
c6ce1e6
Merge branch 'develop' into rigel/blockly-mint
cwgoes Nov 20, 2018
9c74ae1
lint
rigelrozanski Nov 20, 2018
00b6497
Linter fix
cwgoes Nov 20, 2018
8b082e9
defensive checks
rigelrozanski Nov 20, 2018
3f8176b
some debugging output
rigelrozanski Nov 20, 2018
17e3bf5
Rename initGenesis to initFromGenesisState
cwgoes Nov 21, 2018
b8c2d43
Merge branch 'develop' into cwgoes/export-for-zero-height
cwgoes Nov 21, 2018
3cc2495
correct the defensive checks addresses
rigelrozanski Nov 21, 2018
94c888a
delegation share invariance
rigelrozanski Nov 21, 2018
1480510
...
rigelrozanski Nov 21, 2018
eb2bdcc
...
rigelrozanski Nov 21, 2018
f4782fe
del accum invariance
rigelrozanski Nov 21, 2018
d510df1
better invar output
rigelrozanski Nov 21, 2018
fb8aa02
PositiveDelegationInvariant
rigelrozanski Nov 22, 2018
2abab59
resolved Dec bug
rigelrozanski Nov 22, 2018
d41eeb1
PENDING.md
rigelrozanski Nov 22, 2018
b678bcc
Merge branch 'develop' into rigel/blockly-mint
rigelrozanski Nov 22, 2018
fffc100
test cover fix
rigelrozanski Nov 22, 2018
0da12f8
Merge branch 'rigel/blockly-mint' of https://github.com/cosmos/cosmos…
rigelrozanski Nov 22, 2018
ccfbfcf
...
rigelrozanski Nov 22, 2018
7c97582
missing iter.Close(), line length, reduce tab
rigelrozanski Nov 22, 2018
e38af2a
typo
rigelrozanski Nov 22, 2018
faa14f1
Merge branch 'develop' into cwgoes/export-for-zero-height
cwgoes Nov 22, 2018
822944b
Set validator.UnbondingHeight to 0
cwgoes Nov 22, 2018
dbf5a4b
Remove defer() usage
cwgoes Nov 22, 2018
685c55d
Tiny cleanup
cwgoes Nov 22, 2018
7aa1182
Merge branch 'rigel/blockly-mint' into cwgoes/export-for-zero-height
cwgoes Nov 22, 2018
b466775
demonstrate defensive accum height for zero-delegations
rigelrozanski Nov 22, 2018
05ddb7b
add back in the zero-delegation invar
rigelrozanski Nov 22, 2018
607eaca
note on defensive code
rigelrozanski Nov 22, 2018
3094324
Turn the ability to withdraw rewards into an invariant
cwgoes Nov 23, 2018
b1a14c1
Fix linter nit
cwgoes Nov 23, 2018
3674cb3
Correctly set height
cwgoes Nov 23, 2018
d2f59fc
Update x/distribution/types/delegator_info.go
alexanderbez Nov 23, 2018
c0239e0
Update x/distribution/types/delegator_info.go
alexanderbez Nov 23, 2018
4959289
Update Decimal Format()
jaekwon Nov 24, 2018
9a66906
Clip withdrawal tokens
jaekwon Nov 24, 2018
0088c22
oops
jaekwon Nov 24, 2018
acc5349
Make simulator operations predetermined
jaekwon Nov 25, 2018
308ee7e
error upon invariance break
jaekwon Nov 25, 2018
2ae8b2e
Use new dbm.ReverseIterator API
jaekwon Nov 26, 2018
19cd1f3
Merge branch 'jae/export-for-zero-height' into jae/simulator_more_det…
jaekwon Nov 26, 2018
cd0d21c
Simulation speed improvements
jaekwon Nov 26, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
missing iter.Close(), line length, reduce tab
rigelrozanski committed Nov 22, 2018
commit 7c975828d5a3e84a4f0953d0acead6c1431bdc01
200 changes: 106 additions & 94 deletions cmd/gaia/app/export.go
Original file line number Diff line number Diff line change
@@ -17,104 +17,14 @@ import (
)

// export the state of gaia for a genesis file
func (app *GaiaApp) ExportAppStateAndValidators(forZeroHeight bool) (appState json.RawMessage, validators []tmtypes.GenesisValidator, err error) {
func (app *GaiaApp) ExportAppStateAndValidators(forZeroHeight bool) (
appState json.RawMessage, validators []tmtypes.GenesisValidator, err error) {

// as if they could withdraw from the start of the next block
ctx := app.NewContext(true, abci.Header{Height: app.LastBlockHeight()})

// prepare for fresh start at zero height
if forZeroHeight {

/* TODO XXX check some invariants */

height := ctx.BlockHeight()

valAccum := sdk.ZeroDec()
app.distrKeeper.IterateValidatorDistInfos(ctx, func(_ int64, vdi distr.ValidatorDistInfo) bool {
lastValPower := app.stakeKeeper.GetLastValidatorPower(ctx, vdi.OperatorAddr)
valAccum = valAccum.Add(vdi.GetValAccum(height, sdk.NewDecFromInt(lastValPower)))
return false
})

lastTotalPower := sdk.NewDecFromInt(app.stakeKeeper.GetLastTotalPower(ctx))
totalAccum := app.distrKeeper.GetFeePool(ctx).GetTotalValAccum(height, lastTotalPower)

if !totalAccum.Equal(valAccum) {
panic(fmt.Errorf("validator accum invariance: \n\tfee pool totalAccum: %v"+
"\n\tvalidator accum \t%v\n", totalAccum.String(), valAccum.String()))
}

fmt.Printf("accum invariant ok!\n")

/* END TODO XXX */

/* Handle fee distribution state. */

// withdraw all delegator & validator rewards
app.distrKeeper.IterateValidatorDistInfos(ctx, func(_ int64, valInfo distr.ValidatorDistInfo) (stop bool) {
err := app.distrKeeper.WithdrawValidatorRewardsAll(ctx, valInfo.OperatorAddr)
if err != nil {
panic(err)
}
return false
})
app.distrKeeper.IterateDelegationDistInfos(ctx, func(_ int64, distInfo distr.DelegationDistInfo) (stop bool) {
err := app.distrKeeper.WithdrawDelegationReward(ctx, distInfo.DelegatorAddr, distInfo.ValOperatorAddr)
if err != nil {
panic(err)
}
return false
})

// delete all distribution infos
// these will be recreated in InitGenesis
app.distrKeeper.RemoveValidatorDistInfos(ctx)
app.distrKeeper.RemoveDelegationDistInfos(ctx)

// assert that the fee pool is empty
feePool := app.distrKeeper.GetFeePool(ctx)
if !feePool.TotalValAccum.Accum.IsZero() {
panic("unexpected leftover validator accum")
}
bondDenom := app.stakeKeeper.GetParams(ctx).BondDenom
if !feePool.ValPool.AmountOf(bondDenom).IsZero() {
panic(fmt.Sprintf("unexpected leftover validator pool coins: %v", feePool.ValPool.AmountOf(bondDenom).String()))
}

// reset fee pool height, save fee pool
feePool.TotalValAccum.UpdateHeight = 0
app.distrKeeper.SetFeePool(ctx, feePool)

/* Handle stake state. */

// iterate through validators by power descending, reset bond height, update bond intra-tx counter
store := ctx.KVStore(app.keyStake)
iter := sdk.KVStoreReversePrefixIterator(store, stake.ValidatorsByPowerIndexKey)
counter := int16(0)
for ; iter.Valid(); iter.Next() {
addr := sdk.ValAddress(iter.Value())
validator, found := app.stakeKeeper.GetValidator(ctx, addr)
if !found {
panic("expected validator, not found")
}
validator.BondHeight = 0
validator.BondIntraTxCounter = counter
// AFAICT we do not need to reset unbonding height since it is not used.
app.stakeKeeper.SetValidator(ctx, validator)
counter++
}

/* Handle slashing state. */

// we have to clear the slashing periods, since they reference heights
app.slashingKeeper.DeleteValidatorSlashingPeriods(ctx)

// reset start height on signing infos
app.slashingKeeper.IterateValidatorSigningInfos(ctx, func(addr sdk.ConsAddress, info slashing.ValidatorSigningInfo) (stop bool) {
info.StartHeight = 0
app.slashingKeeper.SetValidatorSigningInfo(ctx, addr, info)
return false
})

prepForZeroHeightGenesis(ctx)
}

// iterate to get the accounts
@@ -142,3 +52,105 @@ func (app *GaiaApp) ExportAppStateAndValidators(forZeroHeight bool) (appState js
validators = stake.WriteValidators(ctx, app.stakeKeeper)
return appState, validators, nil
}

// prepare for fresh start at zero height
func prepForZeroHeightGenesis(ctx sdk.Context) {

/* TODO XXX check some invariants */

height := ctx.BlockHeight()

valAccum := sdk.ZeroDec()
vdiIter := func(_ int64, vdi distr.ValidatorDistInfo) bool {
lastValPower := app.stakeKeeper.GetLastValidatorPower(ctx, vdi.OperatorAddr)
valAccum = valAccum.Add(vdi.GetValAccum(height, sdk.NewDecFromInt(lastValPower)))
return false
}
app.distrKeeper.IterateValidatorDistInfos(ctx, vdiIter)

lastTotalPower := sdk.NewDecFromInt(app.stakeKeeper.GetLastTotalPower(ctx))
totalAccum := app.distrKeeper.GetFeePool(ctx).GetTotalValAccum(height, lastTotalPower)

if !totalAccum.Equal(valAccum) {
panic(fmt.Errorf("validator accum invariance: \n\tfee pool totalAccum: %v"+
"\n\tvalidator accum \t%v\n", totalAccum.String(), valAccum.String()))
}

fmt.Printf("accum invariant ok!\n")

/* END TODO XXX */

/* Handle fee distribution state. */

// withdraw all delegator & validator rewards
vdiIter = func(_ int64, valInfo distr.ValidatorDistInfo) (stop bool) {
err := app.distrKeeper.WithdrawValidatorRewardsAll(ctx, valInfo.OperatorAddr)
if err != nil {
panic(err)
}
return false
}
app.distrKeeper.IterateValidatorDistInfos(ctx, vdiIter)

ddiIter := func(_ int64, distInfo distr.DelegationDistInfo) (stop bool) {
err := app.distrKeeper.WithdrawDelegationReward(
ctx, distInfo.DelegatorAddr, distInfo.ValOperatorAddr)
if err != nil {
panic(err)
}
return false
}
app.distrKeeper.IterateDelegationDistInfos(ctx, ddiIter)

// delete all distribution infos
// these will be recreated in InitGenesis
app.distrKeeper.RemoveValidatorDistInfos(ctx)
app.distrKeeper.RemoveDelegationDistInfos(ctx)

// assert that the fee pool is empty
feePool := app.distrKeeper.GetFeePool(ctx)
if !feePool.TotalValAccum.Accum.IsZero() {
panic("unexpected leftover validator accum")
}
bondDenom := app.stakeKeeper.GetParams(ctx).BondDenom
if !feePool.ValPool.AmountOf(bondDenom).IsZero() {
panic(fmt.Sprintf("unexpected leftover validator pool coins: %v",
feePool.ValPool.AmountOf(bondDenom).String()))
}

// reset fee pool height, save fee pool
feePool.TotalValAccum.UpdateHeight = 0
app.distrKeeper.SetFeePool(ctx, feePool)

/* Handle stake state. */

// iterate through validators by power descending, reset bond height, update bond intra-tx counter
store := ctx.KVStore(app.keyStake)
iter := sdk.KVStoreReversePrefixIterator(store, stake.ValidatorsByPowerIndexKey)
counter := int16(0)
for ; iter.Valid(); iter.Next() {
addr := sdk.ValAddress(iter.Value())
validator, found := app.stakeKeeper.GetValidator(ctx, addr)
if !found {
panic("expected validator, not found")
}
validator.BondHeight = 0
validator.BondIntraTxCounter = counter
// AFAICT we do not need to reset unbonding height since it is not used.
app.stakeKeeper.SetValidator(ctx, validator)
counter++
}
iter.Close()

/* Handle slashing state. */

// we have to clear the slashing periods, since they reference heights
app.slashingKeeper.DeleteValidatorSlashingPeriods(ctx)

// reset start height on signing infos
app.slashingKeeper.IterateValidatorSigningInfos(ctx, func(addr sdk.ConsAddress, info slashing.ValidatorSigningInfo) (stop bool) {
info.StartHeight = 0
app.slashingKeeper.SetValidatorSigningInfo(ctx, addr, info)
return false
})
}