Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(perp): Remove unnecessary panics | #1 #1126

Merged
merged 12 commits into from
Jan 4, 2023

Conversation

Unique-Divine
Copy link
Member

@Unique-Divine Unique-Divine commented Jan 2, 2023

Description

Remove unnecessary panics in x/perp

  • Related to refactor: Remove unnecessary panics #1125
  • test(oracle): Makes TestFuzz_PickReferencePair stop harassing us
  • feat(common): adds convenience functions and tests for:
    1. Combining errors with CombineErrors and CombineErrorsGeneric. The difference
    2. Converting values to error type if they are (1) are strings, (2) fit the fmt.Stringer interface and have .String() methods, or (3) are already errors.
    3. Managing sets of strings with StringSet

Adds TryCatch which is used in later PRs for #1125

TryCatch is an implementation of the try-catch block from languages like C++ and JS.
Given a 'callback' function, TryCatch defers and recovers from any panics or
errors, allowing one to safely handle multiple panics in succession.

Typically, you'll write something like: err := TryCatch(aRiskyFunction)()

Usage example:

var calmPanic error = TryCatch(func() {
  panic("something crazy")
})()
fmt.Println(calmPanic.Error()) // prints "something crazy"

Note that TryCatch doesn't return an error. It returns a function that returns
an error. Only when you call the output of TryCatch will it "feel" like a
try-catch block from other languages.

This means that TryCatch can be used to restart go routines that panic as well.


Debugging notes on TestFuzz_ReferencePair

Solution (found this after some tinkering)

The familiar error (cases A and B) comes from iterating over a collections.Range[string] that includes the blank key string, "" (hexadecimal 00). See TestKeeper_GetVoteTargets, which now covers this case in its tests


Case A - Use whitelisted pair with string length 1

Attempting to Pair.Insert a key of length 0 seems valid according to this workflow run.

--- FAIL: TestFuzz_PickReferencePair (0.01s)
    tally_fuzz_test.go:136: 
        	Error Trace:	/home/runner/work/nibiru/nibiru/x/oracle/keeper/tally_fuzz_test.go:136
        	Error:      	func (assert.PanicTestFunc)(0x1120680) should panic
        	            		Panic value:	<nil>
        	Test:       	TestFuzz_PickReferencePair
        	Messages:   	attempted to insert key: 
FAIL
coverage: 91.0% of statements
FAIL	github.com/NibiruChain/nibiru/x/oracle/keeper	0.288s

Case B - Use whitelisted pair with string length 0

In this call of stringKey.Decode that results from RemoveInvalidBallots, we can get a different error in another workflow:

image

Full error stack:

--- FAIL: TestFuzz_PickReferencePair (0.01s)
    tally_fuzz_test.go:135: 
        	Error Trace:	/home/runner/work/nibiru/nibiru/x/oracle/keeper/tally_fuzz_test.go:135
        	Error:      	func (assert.PanicTestFunc)(0x1120340) should not panic
        	            		Panic value:	invalid StringKey bytes. StringKey must be at least length 2. 
        	            	bytesAsHex: 00
        	            		Panic stack:	goroutine 155 [running]:
        	            	runtime/debug.Stack()
        	            		/opt/hostedtoolcache/go/1.18.9/x64/src/runtime/debug/stack.go:24 +0x65
        	            	github.com/stretchr/testify/assert.didPanic.func1()
        	            		/home/runner/go/pkg/mod/github.com/stretchr/[email protected]/assert/assertions.go:1050 +0x6c
        	            	panic({0x12529a0, 0xc0011ca2b0})
        	            		/opt/hostedtoolcache/go/1.18.9/x64/src/runtime/panic.go:838 +0x207
        	            	github.com/NibiruChain/collections.stringKey.Decode({}, {0xc00117da28?, 0xc00117da28?, 0x0?})
        	            		/home/runner/go/pkg/mod/github.com/!nibiru!chain/[email protected]/keys.go:40 +0x1d5
        	            	github.com/NibiruChain/collections.Iterator[...].Key({{0x1c2cac0, 0x25c50c0}, {0x1c2e4a0, 0x25c54c8}, {0x1c32130, 0xc00117ea20}, {0x0, 0x0, 0x0}})
        	            		/home/runner/go/pkg/mod/github.com/!nibiru!chain/[email protected]/iter.go:174 +0xdb
        	            	github.com/NibiruChain/collections.Iterator[...].Keys({{0x1c2cac0, 0x25c50c0}, {0x1c2e4a0, 0x25c54c8}, {0x1c32130, 0xc00117ea20}, {0x0, 0x0, 0x0}})
        	            		/home/runner/go/pkg/mod/github.com/!nibiru!chain/[email protected]/iter.go:198 +0x205
        	            	github.com/NibiruChain/collections.KeySetIterator[...].Keys(...)
        	            		/home/runner/go/pkg/mod/github.com/!nibiru!chain/[email protected]/keyset.go:61
        	            	github.com/NibiruChain/nibiru/x/oracle/keeper.Keeper.GetWhitelistedPairs({{0x7fb1ccf2dfd8, 0xc0010e8760}, {0x1c1e798, 0xc0010e8720}, {{0x7fb1ccf2dfd8, 0xc0010e8760}, 0xc000f82d50, {0x1c1e798, 0xc0010e8700}, {0x1c1e7e8, ...}, ...}, ...}, ...)
        	            		/home/runner/work/nibiru/nibiru/x/oracle/keeper/vote_target.go:16 +0x330
        	            	github.com/NibiruChain/nibiru/x/oracle/keeper.Keeper.getWhitelistedPairsMap({{0x7fb1ccf2dfd8, 0xc0010e8760}, {0x1c1e798, 0xc0010e8720}, {{0x7fb1ccf2dfd8, 0xc0010e8760}, 0xc000f82d50, {0x1c1e798, 0xc0010e8700}, {0x1c1e7e8, ...}, ...}, ...}, ...)
        	            		/home/runner/work/nibiru/nibiru/x/oracle/keeper/update_exchange_rates.go:77 +0xb8
        	            	github.com/NibiruChain/nibiru/x/oracle/keeper.Keeper.RemoveInvalidBallots({{0x7fb1ccf2dfd8, 0xc0010e8760}, {0x1c1e798, 0xc0010e8720}, {{0x7fb1ccf2dfd8, 0xc0010e8760}, 0xc000f82d50, {0x1c1e798, 0xc0010e8700}, {0x1c1e7e8, ...}, ...}, ...}, ...)
        	            		/home/runner/work/nibiru/nibiru/x/oracle/keeper/tally.go:54 +0xb8
        	            	github.com/NibiruChain/nibiru/x/oracle/keeper.TestFuzz_PickReferencePair.func7()
        	            		/home/runner/work/nibiru/nibiru/x/oracle/keeper/tally_fuzz_test.go:136 +0x98
        	            	github.com/stretchr/testify/assert.didPanic(0xc0010b9d40?)
        	            		/home/runner/go/pkg/mod/github.com/stretchr/[email protected]/assert/assertions.go:1055 +0x8c
        	            	github.com/stretchr/testify/assert.NotPanics({0x1c1bea0, 0xc0010b9d40}, 0xc001162eb8, {0xc001135678, 0x2, 0x2})
        	            		/home/runner/go/pkg/mod/github.com/stretchr/[email protected]/assert/assertions.go:1126 +0x7b
        	            	github.com/stretchr/testify/require.NotPanics({0x1c1fa58, 0xc0010b9d40}, 0xc000f82f00?, {0xc001135678, 0x2, 0x2})
        	            		/home/runner/go/pkg/mod/github.com/stretchr/[email protected]/require/require.go:1486 +0x8b
        	            	github.com/NibiruChain/nibiru/x/oracle/keeper.TestFuzz_PickReferencePair(0xc0010b9d40?)
        	            		/home/runner/work/nibiru/nibiru/x/oracle/keeper/tally_fuzz_test.go:135 +0x5b4
        	            	testing.tRunner(0xc0010b9d40, 0x1a99458)
        	            		/opt/hostedtoolcache/go/1.18.9/x64/src/testing/testing.go:[143](https://github.com/NibiruChain/nibiru/actions/runs/3825137204/jobs/6507881191#step:4:144)9 +0x102
        	            	created by testing.(*T).Run
        	            		/opt/hostedtoolcache/go/1.18.9/x64/src/testing/testing.go:[148](https://github.com/NibiruChain/nibiru/actions/runs/3825137204/jobs/6507881191#step:4:149)6 +0x35f
        	Test:       	TestFuzz_PickReferencePair
        	Messages:   	
FAIL
coverage: 91.0% of statements
FAIL	github.com/NibiruChain/nibiru/x/oracle/keeper	0.347s

@Unique-Divine Unique-Divine requested a review from a team as a code owner January 2, 2023 21:07
@Unique-Divine Unique-Divine force-pushed the realu/remove-panics-perp branch from 6771802 to 5bed36e Compare January 2, 2023 21:08
@Unique-Divine Unique-Divine changed the title refactor(perp): Remove unnecessary panics refactor(perp): Remove unnecessary panics - #1 Jan 2, 2023
@Unique-Divine Unique-Divine changed the title refactor(perp): Remove unnecessary panics - #1 refactor(perp): Remove unnecessary panics | #1 Jan 2, 2023
@Unique-Divine Unique-Divine added ⚠️ do not merge On hold for merge or not desired tests and removed ⚠️ do not merge On hold for merge or not desired labels Jan 2, 2023
x/common/common_test.go Outdated Show resolved Hide resolved
x/common/error.go Outdated Show resolved Hide resolved
x/common/error.go Outdated Show resolved Hide resolved
x/common/error.go Outdated Show resolved Hide resolved
x/oracle/keeper/tally_fuzz_test.go Outdated Show resolved Hide resolved
x/oracle/keeper/tally_fuzz_test.go Outdated Show resolved Hide resolved
// Cases for duck typing at runtime

// case: error
if tcErr := TryCatch(func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't the type switch statement catch all these cases already?

Copy link
Member Author

@Unique-Divine Unique-Divine Jan 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's how I thought it would work at first too. The reason it skips those blocks is that any and interface{} are actual types. They don't actual mean "any type" but rather "an object of type any".

Let's say we pass in an argument, arg, that fits the fmt.Stringer interface like sdk.Coin or sdk.AccAddress. When arg enters this function, it won't actually have the type of fmt.Stringer, so it will pass all of the case statements and end up in default.

Then, when it reaches the arg := arg.(fmt.Stringer) block, it will enter the function with fmt.Stringer as its value for arg.(type). Since arg actually is an instance of that fits this interface, the TryCatch will run smoothly.

However, it has to guess by manually attempting to cast as each type. Go assumes the type casting will work and forces the object to have a certain type, then it panics if something doesn't make sense, then defers, then recovers, and finally moves to the next TryCatch.

This basically shows why Python is so slow.

x/common/error_test.go Outdated Show resolved Hide resolved
@Unique-Divine Unique-Divine merged commit 0c432ed into master Jan 4, 2023
@Unique-Divine Unique-Divine deleted the realu/remove-panics-perp branch January 4, 2023 07:28
Unique-Divine added a commit that referenced this pull request Jan 6, 2023
* chore: re-run linter

* test(oracle): stop the tyrannical behavior of tally_fuzz_test.go

* test(oracle): try assert.NotPanics for keys of length 0

* test(oracle): try assert.NotPanics for keys of length 0

* feat,test(common): Create StringSet for easy set management

* test(oracle): Increase coverage and end the collections panics in
GetWhitelistedPairs

* fix(oracle): small bug in TestFuzz_PickReferencePair

* test,feat(common): address PR feedback and add a more generic version fo CombineErrors

* linter

* feat(oracle): add twap to oracle and unwire pricefeed (#1120)

* unwire pricefeed and add twap

* refactor(perp): Remove unnecessary panics | #1 (#1126)

* remove unnecessary panics in x/perp

* change log

* chore: re-run linter

* test(oracle): stop the tyrannical behavior of tally_fuzz_test.go

* test(oracle): try assert.NotPanics for keys of length 0

* feat,test(common): Create StringSet for easy set management

* test(oracle): Increase coverage and end the collections panics in
GetWhitelistedPairs

* fix(oracle): small bug in TestFuzz_PickReferencePair

* test,feat(common): address PR feedback and add a more generic version fo CombineErrors

* linter

* Update x/common/error_test.go

Co-authored-by: NibiruHeisenberg <[email protected]>

Co-authored-by: NibiruHeisenberg <[email protected]>

* fix: distribution module account name (#1131)

* refactor(dex): remove unnecessary panics

* refactor(stablecoin): remove unnecessary panics

Co-authored-by: Matthias <[email protected]>
Co-authored-by: NibiruHeisenberg <[email protected]>
Unique-Divine added a commit that referenced this pull request Jan 10, 2023
* remove unnecessary panics in x/perp

* refactor(dex): remove panics in liquidity.go

* change log

* chore: change log

* refactor: remove panics from x/dex (#1133)

* chore: re-run linter

* test(oracle): stop the tyrannical behavior of tally_fuzz_test.go

* test(oracle): try assert.NotPanics for keys of length 0

* test(oracle): try assert.NotPanics for keys of length 0

* feat,test(common): Create StringSet for easy set management

* test(oracle): Increase coverage and end the collections panics in
GetWhitelistedPairs

* fix(oracle): small bug in TestFuzz_PickReferencePair

* test,feat(common): address PR feedback and add a more generic version fo CombineErrors

* linter

* feat(oracle): add twap to oracle and unwire pricefeed (#1120)

* unwire pricefeed and add twap

* refactor(perp): Remove unnecessary panics | #1 (#1126)

* remove unnecessary panics in x/perp

* change log

* chore: re-run linter

* test(oracle): stop the tyrannical behavior of tally_fuzz_test.go

* test(oracle): try assert.NotPanics for keys of length 0

* feat,test(common): Create StringSet for easy set management

* test(oracle): Increase coverage and end the collections panics in
GetWhitelistedPairs

* fix(oracle): small bug in TestFuzz_PickReferencePair

* test,feat(common): address PR feedback and add a more generic version fo CombineErrors

* linter

* Update x/common/error_test.go

Co-authored-by: NibiruHeisenberg <[email protected]>

Co-authored-by: NibiruHeisenberg <[email protected]>

* fix: distribution module account name (#1131)

* refactor(dex): remove unnecessary panics

* refactor(stablecoin): remove unnecessary panics

Co-authored-by: Matthias <[email protected]>
Co-authored-by: NibiruHeisenberg <[email protected]>

Co-authored-by: Matthias <[email protected]>
Co-authored-by: NibiruHeisenberg <[email protected]>
k-yang added a commit that referenced this pull request Jan 11, 2023
…sible form public fns (#1134)

* remove unnecessary panics in x/perp

* refactor(dex): remove panics in liquidity.go

* change log

* chore: change log

* chore: re-run linter

* test(oracle): stop the tyrannical behavior of tally_fuzz_test.go

* test(oracle): try assert.NotPanics for keys of length 0

* test(oracle): try assert.NotPanics for keys of length 0

* feat,test(common): Create StringSet for easy set management

* test(oracle): Increase coverage and end the collections panics in
GetWhitelistedPairs

* fix(oracle): small bug in TestFuzz_PickReferencePair

* test,feat(common): address PR feedback and add a more generic version fo CombineErrors

* linter

* refactor(dex): remove unnecessary panics

* refactor(stablecoin): remove unnecessary panics

* refactor(perp): make calls of public fns from vpool less dangerous in
x/perp

* checkpoint #wip vpool

* fix changelog issue from merge conflict

* refactor: remove panics from x/dex (#1133)

* chore: re-run linter

* test(oracle): stop the tyrannical behavior of tally_fuzz_test.go

* test(oracle): try assert.NotPanics for keys of length 0

* test(oracle): try assert.NotPanics for keys of length 0

* feat,test(common): Create StringSet for easy set management

* test(oracle): Increase coverage and end the collections panics in
GetWhitelistedPairs

* fix(oracle): small bug in TestFuzz_PickReferencePair

* test,feat(common): address PR feedback and add a more generic version fo CombineErrors

* linter

* feat(oracle): add twap to oracle and unwire pricefeed (#1120)

* unwire pricefeed and add twap

* refactor(perp): Remove unnecessary panics | #1 (#1126)

* remove unnecessary panics in x/perp

* change log

* chore: re-run linter

* test(oracle): stop the tyrannical behavior of tally_fuzz_test.go

* test(oracle): try assert.NotPanics for keys of length 0

* feat,test(common): Create StringSet for easy set management

* test(oracle): Increase coverage and end the collections panics in
GetWhitelistedPairs

* fix(oracle): small bug in TestFuzz_PickReferencePair

* test,feat(common): address PR feedback and add a more generic version fo CombineErrors

* linter

* Update x/common/error_test.go

Co-authored-by: NibiruHeisenberg <[email protected]>

Co-authored-by: NibiruHeisenberg <[email protected]>

* fix: distribution module account name (#1131)

* refactor(dex): remove unnecessary panics

* refactor(stablecoin): remove unnecessary panics

Co-authored-by: Matthias <[email protected]>
Co-authored-by: NibiruHeisenberg <[email protected]>

* test,refactor(vpool): make it impossible to write invalid pools into
state.

* refactor: Make it impossible to cause panics in vpool from public fns in
the perp module

* chore(CHANGELOG)

Co-authored-by: Matthias <[email protected]>
Co-authored-by: NibiruHeisenberg <[email protected]>
Co-authored-by: Kevin Yang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants