Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

txnbuild: Add support for new CAP-21 preconditions. #4303

Merged
merged 19 commits into from
Mar 25, 2022
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions txnbuild/ledgerbounds.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
package txnbuild

import "github.com/stellar/go/support/errors"

// Ledgerbounds represent a transaction precondition that controls the ledger
// range for which a transaction is valid. Setting MaxLedger = 0 indicates there
// is no maximum ledger.
type Ledgerbounds struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: Ledgerbounds => LedgerBounds

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consistency with txnbuild.Timebounds? 🤔

Copy link
Member

@leighmcculloch leighmcculloch Mar 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You know, we could safely fix the Timebounds typo as part of this protocol release. By doing these things:

  • We're already moving the Timebounds field, so simply name the new field TimeBounds.
  • We can rename txnbuild.Timebounds to txnbuild.TimeBounds.
  • We can use a type alias to specify that txnbuild.Timebounds is the same type as txnbuild.TimeBounds. Just add this:
    type Timebounds = TimeBounds
    The = will make it so that whenever the compiler sees Timebounds it'll pretend it saw TimeBounds. All the functions will be automatically lifted. No type casting required because they are one type, not two. Note this is different to type definitions that do not have the equals.

Copy link
Contributor Author

@Shaptic Shaptic Mar 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm good with that! We still have inconsistencies (see #4297 (comment)), and we can't rename the constructors w/o more breaking changes (e.g. NewTimebounds), but it's still progress 👷‍♂️. Both renames done in latest push 👍

MinLedger uint32
MaxLedger uint32
}

func (lb *Ledgerbounds) Validate() error {
if lb.MaxLedger > 0 && lb.MaxLedger < lb.MinLedger {
return errors.New("invalid ledgerbound: max ledger < min ledger")
}

return nil
}
153 changes: 153 additions & 0 deletions txnbuild/preconditions.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
package txnbuild

import (
"github.com/stellar/go/support/errors"
"github.com/stellar/go/xdr"
)

// Preconditions is a container for all transaction preconditions.
type Preconditions struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the benefit for using this abstraction model rather than using xdr.PreconditionsV2 direct? This and TimeBounds and LedgerBounds appear to be 1-1 proxy of their xdr counterparts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, didn't see this comment for some reason. They're 1-to-1 now, but the txnbuild package in general avoids exposing XDR to SDK consumers.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, see that de-coupling at top level, sounds good, there are nested references from here into xdr types such as SignerKey and Duration, and was looking at it from bottom up, thanks!

// Transaction is only valid during a certain time range. This is private
// because it should mirror the one set via TransactionParams, and this
// association should be done via `NewPreconditions()`.
timebounds *Timebounds
// Transaction is valid for ledger numbers n such that minLedger <= n <
// maxLedger (if maxLedger == 0, then only minLedger is checked)
Ledgerbounds *Ledgerbounds
// If nil, the transaction is only valid when sourceAccount's sequence
// number "N" is seqNum - 1. Otherwise, valid when N satisfies minSeqNum <=
// N < tx.seqNum.
MinSequenceNumber *int64
// Transaction is valid if the current ledger time is at least
// minSequenceNumberAge greater than the source account's seqTime.
MinSequenceNumberAge xdr.Duration
// Transaction is valid if the current ledger number is at least
// minSequenceNumberLedgerGap greater than the source account's seqLedger.
MinSequenceNumberLedgerGap uint32
// Transaction is valid if there is a signature corresponding to every
// Signer in this array, even if the signature is not otherwise required by
// the source account or operations.
ExtraSigners []xdr.SignerKey
}

// NewPreconditions creates a set of preconditions with timebounds enabled
func NewPreconditions(timebounds *Timebounds) Preconditions {
cond := Preconditions{}
if err := cond.SetTimebounds(timebounds); err != nil {
panic(err)
}
return cond
}

func NewPreconditionsFromTimebounds(minTime, maxTime int64) Preconditions {
tb := NewTimebounds(minTime, maxTime)
return NewPreconditions(&tb)
}

// SetTimebounds enables the timebound precondition.
//
// Note that timebounds are a *required* precondition, but they're passed here
// by pointer in order to align with `TransactionParams.Timebound`.
func (cond *Preconditions) SetTimebounds(timebounds *Timebounds) error {
Shaptic marked this conversation as resolved.
Show resolved Hide resolved
if timebounds == nil {
return errors.New("timebounds are required")
}

if err := timebounds.Validate(); err != nil {
return err
}

if cond.timebounds != nil {
// only fail if they differ
if cond.timebounds.MinTime != timebounds.MinTime ||
cond.timebounds.MaxTime != timebounds.MaxTime {
return errors.New("timebounds set twice")
}
Shaptic marked this conversation as resolved.
Show resolved Hide resolved
}

cond.timebounds = timebounds
return nil
}

func (cond *Preconditions) Timebounds() Timebounds {
return *cond.timebounds
}

// Validate ensures that all enabled preconditions are valid.
func (cond *Preconditions) Validate() error {
var err error

if err = cond.timebounds.Validate(); err != nil {
return err
}

if ok := cond.ValidateSigners(); !ok {
return errors.New("invalid signers")
}

if cond.Ledgerbounds != nil {
err = cond.Ledgerbounds.Validate()
if err != nil {
return err
}
}

return nil
}

func (cond *Preconditions) ValidateSigners() bool {
Shaptic marked this conversation as resolved.
Show resolved Hide resolved
return len(cond.ExtraSigners) <= 2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this validation rule worth maintaining on client side, or let it get submitted and have core return txsub errors since it is source of truth on this aspect?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will already blow up client-side because the XDR won't encode (not stellar core validation):

SignerKey extraSigners<2>;

this was just being a little friendlier about it, since it's not fixed to be 2 w/in Preconditions. no strong opinion on keeping it, though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting, on the java sdk side, I checked the generated java from xdrgen, PreconditionsV2.java, it doesn't capture the SignerKey extraSigners<2> size limit like it is on go with xdrmaxsize, I may need to hardcode that as a constant or just document that it's by default 2 and submission will be final validation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's interesting - seems like a limitation of the generator? 🤔 definitely think it's worth a better error message from the SDK if possible

}

// HasV2Conditions determines whether or not this has conditions on top of
// the (required) timebound precondition.
func (cond *Preconditions) HasV2Conditions() bool {
Shaptic marked this conversation as resolved.
Show resolved Hide resolved
return (cond.Ledgerbounds != nil ||
cond.MinSequenceNumber != nil ||
cond.MinSequenceNumberAge > xdr.Duration(0) ||
cond.MinSequenceNumberLedgerGap > 0 ||
len(cond.ExtraSigners) > 0)
}

// BuildXDR will create a precondition structure that varies depending on
// whether or not there are additional preconditions besides timebounds (which
// are required).
func (cond *Preconditions) BuildXDR() xdr.Preconditions {
xdrCond := xdr.Preconditions{}
xdrTimeBounds := xdr.TimeBounds{
MinTime: xdr.TimePoint(cond.timebounds.MinTime),
MaxTime: xdr.TimePoint(cond.timebounds.MaxTime),
}

// Only build PRECOND_V2 structure if we need to
if cond.HasV2Conditions() {
xdrPrecond := xdr.PreconditionsV2{
TimeBounds: &xdrTimeBounds,
MinSeqAge: cond.MinSequenceNumberAge,
MinSeqLedgerGap: xdr.Uint32(cond.MinSequenceNumberLedgerGap),
ExtraSigners: cond.ExtraSigners,
}

// micro-optimization: if the ledgerbounds will always succeed, omit them
if cond.Ledgerbounds != nil && !(cond.Ledgerbounds.MinLedger == 0 &&
cond.Ledgerbounds.MaxLedger == 0) {
xdrPrecond.LedgerBounds = &xdr.LedgerBounds{
MinLedger: xdr.Uint32(cond.Ledgerbounds.MinLedger),
MaxLedger: xdr.Uint32(cond.Ledgerbounds.MaxLedger),
}
}

if cond.MinSequenceNumber != nil {
seqNum := xdr.SequenceNumber(*cond.MinSequenceNumber)
xdrPrecond.MinSeqNum = &seqNum
}

xdrCond.Type = xdr.PreconditionTypePrecondV2
xdrCond.V2 = &xdrPrecond
} else {
xdrCond.Type = xdr.PreconditionTypePrecondTime
xdrCond.TimeBounds = &xdrTimeBounds
}

return xdrCond
}
173 changes: 173 additions & 0 deletions txnbuild/preconditions_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
package txnbuild

import (
"testing"

"github.com/stellar/go/xdr"
"github.com/stretchr/testify/assert"
)

var Signers = []xdr.SignerKey{
xdr.MustSigner("GAOQJGUAB7NI7K7I62ORBXMN3J4SSWQUQ7FOEPSDJ322W2HMCNWPHXFB"),
xdr.MustSigner("GA7QYNF7SOWQ3GLR2BGMZEHXAVIRZA4KVWLTJJFC7MGXUA74P7UJVSGZ"),
xdr.MustSigner("PA7QYNF7SOWQ3GLR2BGMZEHXAVIRZA4KVWLTJJFC7MGXUA74P7UJUAAAAAQACAQDAQCQMBYIBEFAWDANBYHRAEISCMKBKFQXDAMRUGY4DUPB6IBZGM"),
}

var (
seqNum = int64(14)
xdrSeqNum = xdr.SequenceNumber(seqNum)
xdrCond = xdr.Preconditions{
Type: xdr.PreconditionTypePrecondV2,
V2: &xdr.PreconditionsV2{
TimeBounds: &xdr.TimeBounds{
MinTime: xdr.TimePoint(27),
MaxTime: xdr.TimePoint(42),
},
LedgerBounds: &xdr.LedgerBounds{
MinLedger: xdr.Uint32(27),
MaxLedger: xdr.Uint32(42),
},
MinSeqNum: &xdrSeqNum,
MinSeqAge: xdr.Duration(27),
MinSeqLedgerGap: xdr.Uint32(42),
ExtraSigners: Signers[:1],
},
}
tb = NewTimebounds(27, 42)
pc = Preconditions{
timebounds: &tb,
Ledgerbounds: &Ledgerbounds{27, 42},
MinSequenceNumber: &seqNum,
MinSequenceNumberAge: xdr.Duration(27),
MinSequenceNumberLedgerGap: 42,
ExtraSigners: Signers[:1],
}
Shaptic marked this conversation as resolved.
Show resolved Hide resolved
)

// TestClassifyingPreconditions ensures that Preconditions will correctly
// differentiate V1 (timebounds-only) or V2 (all other) preconditions correctly.
func TestClassifyingPreconditions(t *testing.T) {
tbpc := NewPreconditions(&tb)
assert.False(t, (&Preconditions{}).HasV2Conditions())
assert.False(t, tbpc.HasV2Conditions())
assert.True(t, pc.HasV2Conditions())
}

// TestPreconditions ensures correct XDR is generated for a (non-exhaustive)
// handful of precondition combinations.
func TestPreconditions(t *testing.T) {
preconditionModifiers := []struct {
Name string
Modifier func() (xdr.Preconditions, Preconditions)
}{
{
"unchanged",
func() (xdr.Preconditions, Preconditions) {
return xdrCond, pc
},
},
{
"only timebounds",
func() (xdr.Preconditions, Preconditions) {
tb := NewTimebounds(1, 2)
return xdr.Preconditions{
Type: xdr.PreconditionTypePrecondTime,
TimeBounds: &xdr.TimeBounds{
MinTime: xdr.TimePoint(1),
MaxTime: xdr.TimePoint(2),
},
}, NewPreconditions(&tb)
},
},
{
"unbounded ledgerbounds",
func() (xdr.Preconditions, Preconditions) {
newCond, newPc := clone(xdrCond, pc)
newCond.V2.LedgerBounds.MaxLedger = 0
newPc.Ledgerbounds.MaxLedger = 0
return newCond, newPc
},
},
{
"nil ledgerbounds",
func() (xdr.Preconditions, Preconditions) {
newCond, newPc := clone(xdrCond, pc)
newCond.V2.LedgerBounds = nil
newPc.Ledgerbounds = nil
return newCond, newPc
},
},
{
"nil minSeq",
func() (xdr.Preconditions, Preconditions) {
newCond, newPc := clone(xdrCond, pc)
newCond.V2.MinSeqNum = nil
newPc.MinSequenceNumber = nil
return newCond, newPc
},
},
}
for _, testCase := range preconditionModifiers {
t.Run(testCase.Name, func(t *testing.T) {
xdrCond, precond := testCase.Modifier()
Shaptic marked this conversation as resolved.
Show resolved Hide resolved
assert.NoError(t, precond.Validate())

expectedBytes, err := xdrCond.MarshalBinary()
assert.NoError(t, err)

actualBytes, err := precond.BuildXDR().MarshalBinary()
assert.NoError(t, err)
Shaptic marked this conversation as resolved.
Show resolved Hide resolved
assert.Equal(t, expectedBytes, actualBytes)
})
}
}

// TestPreconditionsValidation ensures that validation fails when necessary.
func TestPreconditionsValidation(t *testing.T) {
t.Run("too many signers", func(t *testing.T) {
pc.ExtraSigners = Signers
assert.Error(t, pc.Validate())
})
}

func clone(pcXdr xdr.Preconditions, pc Preconditions) (xdr.Preconditions, Preconditions) {
return cloneXdrPreconditions(pcXdr), clonePreconditions(pc)
}

func cloneXdrPreconditions(pc xdr.Preconditions) xdr.Preconditions {
binary, err := pc.MarshalBinary()
if err != nil {
panic(err)
}

clone := xdr.Preconditions{}
if err = clone.UnmarshalBinary(binary); err != nil {
panic(err)
}

return clone
}

func clonePreconditions(precond Preconditions) Preconditions {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 The way this set of tests work by cloning things is particularly complicated. How about just writing out each input verbosely? It would be much easier to understand and maintain.

The inspiration for why I'm suggesting this is that DRY tests are often harder to maintain:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously your perspective has a lot of weight, so I'm trying to maximize my own edification here by offering counterarguments. I feel like I should write my own blog post in response 😂 so a couple of thoughts follow.


How is it easier to understand? More than once I've opened a test file for a complex piece of code, and seen what appeared to be a lot of duplication across tests. It begged a really important question, though: are they actually the same? If you're reading page after page of code for related tests, you need to read each test really carefully to identify where it differs. If the "setup" is kept in one place, the tests can focus exclusively on their differences. That makes it easier to understand, not harder, imo.

Small case study: Take a look at the various operation tests in txnbuild/transaction_test.go. Different test groups have different authors, so irrelevant deltas get introduced. Why do the sequence numbers differ across the tests? Is it just an author change, or does it impact the test somehow? Some of them use NewTransaction, while others use newSignedTransaction. Does that make a difference, and why? Should I pay attention to each test's TransactionParams, or are they just copy-pasted across all test cases?

All of these questions are overhead to understanding the tests. If there was a single method that prepared the common parts of the test case, each one could focus on the differences between them. That's the goal of the "modifier" design I threw in: it's very clear (though I will admit I'm biased as the author and reader) that the only difference between each subtest is the fields set on the two preconditions.


Then there's the aspect of maintainability. How does that get easier? Within this PR itself, we had a breaking change re: timebounds that resulted in many lines changing. If there was a common place for the tests to prepare things (like for transaction_test.go as I mentioned above), there'd be a single delta of Timebounds: NewInfiniteTimeout() -> Preconditions: Preconditions{Timebounds: NewInfiniteTimeout()}, because that line is common to all of the tests. Now, as a reviewer I'd have to look at each line that changed and make sure it was updated correctly and no copy-paste or search&replace error snuck in.

Adding tests becomes a chore, too. Like I mentioned earlier (#4303 (comment)), there are a lot of ways to combine preconditions, now. We want all of them to be possible, and of course we can't exhaust them all (or can rely on bug reports for ones that somehow aren't possible), but it'd be nice to get the basics covered. Adding a new combination is a few LoC. Furthermore, extending the tests (like you suggested above to add a round-trip, #4303 (comment)) is so much easier when they're all testing common functionality. That extension took ~7 LoC in cf15638. If the tests were split out, that'd be N copy-pastes and 7*N lines of testing code that all do the same thing.


I guess I just... don't really get it? Yes, there's some degree of "cleverness" when reading the current code for the first time, but it comes at the (to me, huge) benefit of very clearly isolating the differences between each subtest case and allowing simple extensibility to add new combinations of preconditions.

As a final, disjoint thought: Something that makes even more sense than my current list of subtests is just "fuzzing" the fields on Precondition to ensure they all work. That's the ultimate goal here, anyway: answering the question of "Can all valid preconditions be created?"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a tradeoff for sure. Even if DRY tests look good on day one, imx they are harder to maintain over time, because the way they are structured are making assumptions about the tests we'll have to write in the future or the ways we'll need to modify these tests, requiring test refactors, etc. My comment was a suggestion, not a request.

If you are going to keep the cloning, might I suggest replacing it with a helper that creates the original structure. This way you don't need to write cloning code. (Cloning code is unavoidably brittle code.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point about test evolution. Cross that bridge when we get to it, then? 😆 Either way, just want to reiterate that I really appreciate the feedback and suggestions, as it makes me challenge my own assumptions and get a better understanding of the trade-offs I may be making w/o realizing.

You're on the money about cloning: I hated that aspect of it. I opted to make "test fixtures" with a helper function in 84c882e8c9bd52ef93954392619d2e686f503882; I think it's much less brittle now overall!

tb := NewTimebounds(precond.timebounds.MinTime, precond.timebounds.MaxTime)
cond := NewPreconditions(&tb)
if precond.Ledgerbounds != nil {
cond.Ledgerbounds = &Ledgerbounds{
MinLedger: precond.Ledgerbounds.MinLedger,
MaxLedger: precond.Ledgerbounds.MaxLedger,
}
}

if precond.MinSequenceNumber != nil {
cond.MinSequenceNumber = precond.MinSequenceNumber
}

cond.MinSequenceNumberAge = precond.MinSequenceNumberAge
cond.MinSequenceNumberLedgerGap = precond.MinSequenceNumberLedgerGap

if len(precond.ExtraSigners) > 0 {
cond.ExtraSigners = append(cond.ExtraSigners, precond.ExtraSigners...)
}

return cond
}
2 changes: 1 addition & 1 deletion txnbuild/timebounds.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ type Timebounds struct {
// using a factory method. This is done to ensure that default Timebound structs (which have no limits) are not
// valid - you must explicitly specifiy the Timebound you require.
func (tb *Timebounds) Validate() error {
if !tb.wasBuilt {
if tb == nil || !tb.wasBuilt {
return errors.New("timebounds must be constructed using NewTimebounds(), NewTimeout(), or NewInfiniteTimeout()")
}
if tb.MinTime < 0 {
Expand Down
Loading