Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

txscript: Zero alloc optimization refactor. #1656

Merged
merged 122 commits into from
Mar 26, 2019
Merged
Changes from 1 commit
Commits
Show all changes
122 commits
Select commit Hold shift + click to select a range
60d41d1
txscript: Deprecate HasP2SHScriptSigStakeOpCodes.
davecgh Mar 13, 2019
b1b64d9
txscript: Deprecate IsStakeOutput.
davecgh Mar 13, 2019
7f1da3b
txscript: Deprecate GetMultisigMandN.
davecgh Mar 13, 2019
e6a5701
txscript: Move init func in benchmarks to top.
davecgh Mar 13, 2019
2f8f078
txscript: Add benchmark for script parsing.
davecgh Mar 13, 2019
cb86bc0
txscript: Introduce zero-alloc script tokenizer.
davecgh Mar 13, 2019
9b2ec27
txscript: Add benchmark for DisasmString.
davecgh Mar 13, 2019
e332430
txscript: Optimize script disasm.
davecgh Mar 13, 2019
f306a72
txscript: Introduce raw script sighash calc func.
davecgh Mar 13, 2019
c57dc2d
txscript: Optimize CalcSignatureHash.
davecgh Mar 13, 2019
06f769e
txscript: Convert sighash calc tests.
davecgh Mar 13, 2019
44cbc31
txscript: Make isSmallInt accept raw opcode.
davecgh Mar 13, 2019
0ed8e25
txscript: Make asSmallInt accept raw opcode.
davecgh Mar 13, 2019
082d1ed
txscript: Make isStakeOpcode accept raw opcode.
davecgh Mar 13, 2019
c705a0e
txscript: Add benchmark for IsPayToScriptHash.
davecgh Mar 13, 2019
9f2f038
txscript: Optimize IsPayToScriptHash.
davecgh Mar 13, 2019
356492b
txscript: Add benchmarks for IsMutlsigScript.
davecgh Mar 13, 2019
7b8259b
txscript: Optimize IsMultisigScript.
davecgh Mar 13, 2019
d7492c3
txscript: Add benchmarks for IsMutlsigSigScript.
davecgh Mar 13, 2019
462eea3
txscript: Optimize IsMultisigSigScript.
davecgh Mar 13, 2019
2d70450
txscript: Add benchmark for GetSigOpCount.
davecgh Mar 13, 2019
ffa6fb9
txscript: Optimize GetSigOpCount.
davecgh Mar 13, 2019
51f7639
txscript: Add tests for stake-tagged script hash.
davecgh Mar 13, 2019
bc56df1
txscript: Add benchmark for isAnyKindOfScriptHash.
davecgh Mar 13, 2019
a598838
txscript: Optimize isAnyKindOfScriptHash.
davecgh Mar 13, 2019
93b039d
txscript: Add benchmark for IsPushOnlyScript.
davecgh Mar 13, 2019
9e8bfd2
txscript: Optimize IsPushOnlyScript.
davecgh Mar 13, 2019
af67951
txscript: Optimize new engine push only script.
davecgh Mar 13, 2019
281f794
txscript: Check p2sh push before parsing scripts.
davecgh Mar 13, 2019
75e71d8
txscript: Add benchmark for GetPreciseSigOpCount.
davecgh Mar 13, 2019
a1b24ad
txscript: Optimize GetPreciseSigOpCount.
davecgh Mar 13, 2019
4dc1ffb
txscript: Add benchmark for GetScriptClass.
davecgh Mar 13, 2019
4f27dfc
txscript: Make typeOfScript accept raw script.
davecgh Mar 13, 2019
bdd51a0
txscript: Optimize typeOfScript pay-to-script-hash.
davecgh Mar 13, 2019
efd7e2f
txscript: Remove unused isScriptHash function.
davecgh Mar 13, 2019
900af9b
txscript: Optimize typeOfScript multisig.
davecgh Mar 13, 2019
07168b8
txscript: Remove unused isMultiSig function.
davecgh Mar 13, 2019
c4a1527
txscript: Add benchmark for pay-to-pubkey scripts.
davecgh Mar 13, 2019
1b067cb
txscript: Optimize typeOfScript pay-to-pubkey.
davecgh Mar 13, 2019
b273b83
txscript: Remove unused isPubkey function.
davecgh Mar 13, 2019
e64e21c
txscript: Add bench for pay-to-alt-pubkey scripts.
davecgh Mar 13, 2019
2602647
txscript: Optimize typeOfScript pay-to-alt-pubkey.
davecgh Mar 13, 2019
d07d626
txscript: Add bench for pay-to-pubkey-hash scripts.
davecgh Mar 13, 2019
406f851
txscript: Optimize typeOfScript pay-to-pubkey-hash.
davecgh Mar 13, 2019
ffe80c7
txscript: Remove unused isPubkeyHash function.
davecgh Mar 13, 2019
95175cb
txscript: Add bench for pay-to-alt-pubkey-hash scripts.
davecgh Mar 13, 2019
e7a172c
txscript: Optimize typeOfScript pay-to-alt-pk-hash.
davecgh Mar 13, 2019
15416b0
txscript: Add bench for null scripts.
davecgh Mar 13, 2019
f589dd8
txscript: Optimize typeOfScript nulldata detection.
davecgh Mar 13, 2019
63e7418
txscript: Remove unused isNullData function.
davecgh Mar 13, 2019
6de8d69
txscript: Add bench for stake submission scripts.
davecgh Mar 13, 2019
5455dbc
txscript: Optimize typeOfScript stakesub detection.
davecgh Mar 13, 2019
974ae66
txscript: Remove unused isStakeSubmission function.
davecgh Mar 13, 2019
c07f9cb
txscript: Add bench for stake generation scripts.
davecgh Mar 13, 2019
5a24f50
txscript: Optimize typeOfScript stakegen detection.
davecgh Mar 13, 2019
8468b0d
txscript: Remove unused isStakeGen function.
davecgh Mar 13, 2019
9b3f4c9
txscript: Add bench for stake revocation scripts.
davecgh Mar 13, 2019
8dfae89
txscript: Optimize typeOfScript stakerev detection.
davecgh Mar 13, 2019
61872f9
txscript: Remove unused isStakeRevocation function.
davecgh Mar 13, 2019
123a733
txscript: Add bench for stake change scripts.
davecgh Mar 13, 2019
89d4941
txscript: Optimize typeOfScript stakechange detect.
davecgh Mar 13, 2019
28765fa
txscript: Remove unused isSStxChange function.
davecgh Mar 13, 2019
bfab5db
txscript: Add benchmark for ContainsStakeOpCodes.
davecgh Mar 13, 2019
0e021a9
txscript: Optimize ContainsStakeOpCodes.
davecgh Mar 13, 2019
ccd0a92
txscript: Add benchmark for ExtractCoinbaseNullData.
davecgh Mar 13, 2019
6a0a77f
txscript: Optimize ExtractCoinbaseNullData.
davecgh Mar 13, 2019
8f44276
txscript: Convert CalcScriptInfo.
davecgh Mar 13, 2019
7d8ce2d
txscript: Remove unused isPushOnly function.
davecgh Mar 13, 2019
5a4d9c9
txscript: Remove unused getSigOpCount function.
davecgh Mar 13, 2019
04e70a1
txscript: Add CalcMultiSigStats benchmark.
davecgh Mar 13, 2019
c596826
txscript: Optimize CalcMultiSigStats.
davecgh Mar 13, 2019
b0f5561
txscript: Add multisig redeem script extract bench.
davecgh Mar 13, 2019
1466a2a
txscript: Optimize multi sig redeem script func.
davecgh Mar 13, 2019
d82dfb7
txscript: Convert GetScriptHashFromP2SHScript.
davecgh Mar 13, 2019
9b74ada
txscript: Add benchmark for PushedData.
davecgh Mar 13, 2019
9e7de33
txscript: Optimize PushedData.
davecgh Mar 13, 2019
a1da017
txscript: Add benchmark for IsUnspendable.
davecgh Mar 13, 2019
bb365f2
txscript: Optimize IsUnspendable.
davecgh Mar 13, 2019
67d7385
txscript: Make canonicalPush accept raw opcode.
davecgh Mar 13, 2019
ceb1f72
txscript: Add tests for atomic swap extraction.
davecgh Mar 13, 2019
605a9a4
txscript: Add ExtractAtomicSwapDataPushes benches.
davecgh Mar 13, 2019
4774fda
txscript: Optimize ExtractAtomicSwapDataPushes.
davecgh Mar 13, 2019
5e90e59
txscript: Add ExtractPkScriptAddrs benchmarks.
davecgh Mar 13, 2019
49b3f9f
txscript: Optimize ExtractPkScriptAddrs scripthash.
davecgh Mar 13, 2019
bdd98b1
txscript: Optimize ExtractPkScriptAddrs pubkeyhash.
davecgh Mar 13, 2019
bbca815
txscript: Optimize ExtractPkScriptAddrs altpubkeyhash.
davecgh Mar 13, 2019
9e5744d
txscript: Optimize ExtractPkScriptAddrs pubkey.
davecgh Mar 13, 2019
8280e50
txscript: Optimize ExtractPkScriptAddrs altpubkey.
davecgh Mar 13, 2019
6c1c2d1
txscript: Optimize ExtractPkScriptAddrs multisig.
davecgh Mar 13, 2019
57219fc
txscript: Optimize ExtractPkScriptAddrs stakesub.
davecgh Mar 13, 2019
8c763f1
txscript: Optimize ExtractPkScriptAddrs stakegen.
davecgh Mar 13, 2019
a754af9
txscript: Optimize ExtractPkScriptAddrs stakerev.
davecgh Mar 13, 2019
c9bce2a
txscript: Optimize ExtractPkScriptAddrs stakechange.
davecgh Mar 13, 2019
d1944a9
txscript: Optimize ExtractPkScriptAddrs nulldata.
davecgh Mar 13, 2019
aaebc79
txscript: Add ExtractPkScriptAltSigType benchmark.
davecgh Mar 13, 2019
2fd1dcb
txscript: Optimize ExtractPkScriptAltSigType.
davecgh Mar 13, 2019
007264e
txscript: Remove unused extractOneBytePush func.
davecgh Mar 13, 2019
b19b6cb
txscript: Remove unused isPubkeyAlt function.
davecgh Mar 13, 2019
9e3269e
txscript: Remove unused isPubkeyHashAlt function.
davecgh Mar 13, 2019
a247a52
txscript: Remove unused isOneByteMaxDataPush func.
davecgh Mar 13, 2019
251c9be
txscript: mergeMultiSig function def order cleanup.
davecgh Mar 13, 2019
7cf42b0
txscript: Use raw scripts in RawTxInSignature.
davecgh Mar 13, 2019
706b3a1
txscript: Use raw scripts in RawTxInSignatureAlt.
davecgh Mar 13, 2019
d8d561d
txscript: Correct p2pkSignatureScriptAlt comment.
davecgh Mar 13, 2019
85c7a9b
txscript: Use raw scripts in SignTxOutput.
davecgh Mar 13, 2019
f93c1de
txscript: Implement efficient opcode data removal.
davecgh Mar 13, 2019
d3518fc
txscript: Make isDisabled accept raw opcode.
davecgh Mar 13, 2019
b626552
txscript: Make alwaysIllegal accept raw opcode.
davecgh Mar 13, 2019
cfd3753
txscript: Make isConditional accept raw opcode.
davecgh Mar 13, 2019
e915598
txscript: Make min push accept raw opcode and data.
davecgh Mar 13, 2019
280c062
txscript: Convert to use non-parsed opcode disasm.
davecgh Mar 13, 2019
75c48ea
txscript: Refactor engine to use raw scripts.
davecgh Mar 13, 2019
bd040ae
txscript: Remove unused removeOpcodeByData func.
davecgh Mar 13, 2019
c598f59
txscript: Rename removeOpcodeByDataRaw func.
davecgh Mar 13, 2019
2e082a8
txscript: Remove unused calcSignatureHash func.
davecgh Mar 13, 2019
d72f6d3
txscript: Rename calcSignatureHashRaw func.
davecgh Mar 13, 2019
cef8086
txscript: Remove unused parseScript func.
davecgh Mar 13, 2019
861032a
txscript: Remove unused unparseScript func.
davecgh Mar 13, 2019
ceb58f4
txscript: Remove unused parsedOpcode.bytes func.
davecgh Mar 13, 2019
fcb1f3a
txscript: Remove unused parseScriptTemplate func.
davecgh Mar 13, 2019
5059be9
txscript: Make executeOpcode take opcode and data.
davecgh Mar 13, 2019
6adbaa6
txscript: Make op callbacks take opcode and data.
davecgh Mar 13, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
txscript: Use raw scripts in SignTxOutput.
This converts SignTxOutput and supporting funcs, namely sign,
mergeScripts and mergeMultiSig, to make use of the new tokenizer as well
as some recently added funcs that deal with raw scripts in order to
remove the reliance on parsed opcodes as a step towards utlimately
removing them altogether and updates the comments to explicitly call out
the script version semantics.

It is worth noting that this has the side effect of optimizing the
function as well, however, since this change is not focused on the
optimization aspects, no benchmarks are provided.
davecgh committed Mar 26, 2019
commit 85c7a9ba8eb4e7a305e0892aad404e173297a424
90 changes: 53 additions & 37 deletions txscript/sign.go
Original file line number Diff line number Diff line change
@@ -357,37 +357,44 @@ func sign(chainParams *chaincfg.Params, tx *wire.MsgTx, idx int,
// pkScript. Since this function is internal only we assume that the arguments
// have come from other functions internally and thus are all consistent with
// each other, behaviour is undefined if this contract is broken.
//
// NOTE: This function is only valid for version 0 scripts. Since the function
// does not accept a script version, the results are undefined for other script
// versions.
func mergeMultiSig(tx *wire.MsgTx, idx int, addresses []dcrutil.Address,
nRequired int, pkScript, sigScript, prevScript []byte) []byte {

// This is an internal only function and we already parsed this script
// as ok for multisig (this is how we got here), so if this fails then
// all assumptions are broken and who knows which way is up?
pkPops, _ := parseScript(pkScript)

sigPops, err := parseScript(sigScript)
if err != nil || len(sigPops) == 0 {
// Nothing to merge if either the new or previous signature scripts are
// empty.
if len(sigScript) == 0 {
return prevScript
}

prevPops, err := parseScript(prevScript)
if err != nil || len(prevPops) == 0 {
if len(prevScript) == 0 {
return sigScript
}

// Convenience function to avoid duplication.
extractSigs := func(pops []parsedOpcode, sigs [][]byte) [][]byte {
for _, pop := range pops {
if len(pop.data) != 0 {
sigs = append(sigs, pop.data)
var possibleSigs [][]byte
extractSigs := func(script []byte) error {
const scriptVersion = 0
tokenizer := MakeScriptTokenizer(scriptVersion, script)
for tokenizer.Next() {
if data := tokenizer.Data(); len(data) != 0 {
possibleSigs = append(possibleSigs, data)
}
}
return sigs
return tokenizer.Err()
}

possibleSigs := make([][]byte, 0, len(sigPops)+len(prevPops))
possibleSigs = extractSigs(sigPops, possibleSigs)
possibleSigs = extractSigs(prevPops, possibleSigs)
// Attempt to extract signatures from the two scripts. Return the other
// script that is intended to be merged in the case signature extraction
// fails for some reason.
if err := extractSigs(sigScript); err != nil {
return prevScript
}
if err := extractSigs(prevScript); err != nil {
return sigScript
}

// Now we need to match the signatures to pubkeys, the only real way to
// do that is to try to verify them all and match it to the pubkey
@@ -417,7 +424,7 @@ sigLoop:
// however, assume no sigs etc are in the script since that
// would make the transaction nonstandard and thus not
// MultiSigTy, so we just need to hash the full thing.
hash, err := calcSignatureHash(pkPops, hashType, tx, idx, nil)
hash, err := calcSignatureHashRaw(pkScript, hashType, tx, idx, nil)
if err != nil {
// Decred -- is this the right handling for SIGHASH_SINGLE error ?
// TODO make sure this doesn't break anything.
@@ -447,9 +454,7 @@ sigLoop:
}
}

// Extra opcode to handle the extra arg consumed (due to previous bugs
// in the reference implementation).
builder := NewScriptBuilder() //.AddOp(OP_FALSE)
builder := NewScriptBuilder()
doneSigs := 0
// This assumes that addresses are in the same order as in the script.
for _, addr := range addresses {
@@ -479,6 +484,10 @@ sigLoop:
// The return value is the best effort merging of the two scripts. Calling this
// function with addresses, class and nrequired that do not match pkScript is
// an error and results in undefined behaviour.
//
// NOTE: This function is only valid for version 0 scripts. Since the function
// does not accept a script version, the results are undefined for other script
// versions.
func mergeScripts(chainParams *chaincfg.Params, tx *wire.MsgTx, idx int,
pkScript []byte, class ScriptClass, addresses []dcrutil.Address,
nRequired int, sigScript, prevScript []byte) []byte {
@@ -487,32 +496,34 @@ func mergeScripts(chainParams *chaincfg.Params, tx *wire.MsgTx, idx int,
// inefficient in that they will recompute already known data.
// some internal refactoring could probably make this avoid needless
// extra calculations.
const scriptVersion = 0
switch class {
case ScriptHashTy:
// Remove the last push in the script and then recurse.
// this could be a lot less inefficient.
sigPops, err := parseScript(sigScript)
if err != nil || len(sigPops) == 0 {
// Nothing to merge if either the new or previous signature
// scripts are empty or fail to parse.
if len(sigScript) == 0 ||
checkScriptParses(scriptVersion, sigScript) != nil {

return prevScript
}
prevPops, err := parseScript(prevScript)
if err != nil || len(prevPops) == 0 {
if len(prevScript) == 0 ||
checkScriptParses(scriptVersion, prevScript) != nil {

return sigScript
}

// assume that script in sigPops is the correct one, we just
// made it.
script := sigPops[len(sigPops)-1].data
// Remove the last push in the script and then recurse.
// this could be a lot less inefficient.
//
// Assume that final script is the correct one since it was just
// made and it is a pay-to-script-hash.
script := finalOpcodeData(scriptVersion, sigScript)

// We already know this information somewhere up the stack,
// therefore the error is ignored.
class, addresses, nrequired, _ :=
ExtractPkScriptAddrs(DefaultScriptVersion, script, chainParams)

// regenerate scripts.
sigScript, _ := unparseScript(sigPops)
prevScript, _ := unparseScript(prevPops)

// Merge
mergedScript := mergeScripts(chainParams, tx, idx, script,
class, addresses, nrequired, sigScript, prevScript)
@@ -523,6 +534,7 @@ func mergeScripts(chainParams *chaincfg.Params, tx *wire.MsgTx, idx int,
builder.AddData(script)
finalScript, _ := builder.Script()
return finalScript

case MultiSigTy:
return mergeMultiSig(tx, idx, addresses, nRequired, pkScript,
sigScript, prevScript)
@@ -551,8 +563,7 @@ type KeyDB interface {
type KeyClosure func(dcrutil.Address) (chainec.PrivateKey, bool, error)

// GetKey implements KeyDB by returning the result of calling the closure.
func (kc KeyClosure) GetKey(address dcrutil.Address) (chainec.PrivateKey,
bool, error) {
func (kc KeyClosure) GetKey(address dcrutil.Address) (chainec.PrivateKey, bool, error) {
return kc(address)
}

@@ -577,9 +588,14 @@ func (sc ScriptClosure) GetScript(address dcrutil.Address) ([]byte, error) {
// getScript. If previousScript is provided then the results in previousScript
// will be merged in a type-dependent manner with the newly generated.
// signature script.
//
// NOTE: This function is only valid for version 0 scripts. Since the function
// does not accept a script version, the results are undefined for other script
// versions.
func SignTxOutput(chainParams *chaincfg.Params, tx *wire.MsgTx, idx int,
pkScript []byte, hashType SigHashType, kdb KeyDB, sdb ScriptDB,
previousScript []byte, sigType dcrec.SignatureType) ([]byte, error) {

sigScript, class, addresses, nrequired, err := sign(chainParams, tx,
idx, pkScript, hashType, kdb, sdb, sigType)
if err != nil {