Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TestHarness version roll back ambiguity #48

Closed
spoonincode opened this issue Apr 18, 2024 · 2 comments · Fixed by #952
Closed

TestHarness version roll back ambiguity #48

spoonincode opened this issue Apr 18, 2024 · 2 comments · Fixed by #952
Assignees

Comments

@spoonincode
Copy link
Member

TestHarness configures itself differently based on what nodeos --full-version returns. Its behavior is altered if the version begins with v2 or v3.1 with the assumption if it's not one of those versions it must be the "latest". So for spring v1 this code still does what is expected ("latest"), but spring v2 will not follow the expected path.

def configureVersion(self):
if 'v2' in self.nodeosVers:
self.fetchTransactionCommand = lambda: "get transaction"
self.fetchTransactionFromTrace = lambda trx: trx['trx']['id']
self.fetchBlock = lambda blockNum: self.processUrllibRequest("chain", "get_block", {"block_num_or_id":blockNum}, silentErrors=False, exitOnError=True)
self.fetchKeyCommand = lambda: "[trx][trx][ref_block_num]"
self.fetchRefBlock = lambda trans: trans["trx"]["trx"]["ref_block_num"]
self.fetchHeadBlock = lambda node, headBlock: node.processUrllibRequest("chain", "get_block", {"block_num_or_id":headBlock}, silentErrors=False, exitOnError=True)
self.cleosLimit = ""
else:
self.fetchTransactionCommand = lambda: "get transaction_trace"
self.fetchTransactionFromTrace = lambda trx: trx['id']
self.fetchBlock = lambda blockNum: self.processUrllibRequest("trace_api", "get_block", {"block_num":blockNum}, silentErrors=False, exitOnError=True)
self.fetchKeyCommand = lambda: "[transaction][transaction_header][ref_block_num]"
self.fetchRefBlock = lambda trans: trans["block_num"]
self.fetchHeadBlock = lambda node, headBlock: node.processUrllibRequest("chain", "get_block_info", {"block_num":headBlock}, silentErrors=False, exitOnError=True)
if 'v3.1' in self.nodeosVers:
self.cleosLimit = ""
else:
self.cleosLimit = "--time-limit 999"

@bhazzard
Copy link

For now let's remove this code handling of versions.

We only care about performance relative to the previous major version. One way to do that is to run the performance harness independently on each major version. If the new performance harness has changes to tests / metrics that aren't backported, that approach won't work. In that case we may need to reintroduce only for the previous supported version.

@bhazzard bhazzard added this to the Spring v1.1.0-rc1 milestone Oct 17, 2024
@bhazzard bhazzard added 👍 lgtm and removed triage labels Oct 17, 2024
@bhazzard
Copy link

Similar issue: #49

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants