Publish end-to-end bench result in PR comment

[pgrange]

Add the end-to-end benchmarks results to the PR as a comment, together with the transaction cost benchmarks. That way, we can more easily compare one PR with master or another PR.

---

• CHANGELOG updated or not needed
• Documentation updated or not needed
• Haddocks updated or not needed
• No new TODOs introduced or explained herafter

[github-actions]

Test Results

347 tests  ±0   342 ✔️ ±0   20m 28s ⏱️ + 1m 29s
115 suites ±0       5 💤 ±0 
    6 files   ±0       0 ❌ ±0 

Results for commit debbe93. ± Comparison against base commit 730b8e8.

♻️ This comment has been updated with latest results.

[github-actions]

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2023-07-28 09:54:44.12187084 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	7ceb53f05e444cfdabfd0a37a0590090066da457a1f1db30d613b8bd	4289
νCommit	70e70fc13217bfde96932956656c1d540a743b1588c845ca09dc3723	2124
νHead	cda51d313c1c8285b6925ce2413def012db27f544e2bbd79b8173000	9185
μHead	1c0b665fc49bc2e9e2ce4e8252c8f37fe84dd75bd8e086abfdb92685*	4149

• The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4743	15.38	6.02	0.53
2	4945	18.25	7.12	0.57
3	5152	17.19	6.64	0.56
5	5566	21.47	8.24	0.63
10	6588	36.66	14.03	0.84
36	11920	96.49	36.50	1.72

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	599	14.97	5.73	0.34
2	787	19.75	7.76	0.40
3	972	24.85	9.91	0.47
5	1345	36.19	14.60	0.61
10	2287	71.91	28.91	1.04
13	2849	98.33	39.25	1.35

Cost of CollectCom Transaction

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	822	28.13	10.92	0.49
2	114	1136	43.60	17.04	0.68
3	171	1456	61.88	24.34	0.89
4	226	1774	83.47	32.97	1.14

Cost of Close Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	639	18.98	8.46	0.39
2	667	17.60	7.38	0.38
3	969	21.48	10.85	0.44
5	1300	24.27	13.37	0.50
10	2121	31.27	19.66	0.64
50	8725	87.55	70.11	1.75

Cost of Contest Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	709	25.44	11.06	0.47
2	834	26.16	11.87	0.48
3	999	28.31	13.41	0.52
5	1335	31.75	16.15	0.58
10	2161	40.34	23.01	0.74
44	7771	98.80	69.64	1.79

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4856	22.64	9.46	0.62
2	5176	36.97	15.65	0.79
3	5496	54.06	23.07	1.00
4	5676	69.42	29.54	1.17
5	6138	97.28	41.86	1.51

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4770	8.72	3.59	0.46
5	1	56	4804	10.12	4.41	0.48
5	5	285	4946	15.71	7.72	0.55
5	10	570	5126	22.69	11.85	0.64
5	20	1139	5485	36.67	20.11	0.83
5	30	1709	5847	50.65	28.38	1.02
5	40	2276	6208	64.63	36.65	1.21
5	50	2847	6566	78.62	44.92	1.40
5	65	3702	7101	99.62	57.34	1.68

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2023-07-28 09:47:02.017044211 UTC

3-nodes Scenario

A rather typical setup, with 3 nodes forming a Hydra head.

Number of nodes	3
Number of txs	900
Avg. Confirmation Time (ms)	138.080749815
P99	278.6169335899999ms
P95	216.78262364999995ms
P50	128.7036915ms
Number of Invalid txs	0

Baseline Scenario

This scenario represents a minimal case and as such is a good baseline against which to assess the overhead introduced by more complex setups. There is a single hydra-node d with a single client submitting single input and single output transactions with a constant UTxO set of 1.

Number of nodes	1
Number of txs	300
Avg. Confirmation Time (ms)	7.392193940
P99	21.926589999999997ms
P95	14.09596205ms
P50	5.618713ms
Number of Invalid txs	0

[ch1bo]

I would prefer having two separate comments on the PR. One for the on-chain metrics, one for the off-chain metrics?