Submit tx endpoint #1001

v0d1ch · 2023-07-27T20:22:01Z

Why

Our dear colleague @abailly-iohk had the idea to add possibility for the users to submit their transactions to the L1 network using their hydra-node. Our users also confirmed this would, indeed be a nice feature to have thus this PR.

CHANGELOG updated or not needed
Documentation updated or not needed
Haddocks updated or not needed
No new TODOs introduced or explained herafter

github-actions · 2023-07-27T20:38:19Z

Test Results

359 tests +7 354 ✔️ +7 20m 36s ⏱️ - 1m 10s
119 suites +2     5 💤 ±0
    6 files ±0     0 ❌ ±0

Results for commit 09760e6. ± Comparison against base commit f5a3e47.

This pull request removes 7 and adds 14 tests. Note that renamed tests count towards both.

Hydra.API.RestServer ‑ Validate /commit publish api schema
Hydra.API.RestServer ‑ Validate /commit subscribe api schema
Hydra.API.RestServer/API should respond correctly ‑ GET /protocol-parameters works
Hydra.API.RestServer/JSON encoding of (ReasonablySized DraftCommitTxRequest) ‑ allows to encode values with aeson and read them back
Hydra.API.RestServer/JSON encoding of (ReasonablySized DraftCommitTxRequest) ‑ produces the same JSON as is found in golden/ReasonablySized DraftCommitTxRequest.json
Hydra.API.RestServer/JSON encoding of (ReasonablySized DraftCommitTxResponse) ‑ allows to encode values with aeson and read them back
Hydra.API.RestServer/JSON encoding of (ReasonablySized DraftCommitTxResponse) ‑ produces the same JSON as is found in golden/ReasonablySized DraftCommitTxResponse.json

Hydra.API.HTTPServer ‑ Validate /cardano-transaction publish api schema
Hydra.API.HTTPServer ‑ Validate /cardano-transaction subscribe api schema
Hydra.API.HTTPServer ‑ Validate /commit publish api schema
Hydra.API.HTTPServer ‑ Validate /commit subscribe api schema
Hydra.API.HTTPServer/API should respond correctly ‑ GET /protocol-parameters works
Hydra.API.HTTPServer/JSON encoding of (ReasonablySized DraftCommitTxRequest) ‑ allows to encode values with aeson and read them back
Hydra.API.HTTPServer/JSON encoding of (ReasonablySized DraftCommitTxRequest) ‑ produces the same JSON as is found in golden/ReasonablySized DraftCommitTxRequest.json
Hydra.API.HTTPServer/JSON encoding of (ReasonablySized DraftCommitTxResponse) ‑ allows to encode values with aeson and read them back
Hydra.API.HTTPServer/JSON encoding of (ReasonablySized DraftCommitTxResponse) ‑ produces the same JSON as is found in golden/ReasonablySized DraftCommitTxResponse.json
Hydra.API.HTTPServer/JSON encoding of (ReasonablySized SubmitTxRequest) ‑ allows to encode values with aeson and read them back
…

♻️ This comment has been updated with latest results.

github-actions · 2023-07-27T20:40:05Z

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2023-08-01 14:36:13.171671287 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	7ceb53f05e444cfdabfd0a37a0590090066da457a1f1db30d613b8bd	4289
νCommit	70e70fc13217bfde96932956656c1d540a743b1588c845ca09dc3723	2124
νHead	cda51d313c1c8285b6925ce2413def012db27f544e2bbd79b8173000	9185
μHead	1c0b665fc49bc2e9e2ce4e8252c8f37fe84dd75bd8e086abfdb92685*	4149

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4743	14.53	5.68	0.52
2	4951	15.26	5.92	0.53
3	5152	17.34	6.69	0.57
5	5564	23.54	9.06	0.65
10	6585	33.97	12.96	0.81
37	12125	97.52	36.87	1.74

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	599	14.97	5.73	0.34
2	787	19.75	7.76	0.40
3	971	24.85	9.91	0.47
5	1344	36.19	14.60	0.61
10	2284	71.91	28.91	1.04
13	2850	98.33	39.25	1.35

Cost of CollectCom Transaction

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	815	27.79	10.78	0.49
2	114	1144	43.90	17.16	0.68
3	170	1454	62.04	24.40	0.89
4	225	1783	82.64	32.66	1.13

Cost of Close Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	580	16.86	7.02	0.36
2	804	20.38	9.71	0.42
3	969	21.48	10.85	0.44
5	1295	24.57	13.49	0.50
10	2124	31.27	19.66	0.64
50	8725	87.55	70.11	1.75

Cost of Contest Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	676	24.87	10.67	0.46
2	832	26.16	11.87	0.48
3	1005	27.88	13.24	0.52
5	1335	31.75	16.15	0.58
10	2161	40.34	23.01	0.74
44	7772	98.80	69.64	1.79

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4929	23.72	9.97	0.63
2	5175	37.05	15.68	0.79
3	5498	54.31	23.17	1.00
4	5820	74.35	31.87	1.24
5	5998	91.56	39.18	1.44

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4769	8.72	3.59	0.46
5	1	57	4801	10.12	4.41	0.48
5	5	285	4942	15.71	7.72	0.55
5	10	569	5120	22.69	11.85	0.64
5	20	1137	5487	36.67	20.11	0.83
5	30	1708	5846	50.65	28.38	1.02
5	40	2279	6202	64.63	36.65	1.21
5	50	2846	6569	78.62	44.92	1.40
5	65	3700	7102	99.62	57.34	1.68

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2023-08-01 14:31:03.222388326 UTC

3-nodes Scenario

A rather typical setup, with 3 nodes forming a Hydra head.

Number of nodes	3
Number of txs	900
Avg. Confirmation Time (ms)	112.364622456
P99	275.23253210999997ms
P95	234.33567749999997ms
P50	106.518608ms
Number of Invalid txs	0

Baseline Scenario

This scenario represents a minimal case and as such is a good baseline against which to assess the overhead introduced by more complex setups. There is a single hydra-node d with a single client submitting single input and single output transactions with a constant UTxO set of 1.

Number of nodes	1
Number of txs	300
Avg. Confirmation Time (ms)	5.062421136
P99	18.13612402ms
P95	12.606414100000002ms
P50	3.677223ms
Number of Invalid txs	0

hydra-cluster/src/Hydra/Cluster/Scenarios.hs

hydra-node/src/Hydra/API/RestServer.hs

hydra-node/src/Hydra/Chain/CardanoClient.hs

ch1bo

Some refactoring could be avoided and I think we should change the response serialization + API endpoint to something more REST-ful.

hydra-node/src/Hydra/API/RestServer.hs

hydra-node/src/Hydra/Chain/Direct/Handlers.hs

hydra-node/src/Hydra/API/RestServer.hs

hydra-cluster/src/Hydra/Cluster/Scenarios.hs

hydra-node/json-schemas/api.yaml

hydra-node/src/Hydra/API/APIServerLog.hs

ch1bo

Not many changes needed anymore, although I'm puzzled by some of the renamings (and it makes reviewing this PR hard)

hydra-cluster/src/Hydra/Cluster/Scenarios.hs

hydra-cluster/src/Hydra/LogFilter.hs

hydra-cluster/test/Test/EndToEndSpec.hs

hydra-node/exe/hydra-node/Main.hs

hydra-node/golden/ReasonablySized SubmittedTxResponse.json

hydra-cluster/src/Hydra/Cluster/Scenarios.hs

ch1bo · 2023-08-01T11:58:03Z

Only my comments in e2e tests remaining from my side

Co-authored-by: Sebastian Nagel <[email protected]>

- And fix the api schema

- Also rename the corresponding test module RestServerSpec -> HTTPServerSpec

This also merges withAPIServer and runAPIServer together as most complexity comes from arguments being passed in.

This reverts commit 870d22c.

… docs

…TTPRequestReceived. Also changed the definition to use dedicated types and simplify construction in http server.

All branches of the handler were using respond, so we can factor this out and make the scope of handler smaller.

This forced us to write instances for ByteString and Text, which is actually fine as we could give a more precise definition what bytes/text we expect here.

We can interpret the new endpoint as a basic submission endpoint and indeed it is the same implementation as the Chain module uses itself which gets exposed.

AsyncAPI allows to specify multiple message kinds on a channel, which we can use to enumerate the possible responses in HTTP endpoints.

v0d1ch self-assigned this Jul 27, 2023

v0d1ch requested review from ch1bo and ffakenz and removed request for ch1bo July 27, 2023 20:30

ffakenz approved these changes Jul 28, 2023

View reviewed changes

hydra-cluster/src/Hydra/Cluster/Scenarios.hs Show resolved Hide resolved

hydra-node/src/Hydra/API/RestServer.hs Outdated Show resolved Hide resolved

hydra-node/src/Hydra/Chain/CardanoClient.hs Outdated Show resolved Hide resolved

ch1bo requested changes Jul 28, 2023

View reviewed changes

ch1bo linked an issue Jul 28, 2023 that may be closed by this pull request

Submit L1 transactions through hydra-node #966

Closed

v0d1ch force-pushed the submit-tx-endpoint branch 3 times, most recently from 04327da to d373e3d Compare July 31, 2023 09:04

ffakenz approved these changes Jul 31, 2023

View reviewed changes

hydra-node/src/Hydra/API/APIServerLog.hs Outdated Show resolved Hide resolved

ch1bo force-pushed the submit-tx-endpoint branch from d373e3d to 187f33c Compare July 31, 2023 15:11

ch1bo requested changes Aug 1, 2023

View reviewed changes

ch1bo force-pushed the submit-tx-endpoint branch 11 times, most recently from 5a0a7b7 to 3243006 Compare August 1, 2023 11:53

v0d1ch added 2 commits August 1, 2023 15:58

Draft new types needed for the rest endpoint

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.

GPG key ID: 4AEE18F83AFDEB23
Expired

Verified
Learn about vigilant mode

4131de5

Introduce a failing test

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.

GPG key ID: 4AEE18F83AFDEB23
Expired

Verified
Learn about vigilant mode

218ffc8

v0d1ch and others added 25 commits August 1, 2023 15:58

Move APIServerLog to RestServer module

40ddb00

Move api handlers and server code from Server to RestServer

943b1e9

Resolve now expired todo

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.

GPG key ID: 4AEE18F83AFDEB23
Expired

Verified
Learn about vigilant mode

9174855

Changelog entry

26ff657

Update hydra-node/src/Hydra/API/RestServer.hs

d244eda

Co-authored-by: Sebastian Nagel <[email protected]>

Update hydra-node/src/Hydra/Chain/Direct/Handlers.hs

8a8de33

Co-authored-by: Sebastian Nagel <[email protected]>

Rename SubmitTxResponse to TransactionSubmitted

3e07f82

- And fix the api schema

Reuse return400 function

c4df3f1

Clarify what errors submitUserTx should return

7b05c62

Inline messages in the api schema

be8577f

Restore CardanoClient

910fd85

Rename path to "cardano-transaction"

bfee435

Rename RestServer to HTTPServer

2d366d5

- Also rename the corresponding test module RestServerSpec -> HTTPServerSpec

Extract the websocket server out of API.Server

74666b3

This also merges withAPIServer and runAPIServer together as most complexity comes from arguments being passed in.

Move api logging type to APIServerLog module

2be397a

Revert "Inline messages in the api schema"

b3385dd

This reverts commit 870d22c.

Alter test to use makeSignedTransaction

bc65f78

Fix behavior broken links by using pathname:/// for generated haddock…

de3c638

… docs

Remove APIHandshakeError

0022359

Change APIRestInputReceived to not include body and rename it to APIH…

31eaac7

…TTPRequestReceived. Also changed the definition to use dedicated types and simplify construction in http server.

Simplify HTTP handlers by not passing continuation

34d88db

All branches of the handler were using respond, so we can factor this out and make the scope of handler smaller.

Remove imports with orphan instances from APIServerlog

01e33d6

This forced us to write instances for ByteString and Text, which is actually fine as we could give a more precise definition what bytes/text we expect here.

Rename submitUserTx -> submitTx and simplify api.yaml

42412fb

We can interpret the new endpoint as a basic submission endpoint and indeed it is the same implementation as the Chain module uses itself which gets exposed.

Document possible response messages in api.ayml

87c1169

AsyncAPI allows to specify multiple message kinds on a channel, which we can use to enumerate the possible responses in HTTP endpoints.

PR Review changes in Scenarios module

8fe026b

v0d1ch force-pushed the submit-tx-endpoint branch from 2081112 to 8fe026b Compare August 1, 2023 13:59

Remove leftover golden file and fix compilation

09760e6

v0d1ch force-pushed the submit-tx-endpoint branch from 372de80 to 09760e6 Compare August 1, 2023 14:13

v0d1ch merged commit 7cf29ce into master Aug 1, 2023

v0d1ch deleted the submit-tx-endpoint branch August 1, 2023 14:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Submit tx endpoint #1001

Submit tx endpoint #1001

v0d1ch commented Jul 27, 2023

github-actions bot commented Jul 27, 2023 •

edited

Loading

github-actions bot commented Jul 27, 2023 •

edited

Loading

ch1bo left a comment

ch1bo left a comment

ch1bo commented Aug 1, 2023

Submit tx endpoint #1001

Submit tx endpoint #1001

Conversation

v0d1ch commented Jul 27, 2023

Why

github-actions bot commented Jul 27, 2023 • edited Loading

Test Results

github-actions bot commented Jul 27, 2023 • edited Loading

Transactions Costs

Script summary

Cost of Init Transaction

Cost of Commit Transaction

Cost of CollectCom Transaction

Cost of Close Transaction

Cost of Contest Transaction

Cost of Abort Transaction

Cost of FanOut Transaction

End-To-End Benchmark Results

3-nodes Scenario

Baseline Scenario

ch1bo left a comment

Choose a reason for hiding this comment

ch1bo left a comment

Choose a reason for hiding this comment

ch1bo commented Aug 1, 2023

github-actions bot commented Jul 27, 2023 •

edited

Loading

github-actions bot commented Jul 27, 2023 •

edited

Loading