Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(dapi)!: replace getMnListDiff with a streaming endpoint #1859

Merged
merged 68 commits into from
Jun 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
eacb5fa
perf(dapi): replace getMnListDiff with steaming endpoint
shumkov May 26, 2024
b803d32
docs: add a todo
shumkov May 26, 2024
3de8980
chore: remove getMnListDiff
shumkov May 26, 2024
ebb1ac7
style: fix linter
shumkov May 26, 2024
a75d40e
style: fix lint errors
shumkov May 26, 2024
6f7bba0
test: adjust coverage threshold
shumkov May 26, 2024
6b90f64
fix: SML serialisation
shumkov May 28, 2024
c5c0c71
test: fix Identity tests with new endpoint
shumkov May 28, 2024
fe71d8d
fix: missing function for web
shumkov May 28, 2024
9388107
ci: enable ssh for action runs
shumkov May 28, 2024
aa90bc5
refactor: use ReconnectableStream
shumkov May 28, 2024
2cea593
Merge branch 'refs/heads/v1.0-dev' into feat/dapi/mnlist-stream
shumkov May 28, 2024
cf636e5
docs: update dapi-client documentation
shumkov May 28, 2024
1b3655b
docs: add todos
shumkov May 28, 2024
a5f64e0
Merge branch 'refs/heads/v1.0-dev' into feat/dapi/mnlist-stream
shumkov May 30, 2024
21e8381
chore: update autogenerated code
shumkov May 30, 2024
cb29caa
ci: enable timeout
shumkov May 30, 2024
5e6f07b
ˆ
shumkov May 30, 2024
fbce295
chore: improve error handling in init
shumkov May 30, 2024
9254126
chore: improve error handling in init
shumkov May 30, 2024
6946d54
perf: serialise only once
shumkov May 30, 2024
4a3e571
refactor: remove unnecessary handle
shumkov May 30, 2024
e976984
test: stop stream when it receieve data or error
shumkov May 30, 2024
aed353e
style: remove unused import
shumkov May 30, 2024
03edff0
test: fix identity test based on stream
shumkov May 30, 2024
78b66b1
test: cancel -> end
shumkov May 30, 2024
c76926f
test: fix stream cancelation
shumkov May 31, 2024
d80533e
ci: enable tmate for debugging
shumkov Jun 3, 2024
8a6b7c6
Merge remote-tracking branch 'origin/feat/dapi/mnlist-stream' into fe…
shumkov Jun 3, 2024
ffcc909
ci: enable tmate for debugging for all runs
shumkov Jun 3, 2024
800da64
ci: add log level
shumkov Jun 4, 2024
7750086
chore: enable silly logs
shumkov Jun 4, 2024
e716fef
fix: masternode list stream is not stopping when wallet is stopped
shumkov Jun 4, 2024
f90a15b
chore: add logging before cancel
shumkov Jun 4, 2024
74d48b4
fix: invalid return value
shumkov Jun 4, 2024
b82031d
chore: disconnect DAPI client with SDK
shumkov Jun 4, 2024
68fcb16
ci: remove tmate
shumkov Jun 4, 2024
fb548aa
chore: remove silly hardcoded logging
shumkov Jun 4, 2024
08bc1e9
revert: change default logging level to info
shumkov Jun 4, 2024
5aea95a
chore: enable silly logs back
shumkov Jun 4, 2024
4491f7f
ci: print config
shumkov Jun 4, 2024
388e909
ci: add more debug information
shumkov Jun 5, 2024
9b3664f
ci: update setup nodejs action
shumkov Jun 5, 2024
ace6632
chore: disconnect faucet client
shumkov Jun 5, 2024
88bda0a
chore: add wallet id for tests
shumkov Jun 7, 2024
a24071a
Merge remote-tracking branch 'origin/feat/dapi/mnlist-stream' into fe…
shumkov Jun 7, 2024
d8b80f5
chore: add wallet id for tests 2
shumkov Jun 7, 2024
3fa33d5
chore: remove only
shumkov Jun 7, 2024
af57266
docs: add todo
shumkov Jun 7, 2024
6e65ddc
chore: disable parallel run
shumkov Jun 7, 2024
04838f7
debug: add loggers
markin-io Jun 7, 2024
c716f7a
debug: add more log
markin-io Jun 7, 2024
f5adcc2
debug: restart CI
markin-io Jun 7, 2024
355e4c5
fix(dapi-client): add hack
markin-io Jun 7, 2024
0d20680
fix(dapi-client): ignore unknown error
markin-io Jun 7, 2024
a7a6f4b
fix(dapi-client): handle unknown error only if stream is missing
markin-io Jun 7, 2024
6577518
chore: disable auto reconnect for masternode list stream
shumkov Jun 8, 2024
9500dee
chore: skip tests for browser
shumkov Jun 8, 2024
da80738
test: fix `this.skip is not a function`
shumkov Jun 8, 2024
e091343
revert: debug changes
shumkov Jun 11, 2024
5403c0a
chore: fix linting and tests for ReconnctableStream
shumkov Jun 11, 2024
cc08a5d
chore: fix linting and remove debug stuff from test suite
shumkov Jun 11, 2024
8a8d318
chore: remove unused logger
shumkov Jun 11, 2024
87dc475
chore: remove unused logger
shumkov Jun 11, 2024
434a25f
chore: remove unused logger
shumkov Jun 11, 2024
23d0e2e
test: do not skip
shumkov Jun 11, 2024
2296bcc
style: unused import
shumkov Jun 11, 2024
5bdffa5
test: skip tests
shumkov Jun 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/nodejs/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ runs:
using: composite
steps:
- name: Setup Node.JS
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: "20"

Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/tests-dashmate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,12 @@ jobs:
key: build-js-artifacts/${{ github.sha }}

- name: Unpack JS build artifacts archive
run: tar -xf build-js-artifacts-${{ github.sha }}.tar
run: tar -xvf build-js-artifacts-${{ github.sha }}.tar

- name: Replace with pre-built images
run: |
set -x

# Login to ECR
DOCKER_HUB_ORG="${{ vars.AWS_ACCOUNT_ID }}.dkr.ecr.${{ vars.AWS_REGION }}.amazonaws.com"
aws ecr get-login-password --region ${{ vars.AWS_REGION }} | docker login --username AWS --password-stdin $DOCKER_HUB_ORG
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -128,9 +128,9 @@ jobs:
uses: actions/checkout@v4

- name: Setup Node.JS
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: "18"
node-version: "20"

- name: Enable corepack
run: corepack enable
Expand Down
11 changes: 7 additions & 4 deletions .pnp.cjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,15 @@ plugins:
'DAPI-Client/getting-started/quickstart.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/quick-start.html'
'DAPI-Client/usage/application/DAPIClient.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/dapiclient.html'
'DAPI-Client/usage/application/core/broadcastTransaction.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/broadcasttransaction.html'
'DAPI-Client/usage/application/core/generateToAddress.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/generatetoaddress.html'
'DAPI-Client/usage/application/core/getBestBlockHash.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/getbestblockhash.html'
'DAPI-Client/usage/application/core/getBlockByHash.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/getblockbyhash.html'
'DAPI-Client/usage/application/core/getBlockByHeight.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/getblockbyheight.html'
'DAPI-Client/usage/application/core/getBlockHash.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/getblockhash.html'
'DAPI-Client/usage/application/core/getMnListDiff.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/getmnlistdiff.html'
'DAPI-Client/usage/application/core/getStatus.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/getstatus.html'
'DAPI-Client/usage/application/core/getTransaction.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/gettransaction.html'
'DAPI-Client/usage/utils/subscribeToTransactionsWithProofs.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/subscribetotransactionswithproofs.html'
'DAPI-Client/usage/application/core/subscribeToTransactionsWithProofs.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/subscribetotransactionswithproofs.html'
'DAPI-Client/usage/application/core/subscribeToMasternodeList.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/subscribetomasternodelist.html'
'DAPI-Client/usage/application/core/subscribeToBlockHeadersWithChainLocks.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/core/subscribetoblockheaderswithchainlocks.html'
'DAPI-Client/usage/application/platform/broadcastStateTransition.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/platform/broadcaststatetransition.html'
'DAPI-Client/usage/application/getDataContract.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/platform/getdatacontract.html'
'DAPI-Client/usage/application/getDocuments.md': 'https://docs.dash.org/projects/platform/en/stable/docs/dapi-client-js/usage/platform/getdocuments.html'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,37 @@ org.dash.platform.dapi.v0.CoreOuterClass.TransactionsWithProofsResponse> getSubs
return getSubscribeToTransactionsWithProofsMethod;
}

private static volatile io.grpc.MethodDescriptor<org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListRequest,
org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListResponse> getSubscribeToMasternodeListMethod;

@io.grpc.stub.annotations.RpcMethod(
fullMethodName = SERVICE_NAME + '/' + "subscribeToMasternodeList",
requestType = org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListRequest.class,
responseType = org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListResponse.class,
methodType = io.grpc.MethodDescriptor.MethodType.SERVER_STREAMING)
public static io.grpc.MethodDescriptor<org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListRequest,
org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListResponse> getSubscribeToMasternodeListMethod() {
io.grpc.MethodDescriptor<org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListRequest, org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListResponse> getSubscribeToMasternodeListMethod;
if ((getSubscribeToMasternodeListMethod = CoreGrpc.getSubscribeToMasternodeListMethod) == null) {
synchronized (CoreGrpc.class) {
if ((getSubscribeToMasternodeListMethod = CoreGrpc.getSubscribeToMasternodeListMethod) == null) {
CoreGrpc.getSubscribeToMasternodeListMethod = getSubscribeToMasternodeListMethod =
io.grpc.MethodDescriptor.<org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListRequest, org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListResponse>newBuilder()
.setType(io.grpc.MethodDescriptor.MethodType.SERVER_STREAMING)
.setFullMethodName(generateFullMethodName(SERVICE_NAME, "subscribeToMasternodeList"))
.setSampledToLocalTracing(true)
.setRequestMarshaller(io.grpc.protobuf.ProtoUtils.marshaller(
org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListRequest.getDefaultInstance()))
.setResponseMarshaller(io.grpc.protobuf.ProtoUtils.marshaller(
org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListResponse.getDefaultInstance()))
.setSchemaDescriptor(new CoreMethodDescriptorSupplier("subscribeToMasternodeList"))
.build();
}
}
}
return getSubscribeToMasternodeListMethod;
}

/**
* Creates a new async stub that supports all call types for the service
*/
Expand Down Expand Up @@ -405,6 +436,13 @@ public void subscribeToTransactionsWithProofs(org.dash.platform.dapi.v0.CoreOute
io.grpc.stub.ServerCalls.asyncUnimplementedUnaryCall(getSubscribeToTransactionsWithProofsMethod(), responseObserver);
}

/**
*/
public void subscribeToMasternodeList(org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListRequest request,
io.grpc.stub.StreamObserver<org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListResponse> responseObserver) {
io.grpc.stub.ServerCalls.asyncUnimplementedUnaryCall(getSubscribeToMasternodeListMethod(), responseObserver);
}

@java.lang.Override public final io.grpc.ServerServiceDefinition bindService() {
return io.grpc.ServerServiceDefinition.builder(getServiceDescriptor())
.addMethod(
Expand Down Expand Up @@ -470,6 +508,13 @@ public void subscribeToTransactionsWithProofs(org.dash.platform.dapi.v0.CoreOute
org.dash.platform.dapi.v0.CoreOuterClass.TransactionsWithProofsRequest,
org.dash.platform.dapi.v0.CoreOuterClass.TransactionsWithProofsResponse>(
this, METHODID_SUBSCRIBE_TO_TRANSACTIONS_WITH_PROOFS)))
.addMethod(
getSubscribeToMasternodeListMethod(),
io.grpc.stub.ServerCalls.asyncServerStreamingCall(
new MethodHandlers<
org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListRequest,
org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListResponse>(
this, METHODID_SUBSCRIBE_TO_MASTERNODE_LIST)))
.build();
}
}
Expand Down Expand Up @@ -559,6 +604,14 @@ public void subscribeToTransactionsWithProofs(org.dash.platform.dapi.v0.CoreOute
io.grpc.stub.ClientCalls.asyncServerStreamingCall(
getChannel().newCall(getSubscribeToTransactionsWithProofsMethod(), getCallOptions()), request, responseObserver);
}

/**
*/
public void subscribeToMasternodeList(org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListRequest request,
io.grpc.stub.StreamObserver<org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListResponse> responseObserver) {
io.grpc.stub.ClientCalls.asyncServerStreamingCall(
getChannel().newCall(getSubscribeToMasternodeListMethod(), getCallOptions()), request, responseObserver);
}
}

/**
Expand Down Expand Up @@ -639,6 +692,14 @@ public java.util.Iterator<org.dash.platform.dapi.v0.CoreOuterClass.TransactionsW
return io.grpc.stub.ClientCalls.blockingServerStreamingCall(
getChannel(), getSubscribeToTransactionsWithProofsMethod(), getCallOptions(), request);
}

/**
*/
public java.util.Iterator<org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListResponse> subscribeToMasternodeList(
org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListRequest request) {
return io.grpc.stub.ClientCalls.blockingServerStreamingCall(
getChannel(), getSubscribeToMasternodeListMethod(), getCallOptions(), request);
}
}

/**
Expand Down Expand Up @@ -721,6 +782,7 @@ public com.google.common.util.concurrent.ListenableFuture<org.dash.platform.dapi
private static final int METHODID_GET_ESTIMATED_TRANSACTION_FEE = 6;
private static final int METHODID_SUBSCRIBE_TO_BLOCK_HEADERS_WITH_CHAIN_LOCKS = 7;
private static final int METHODID_SUBSCRIBE_TO_TRANSACTIONS_WITH_PROOFS = 8;
private static final int METHODID_SUBSCRIBE_TO_MASTERNODE_LIST = 9;

private static final class MethodHandlers<Req, Resp> implements
io.grpc.stub.ServerCalls.UnaryMethod<Req, Resp>,
Expand Down Expand Up @@ -775,6 +837,10 @@ public void invoke(Req request, io.grpc.stub.StreamObserver<Resp> responseObserv
serviceImpl.subscribeToTransactionsWithProofs((org.dash.platform.dapi.v0.CoreOuterClass.TransactionsWithProofsRequest) request,
(io.grpc.stub.StreamObserver<org.dash.platform.dapi.v0.CoreOuterClass.TransactionsWithProofsResponse>) responseObserver);
break;
case METHODID_SUBSCRIBE_TO_MASTERNODE_LIST:
serviceImpl.subscribeToMasternodeList((org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListRequest) request,
(io.grpc.stub.StreamObserver<org.dash.platform.dapi.v0.CoreOuterClass.MasternodeListResponse>) responseObserver);
break;
default:
throw new AssertionError();
}
Expand Down Expand Up @@ -845,6 +911,7 @@ public static io.grpc.ServiceDescriptor getServiceDescriptor() {
.addMethod(getGetEstimatedTransactionFeeMethod())
.addMethod(getSubscribeToBlockHeadersWithChainLocksMethod())
.addMethod(getSubscribeToTransactionsWithProofsMethod())
.addMethod(getSubscribeToMasternodeListMethod())
.build();
}
}
Expand Down
35 changes: 35 additions & 0 deletions packages/dapi-grpc/clients/core/v0/nodejs/CorePromiseClient.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ const {
GetEstimatedTransactionFeeResponse: PBJSGetEstimatedTransactionFeeResponse,
TransactionsWithProofsRequest: PBJSTransactionsWithProofsRequest,
TransactionsWithProofsResponse: PBJSTransactionsWithProofsResponse,
MasternodeListRequest: PBJSMasternodeListRequest,
MasternodeListResponse: PBJSMasternodeListResponse,
},
},
},
Expand All @@ -59,6 +61,7 @@ const {
BlockHeadersWithChainLocksResponse: ProtocBlockHeadersWithChainLocksResponse,
GetEstimatedTransactionFeeResponse: ProtocGetEstimatedTransactionFeeResponse,
TransactionsWithProofsResponse: ProtocTransactionsWithProofsResponse,
MasternodeListResponse: ProtocMasternodeListResponse,
} = require('./core_protoc');

const getCoreDefinition = require('../../../../lib/getCoreDefinition');
Expand Down Expand Up @@ -398,6 +401,38 @@ class CorePromiseClient {
},
);
}

/**
* @param {MasternodeListRequest} masternodeListRequest The request proto
* @param {?Object<string, string>} metadata User defined call metadata
* @param {CallOptions} [options={}]
* @return {!grpc.web.ClientReadableStream<!MasternodeListResponse>|undefined}
* The XHR Node Readable Stream
*/
subscribeToMasternodeList(masternodeListRequest, metadata = {}, options = {}) {
if (!isObject(metadata)) {
throw new Error('metadata must be an object');
}

return this.client.subscribeToMasternodeList(
masternodeListRequest,
convertObjectToMetadata(metadata),
{
interceptors: [
jsonToProtobufInterceptorFactory(
jsonToProtobufFactory(
ProtocMasternodeListResponse,
PBJSMasternodeListResponse,
),
protobufToJsonFactory(
PBJSMasternodeListRequest,
),
),
],
...options,
},
);
}
}

module.exports = CorePromiseClient;
Loading
Loading