Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protobuf metadata #32

Merged
merged 21 commits into from
Jul 28, 2022
Merged

Protobuf metadata #32

merged 21 commits into from
Jul 28, 2022

Conversation

NRHelmi
Copy link
Contributor

@NRHelmi NRHelmi commented Jun 15, 2022

This PR adds support to metadata protobuf format parsing.

@NRHelmi NRHelmi marked this pull request as ready for review July 10, 2022 15:53
Denis Gursky added 8 commits July 12, 2022 15:24
# Conflicts:
#	package-lock.json
#	package.json
#	tsconfig.json
# Conflicts:
#	index.web.ts
#	src/api/transaction/multipartMock
#	src/transaction/mocks/multipart
#	src/transaction/multipartMock
Comment on lines +92 to +94
if (!file?.arrayBuffer) {
throw new Error(`Unsupported metadata type: ${typeof file}`);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this will happen on older engines, right? Should we have the error message tell users to upgrade their engines?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it won't. It's looking for metadata.proto part, so it won't get here.

Copy link
Member

@NHDaly NHDaly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this LGTM. I haven't finished reading the tests, but the main core of the code looks right :) Thanks both!

@denisgursky denisgursky requested a review from NHDaly July 19, 2022 21:20
@NHDaly NHDaly requested a review from bachdavi July 28, 2022 14:40
Copy link
Member

@bachdavi bachdavi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice and clean :)

Comment on lines +61 to +63
"build": "rm -rf ./dist && webpack build",
"proto-gen": "npx protoc --ts_out src/proto/generated/ --proto_path src/proto src/proto/*.proto",
"dev": "rm -rf ./dist && webpack watch",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are build and dev unrelated cleanups?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

@@ -0,0 +1,11 @@

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit pick: Can we use the proper file extension :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure we can, I just I have no idea about what that would be. What's the proper file extension for protobuf binaries?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.proto i think

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

@vilterp vilterp Jul 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah. I guess we didn't get this right in the actual protocol.

Maybe .pb, i.e. metadata.pb?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to metadata.pb

@@ -0,0 +1,160 @@
syntax = "proto3";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is engine internal and not needed in the client :)
Please delete it!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thx!

Comment on lines +50 to +53
// TODO uncomment and make TransactionAsyncResult.metadata required
// if (!metadataProto) {
// throw new Error('metadata proto part not found');
// }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a TODO for this PR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it'll be removed later. Just don't want it to throw errors on old engines.

Comment on lines +91 to +94
export async function readProtoMetadata(file: File | Blob) {
if (!file?.arrayBuffer) {
throw new Error(`Unsupported metadata type: ${typeof file}`);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need the error handling? In which case would either the File or Blob type not contain the array buffer?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think so. We had an issue when node-fetch was returning a string instead of File | Blob

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's fixed in the engines now, but i guess there could always be an old engine lurking out there 👀
makes sense, thanks

Copy link
Member

@NHDaly NHDaly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Denis! Pair-reviewed with @bachdavi

@@ -0,0 +1,1283 @@
// @generated by protobuf-ts 2.7.0
Copy link
Member

@NHDaly NHDaly Jul 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(and please delete this file too, of course)

@denisgursky
Copy link
Contributor

@NHDaly @bachdavi @vilterp thanks for the review!

@denisgursky denisgursky merged commit b3a3fc6 into main Jul 28, 2022
@denisgursky denisgursky deleted the hnr-proto-metadata branch July 28, 2022 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants