stargate: proto files #679

fedekunze · 2020-12-29T14:59:08Z

This PR contains the protobuf messages for the evm module types, keys and account type. It also contains the required Makefile instructions to generate the golang types from the .proto files. Currently doesn't work because of the required dependencies from SDK v0.40 (i.e still needs the final wiring that will be completed with #546).

noot

currently when I do make proto-all I get:

$ make proto-all 
Formatting Protobuf files
/usr/bin/docker run --rm -v /home/elizabeth/go/src/github.com/cosmos/ethermint:/workspace \
--workdir /workspace tendermintdev/docker-build-proto \
find ./ -not -path "./third_party/*" -name *.proto -exec clang-format -i {} \;
Unable to find image 'tendermintdev/docker-build-proto:latest' locally
latest: Pulling from tendermintdev/docker-build-proto
7597eaba0060: Pull complete 
c7720d71f785: Pull complete 
2c258b1fe3ea: Pull complete 
2d8f0dc37fcf: Pull complete 
52f1f95728be: Pull complete 
Digest: sha256:4e37e90ed47b62ccf8268fcf9d6948179aad98da3a4feaf7fe42696861f26070
Status: Downloaded newer image for tendermintdev/docker-build-proto:latest
Unable to find image 'bufbuild/buf:latest' locally
latest: Pulling from bufbuild/buf
df20fa9351a1: Already exists 
90930e25fd19: Pull complete 
97bea53d2d20: Pull complete 
Digest: sha256:acfe0dca8ffaad3723f1936de3d8b04202eaae3e4f67f7d2449ebc02ccca9b2f
Status: Downloaded newer image for bufbuild/buf:latest
Command "lint" is deprecated, "buf check lint" has been moved to "buf lint", use "buf lint" instead.
We recommend migrating, however this command continues to work.
See https://docs.buf.build/faq for more details.
WARN	File "buf.yaml" has no version set. Please add "version: v1beta1". See https://docs.buf.build/faq for more details.
{"path":"proto/ethermint/crypto/v1alpha1/ethsecp256k1/keys.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"proto/ethermint/evm/v1alpha1/evm.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"proto/ethermint/evm/v1alpha1/genesis.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"proto/ethermint/evm/v1alpha1/query.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"proto/ethermint/evm/v1alpha1/tx.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"proto/ethermint/types/v1alpha1/account.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"cosmos/auth/v1beta1/auth.proto: does not exist"}
{"path":"third_party/proto/tendermint/abci/types.proto","start_line":8,"start_column":8,"end_line":8,"end_column":8,"type":"COMPILE","message":"tendermint/crypto/proof.proto: does not exist"}
{"path":"third_party/proto/tendermint/crypto/keys.proto","start_line":6,"start_column":8,"end_line":6,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"third_party/proto/tendermint/crypto/proof.proto","start_line":6,"start_column":8,"end_line":6,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"third_party/proto/tendermint/types/types.proto","start_line":6,"start_column":8,"end_line":6,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
Makefile:367: recipe for target 'proto-lint' failed
make: *** [proto-lint] Error 1

is this expected?

araskachoi · 2021-01-05T18:22:48Z

currently when I do make proto-all I get:
...
is this expected?

I am also getting this:

danielchoi$ make proto-all
Formatting Protobuf files
/usr/local/bin/docker run --rm -v /Users/danielchoi/Documents/daniel/chainsafe/ethermint-project/ethermint:/workspace \
        --workdir /workspace tendermintdev/docker-build-proto \
        find ./ -not -path "./third_party/*" -name *.proto -exec clang-format -i {} \;
Unable to find image 'tendermintdev/docker-build-proto:latest' locally
latest: Pulling from tendermintdev/docker-build-proto
7597eaba0060: Pull complete 
c7720d71f785: Pull complete 
2c258b1fe3ea: Pull complete 
2d8f0dc37fcf: Pull complete 
52f1f95728be: Pull complete 
Digest: sha256:4e37e90ed47b62ccf8268fcf9d6948179aad98da3a4feaf7fe42696861f26070
Status: Downloaded newer image for tendermintdev/docker-build-proto:latest
Unable to find image 'bufbuild/buf:latest' locally
latest: Pulling from bufbuild/buf
df20fa9351a1: Already exists 
90930e25fd19: Pull complete 
97bea53d2d20: Pull complete 
Digest: sha256:acfe0dca8ffaad3723f1936de3d8b04202eaae3e4f67f7d2449ebc02ccca9b2f
Status: Downloaded newer image for bufbuild/buf:latest
Command "lint" is deprecated, "buf check lint" has been moved to "buf lint", use "buf lint" instead.
We recommend migrating, however this command continues to work.
See https://docs.buf.build/faq for more details.
WARN    File "buf.yaml" has no version set. Please add "version: v1beta1". See https://docs.buf.build/faq for more details.
{"path":"proto/ethermint/crypto/v1alpha1/ethsecp256k1/keys.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"proto/ethermint/evm/v1alpha1/evm.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"proto/ethermint/evm/v1alpha1/genesis.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"proto/ethermint/evm/v1alpha1/query.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"proto/ethermint/evm/v1alpha1/tx.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"proto/ethermint/types/v1alpha1/account.proto","start_line":4,"start_column":8,"end_line":4,"end_column":8,"type":"COMPILE","message":"cosmos/auth/v1beta1/auth.proto: does not exist"}
{"path":"third_party/proto/tendermint/abci/types.proto","start_line":8,"start_column":8,"end_line":8,"end_column":8,"type":"COMPILE","message":"tendermint/crypto/proof.proto: does not exist"}
{"path":"third_party/proto/tendermint/crypto/keys.proto","start_line":6,"start_column":8,"end_line":6,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"third_party/proto/tendermint/crypto/proof.proto","start_line":6,"start_column":8,"end_line":6,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
{"path":"third_party/proto/tendermint/types/types.proto","start_line":6,"start_column":8,"end_line":6,"end_column":8,"type":"COMPILE","message":"gogoproto/gogo.proto: does not exist"}
make: *** [proto-lint] Error 1

araskachoi · 2021-01-05T18:25:49Z

for make proto-gen:

make proto-gen
Generating Protobuf files
/usr/local/bin/docker run --rm -v /Users/danielchoi/Documents/daniel/chainsafe/ethermint-project/ethermint:/workspace --workdir /workspace tendermintdev/sdk-proto-gen sh ./scripts/protocgen.sh
proto/ethermint/evm/v1alpha1/query.proto:5:8:google/api/annotations.proto: does not exist
make: *** [proto-gen] Error 1

for make proto-gen-any:

make proto-gen-any
/usr/local/bin/docker run --rm -v /Users/danielchoi/Documents/daniel/chainsafe/ethermint-project/ethermint:/workspace --workdir /workspace tendermintdev/sdk-proto-gen sh ./scripts/protocgen-any.sh
go: downloading github.com/regen-network/protobuf v1.3.2-alpha.regen.4
path "google/protobuf/any.proto" has no matching file in the module
make: *** [proto-gen-any] Error 1

proto-check-breaking:

Command "breaking" is deprecated, "buf check breaking" has been moved to "buf breaking", use "buf breaking" instead.
We recommend migrating, however this command continues to work.
See https://docs.buf.build/faq for more details.
Flag --against-input has been deprecated, use --against instead.
We recommend migrating, however this flag continues to work.
See https://docs.buf.build/faq for more details.
WARN    File "buf.yaml" has no version set. Please add "version: v1beta1". See https://docs.buf.build/faq for more details.
proto/ethermint/crypto/v1alpha1/ethsecp256k1/keys.proto:4:8:gogoproto/gogo.proto: does not exist
proto/ethermint/evm/v1alpha1/evm.proto:4:8:gogoproto/gogo.proto: does not exist
proto/ethermint/evm/v1alpha1/genesis.proto:4:8:gogoproto/gogo.proto: does not exist
proto/ethermint/evm/v1alpha1/query.proto:4:8:gogoproto/gogo.proto: does not exist
proto/ethermint/evm/v1alpha1/tx.proto:4:8:gogoproto/gogo.proto: does not exist
proto/ethermint/types/v1alpha1/account.proto:4:8:cosmos/auth/v1beta1/auth.proto: does not exist
third_party/proto/tendermint/abci/types.proto:8:8:tendermint/crypto/proof.proto: does not exist
third_party/proto/tendermint/crypto/keys.proto:6:8:gogoproto/gogo.proto: does not exist
third_party/proto/tendermint/crypto/proof.proto:6:8:gogoproto/gogo.proto: does not exist
third_party/proto/tendermint/types/types.proto:6:8:gogoproto/gogo.proto: does not exist
make: *** [proto-check-breaking] Error 1

… into protobuf-stargate

fedekunze · 2021-01-05T19:34:44Z

the ... : does not exist is expected because we need to update to SDK v0.40 to be able to include the SDK's and Tendermint's protobuf types. This PR is just to define the proto files

fedekunze · 2021-01-07T11:56:15Z

@araskachoi @noot can you give this another review?

noot · 2021-01-07T20:17:50Z

third_party/proto/cosmos_proto/cosmos.proto

+extend google.protobuf.MessageOptions {
+    string interface_type = 93001;
+
+    string implements_interface = 93002;
+}
+
+extend google.protobuf.FieldOptions {
+    string accepts_interface = 93001;
+}


did these come from somewhere? where are they defined?

third party dependencies for the other protobuf files

noot · 2021-01-07T20:18:27Z

scripts/protoc-swagger-gen.sh

+# combine swagger files
+# uses nodejs package `swagger-combine`.
+# all the individual swagger files need to be configured in `config.json` for merging
+swagger-combine ./client/docs/config.json -o ./client/docs/swagger-ui/swagger.yaml -f yaml --continueOnConflictingPaths true --includeDefinitions true


what is this file for?

generates a swagger file like this one https://cosmos.network/rpc/master

scripts/protocgen-any.sh

noot · 2021-01-07T20:25:19Z

proto/ethermint/crypto/v1alpha1/ethsecp256k1/keys.proto

+// Tendermint's PubKey interface. It represents the 33-byte compressed public
+// key format.
+message PubKey {
+  option (gogoproto.goproto_stringer) = false;


what does this option do?

it allows you to create your custom String() string function instead of the default one. This is to add the ethereum address when printing the account

noot

since the proto files are going to be compiled into go files with the defined types, are we going to end up using the types that are in the pb.go files, or the types that are currently defined in the codebase?

fedekunze · 2021-01-07T20:36:32Z

since the proto files are going to be compiled into go files with the defined types, are we going to end up using the types that are in the pb.go files, or the types that are currently defined in the codebase?

we will be using the ones defined defined on the pb.go files, I can't generate them because protobuf is not supported on 0.39

… into protobuf-stargate

noot

nice work!

fedekunze added 3 commits December 29, 2020 11:58

stargate: proto files

dfb7ca8

third party proto files

c7c1d27

third party proto files

a335b2f

fedekunze marked this pull request as ready for review December 30, 2020 19:03

Merge branch 'development' into protobuf-stargate

4a02721

fedekunze requested a review from noot as a code owner December 30, 2020 19:03

fedekunze added the Status: Needs Review label Dec 30, 2020

fedekunze requested a review from araskachoi December 30, 2020 19:29

noot reviewed Jan 5, 2021

View reviewed changes

fedekunze added 3 commits January 5, 2021 16:29

add annotations

f4f9fa8

Merge branch 'protobuf-stargate' of https://github.com/cosmos/ethermint…

b1ac5cd

… into protobuf-stargate

rm proto-gen-any

0ad500b

Merge branch 'development' into protobuf-stargate

1514a31

fedekunze requested a review from noot January 5, 2021 19:35

fedekunze added 2 commits January 6, 2021 17:57

Merge branch 'development' into protobuf-stargate

47d7432

Merge branch 'development' into protobuf-stargate

7ee1b59

noot reviewed Jan 7, 2021

View reviewed changes

scripts/protocgen-any.sh Outdated Show resolved Hide resolved

noot reviewed Jan 7, 2021

View reviewed changes

fedekunze added 2 commits January 7, 2021 18:16

rm script any

4b66681

Merge branch 'protobuf-stargate' of https://github.com/cosmos/ethermint…

cae50aa

… into protobuf-stargate

noot approved these changes Jan 7, 2021

View reviewed changes

fedekunze merged commit ed313c9 into development Jan 7, 2021

fedekunze deleted the protobuf-stargate branch January 7, 2021 21:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stargate: proto files #679

stargate: proto files #679

fedekunze commented Dec 29, 2020 •

edited

Loading

noot left a comment

araskachoi commented Jan 5, 2021

araskachoi commented Jan 5, 2021 •

edited

Loading

fedekunze commented Jan 5, 2021

fedekunze commented Jan 7, 2021

noot Jan 7, 2021

fedekunze Jan 7, 2021

noot Jan 7, 2021

fedekunze Jan 7, 2021

noot Jan 7, 2021

fedekunze Jan 7, 2021

noot left a comment

fedekunze commented Jan 7, 2021

noot left a comment

stargate: proto files #679

stargate: proto files #679

Conversation

fedekunze commented Dec 29, 2020 • edited Loading

noot left a comment

Choose a reason for hiding this comment

araskachoi commented Jan 5, 2021

araskachoi commented Jan 5, 2021 • edited Loading

fedekunze commented Jan 5, 2021

fedekunze commented Jan 7, 2021

noot Jan 7, 2021

Choose a reason for hiding this comment

fedekunze Jan 7, 2021

Choose a reason for hiding this comment

noot Jan 7, 2021

Choose a reason for hiding this comment

fedekunze Jan 7, 2021

Choose a reason for hiding this comment

noot Jan 7, 2021

Choose a reason for hiding this comment

fedekunze Jan 7, 2021

Choose a reason for hiding this comment

noot left a comment

Choose a reason for hiding this comment

fedekunze commented Jan 7, 2021

noot left a comment

Choose a reason for hiding this comment

fedekunze commented Dec 29, 2020 •

edited

Loading

araskachoi commented Jan 5, 2021 •

edited

Loading