Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add map-bridge proposal #59

Merged
merged 7 commits into from
Nov 12, 2020
Merged

Add map-bridge proposal #59

merged 7 commits into from
Nov 12, 2020

Conversation

zcheng9
Copy link
Contributor

@zcheng9 zcheng9 commented Sep 22, 2020

Grant Application Checklist

  • The application-template.md has been copied, renamed ( "project_name.md") and updated.
  • A BTC address for the payment of the milestones is provided inside the application.
  • The software of the project will be released under the Apache license version 2.0 as specified in the terms and conditions.
  • The total funding amount of the project is below CHF 30k at the time of submission.

@CLAassistant
Copy link

CLAassistant commented Sep 22, 2020

CLA assistant check
All committers have signed the CLA.

@Noc2
Copy link
Collaborator

Noc2 commented Sep 22, 2020

Thanks for the application. You still needs to sign the terms via the cla assistant

@Noc2 Noc2 added the changes requested The team needs to clarify a few things first. label Sep 22, 2020
@zcheng9
Copy link
Contributor Author

zcheng9 commented Sep 24, 2020

Thanks for the application. You still needs to sign the terms via the cla assistant

Hello, we have signed the terms via the cla assistant. Thank you for you review!

@mmagician
Copy link
Contributor

@zcheng9 Thanks for the application. We are looking into it.

@mmagician mmagician removed the changes requested The team needs to clarify a few things first. label Sep 28, 2020
@mmagician
Copy link
Contributor

@zcheng9 Could you please specify the method signatures for each of the deliverables that you've included in the
milestones? Note that we require proper unit test coverage for any logic written - please also include a point regarding this for each milestone.

What exactly are you planning to develop as part of Substrate module: MMR Trie Structure? Are you aware of the existing MMR implementation (here)? I've looked at their code and it seems pretty feature-complete, as far as MMRs are concerned. Let me know your thoughts.

@zcheng9
Copy link
Contributor Author

zcheng9 commented Sep 30, 2020

Thank you for your reply for our proposal. We have included the method signatures for each of the deliverables and unit test in our milestones.

And for the MMR Implementation, we have checked the code you referred. It seems they have done a quite good job. However, there is some functionality we might need while they do not provide. Fox examples, we need a customized field for each node(time stamp, difficulty, counter, etc.), as well as we need a customized method to merge two child nodes and construct a parent node(not simply hash them together, we need to handle the the customized filed when merging). Furthermore we need to construct merkle proof for a bunch of leaf nodes, some of the branch nodes maybe exist in multiple times, we can use one single merkle proof for a list of leaf nodes. We have a Go implementation for this case. So we might implement MMR Trie based on the code you referred and extend some more functionality we need based on their code.

@mmagician mmagician self-requested a review September 30, 2020 15:31
Copy link
Contributor

@mmagician mmagician left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zcheng9 I don't see the method signatures anywhere.

So are you planning to use (&extend) the existing MMR implementation?

@Noc2 Noc2 added the changes requested The team needs to clarify a few things first. label Oct 8, 2020
@zcheng9
Copy link
Contributor Author

zcheng9 commented Oct 9, 2020

@zcheng9 I don't see the method signatures anywhere.

So are you planning to use (&extend) the existing MMR implementation?

Hello, we have added the method signatures for each milestone.

And for MMR implementation, we indeed plan to extend the existing MMR implementaion.

@mmagician
Copy link
Contributor

@zcheng9 I still do not see signatures anywhere - perhaps you misunderstood what I was asking for, maybe also because I was using the Java-borrowed naming (method signature vs type signature?). Here is an explanation.

Anyway, would you mind jumping on an introductory call with us? Could you reach out to me on Element at @marcin:web3.foundation so we can coordinate better, please?

@zcheng9
Copy link
Contributor Author

zcheng9 commented Oct 20, 2020

@zcheng9 I still do not see signatures anywhere - perhaps you misunderstood what I was asking for, maybe also because I was using the Java-borrowed naming (method signature vs type signature?). Here is an explanation.

Anyway, would you mind jumping on an introductory call with us? Could you reach out to me on Element at @marcin:web3.foundation so we can coordinate better, please?

Sorry for the misunderstanding, I have added you on Element and we can make talk about MAP Bridge on Element.

@Lederstrumpf
Copy link
Contributor

Lederstrumpf commented Oct 20, 2020

@zcheng9 - your milestone 3 includes a cross-chain asset exchange demo, but you do not provide any details on how this automatically follows from implementing FlyClient. Are you proposing HTLC atomic swaps or something else?

@zcheng9
Copy link
Contributor Author

zcheng9 commented Oct 21, 2020

@zcheng9 - your milestone 3 includes a cross-chain asset exchange demo, but you do not provide any details on how this automatically follows from implementing FlyClient. Are you proposing HTLC atomic swaps or something else?

@Lederstrumpf Oh, it should be a cross-chain digit asset transfer demo. We will borrow the idea from pegged sidechain partially. The difference is we will use Flyclient(as reference as ULVP in proposal) for cross chain transaction verifying instead of SPV. The demo would involve two blockchain(A and B), if user locked some token in A chain, B chain could verify such locked transaction through a reference contract deployed in chain B(this step is what ULVP or Flyclient is involved). And a mint contract in chain B could call this reference contract to mint same amount token in chain B once such transaction passed the contest period. Thus the token transfer is complete. Here is a demo we developed earlier for two testchain (mouse and duck).

Copy link
Collaborator

@Noc2 Noc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the application. A few questions/comments from my side:

  • What are your future plans with the project? Do you have a concrete network you want to bridge? Do you want to become a parachain?
  • The price of the milestones needs to be in BTC instead of fiat.
  • Could you reduce the price of milestone 1, given that it’s already implemented?
  • Could you comment on your connection to softbank?

Reduce to 2 milestones and add some method signature.
@zcheng9
Copy link
Contributor Author

zcheng9 commented Oct 29, 2020

Thanks for the application. A few questions/comments from my side:

  • What are your future plans with the project? Do you have a concrete network you want to bridge? Do you want to become a parachain?
  • The price of the milestones needs to be in BTC instead of fiat.
  • Could you reduce the price of milestone 1, given that it’s already implemented?
  • Could you comment on your connection to softbank?

@Noc2 Thank you for your comment.

  1. We plan to extend our bridge to most POW and POS type blockchains in the future. And at this moment, we plan to bridge Ethereum first. We ineed want to become a parachain.

  2. We have changed it in BTC.

  3. The implemented MMR could not satisfy our demand. For example, we need retrieve the whole MMR based on the given root, which means we need to manipulate MMR dynamiclly(this case is similar to MPT, there should a way to retrieve history MMR when given the block header). Thus we need to think about how to store the MMR in memeory and in persistent database. Besides, merging two node in MMR is not simply hash two node together. It is something like the graph here. And this can not be done using the current implemetation. Thus it would involve a lot more jobs. And we think the fund should not be reduced since we think we will put equally effort to the milestone 1. However, we reduce the total cost because we compress our work into 2 milestones.

  4. We were invested by Softbank Investment Pte Ltd(singapore)before.

@mmagician
Copy link
Contributor

@zcheng9 Thanks for adjusting the application. Regarding the existing MMR library:

  1. we need retrieve the whole MMR based on the given root

From my understanding of the existing MMR implementation, the MMR data structure can be stored as a vector of hashes, as shown in the tests. If you want to get a subtree based on a certain hash of a node, you could use:

  1. find_node_index
  2. Get the subtree up until that node index: subtree = hashes[0:desired_node_index]

Wouldn't that work?

  1. Besides, merging two node in MMR is not simply hash two node together

As I understand, you need to define your own fields for the node, right? (e.g. counter)
Could you not extend the existing implementation and, for example, add a counters field to the MerkleMountainRange struct?

@zcheng9
Copy link
Contributor Author

zcheng9 commented Oct 29, 2020

@zcheng9 Thanks for adjusting the application. Regarding the existing MMR library:

  1. we need retrieve the whole MMR based on the given root

From my understanding of the existing MMR implementation, the MMR data structure can be stored as a vector of hashes, as shown in the tests. If you want to get a subtree based on a certain hash of a node, you could use:

  1. find_node_index
  2. Get the subtree up until that node index: subtree = hashes[0:desired_node_index]

Wouldn't that work?

  1. Besides, merging two node in MMR is not simply hash two node together

As I understand, you need to define your own fields for the node, right? (e.g. counter)
Could you not extend the existing implementation and, for example, add a counters field to the MerkleMountainRange struct?

@mmagician Thank you for your comments. I will answer your questions as follows:

  1. This stratege you mentioned is what we use for our current Golang demo. However we find this is not optimal when the retrieve operation is frequent. Every time you need to make a copy of MMR and then get the subtree. We plan to use the design similar to MPT in Ethereum, which can extract the whole MMR based on a given root. Also we need a cache mechnism for all MMR generated.

  2. We need a field we define, as well as the merging method. It's not as simple as adding a couters field. The generate MMR proof and verify MMR proof would use the same merging method we provide too.

As a conclusion, we need do some extra work to make the MMR fit our ULVP. And this is what milestone 1 would do. Besides, milestone 1 would do some jobs other than MMR. For example, after sampling blocks in ULVP, we need to generate MMR proof for a bunch of leaf nodes(not a single leaf node as the implementation you refer). Notice that these proof might have a lot of ovelapped intermediate nodes. Instead generating an array of proofs, we wouldd combine all these proofs into one hybrid proof(overlapped nodes would appear only once in hybrid proof).

We added the detail description of three steps of ultra-light verification protocol, and the asset transfer demo between parachain in Polkadot and Ethereum testnet. Also we gibe some more details of milestone 2.
@zcheng9
Copy link
Contributor Author

zcheng9 commented Nov 6, 2020

@mmagician We have added some more details of our MAP bridge, could you please check it? Thanks.

Add three ways to have MMR root in Ethereum and P2P substrate module details
@zcheng9 zcheng9 requested a review from Noc2 November 10, 2020 07:05
@mmagician mmagician self-requested a review November 11, 2020 07:34
@mmagician mmagician added ready for review The project is ready to be reviewed by the committee members. and removed changes requested The team needs to clarify a few things first. labels Nov 11, 2020
Copy link
Contributor

@mmagician mmagician left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for all the updates, it looks good to me!

@Noc2
Copy link
Collaborator

Noc2 commented Nov 11, 2020

In general, it looks also interesting to me, but I like to get some additional feedback from our researchers first and have shared the application again with them. Also are you aware of https://github.com/Snowfork/polkadot-ethereum?

@zcheng9
Copy link
Contributor Author

zcheng9 commented Nov 11, 2020

In general, it looks also interesting to me, but I like to get some additional feedback from our researchers first and have shared the application again with them. Also are you aware of https://github.com/Snowfork/polkadot-ethereum?

Thanks! We are aware of Snowfork. The different between these two bridge is we do not require trustness on the relayer group to deliver block headers. The way to verify the validity of a block header in our program is based on flyclient paper.

@zcheng9
Copy link
Contributor Author

zcheng9 commented Nov 11, 2020

Thank you for all the updates, it looks good to me!

Thanks.

@AlistairStewart
Copy link

As I understand it, you want to construct an MMR of every block hash on Ethereum in any Ethereum smart contract. Wouldn't this be quite expensive in terms of gas?

@Noc2
Copy link
Collaborator

Noc2 commented Nov 11, 2020

Thanks! We are aware of Snowfork. The different between these two bridge is we do not require trustness on the relayer group to deliver block headers. The way to verify the validity of a block header in our program is based on flyclient paper.

Thanks for the reply. Would you be interested in starting potentially with a different network initially other than ethereum (given that we already have an ethereum bridge under development)? Independent of it I’m happy to go ahead with it.

@zcheng9
Copy link
Contributor Author

zcheng9 commented Nov 11, 2020

As I understand it, you want to construct an MMR of every block hash on Ethereum in any Ethereum smart contract. Wouldn't this be quite expensive in terms of gas?

@AlistairStewart It would be epoch-based MMR, not every block hash. We could set a epoch with 512 blocks or 256 blocks, Each epoch would generate a small MMR and be represented as a MMR root.

@zcheng9
Copy link
Contributor Author

zcheng9 commented Nov 11, 2020

Thanks! We are aware of Snowfork. The different between these two bridge is we do not require trustness on the relayer group to deliver block headers. The way to verify the validity of a block header in our program is based on flyclient paper.

Thanks for the reply. Would you be interested in starting potentially with a different network initially other than ethereum (given that we already have an ethereum bridge under development)? Independent of it I’m happy to go ahead with it.

@Noc2 Thanks for comment. As we mentioned in the future plan, we indeed would promote our work to other network later. And could we start with Ethereum first since we have done a lot of preparation work for it.

@musnit
Copy link

musnit commented Nov 12, 2020

Thanks! We are aware of Snowfork. The different between these two bridge is we do not require trustness on the relayer group to deliver block headers. The way to verify the validity of a block header in our program is based on flyclient paper.

FYI Snowfork's bridge will not require trust in the relays either - we're building an Ethereum Light Client into our parachain and an MMR/FlyClient-like Polkadot Light client into our smart contracts based on the MMR proofs created by this gadget.

@RouvenP RouvenP merged commit dfdeb1d into w3f:master Nov 12, 2020
@github-actions
Copy link
Contributor

Congratulations! As part of the Open Grants Program, we want to help winning teams acknowledge their grants publicly while observing the foundation’s guidelines. To that end, we’ve created a badge for grant-winning teams. Here is a link to the download and guidelines.

If you are interested in making an announcement about your grant and you would like us to broadcast it on our media channels then please get in touch with us at [email protected] prior to making the announcement (at least two weeks notice is preferred) so that we can coordinate on timing and content.

@Ebimk2
Copy link

Ebimk2 commented Dec 17, 2020

Thank you

@Ebimk2
Copy link

Ebimk2 commented Dec 17, 2020

#59

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review The project is ready to be reviewed by the committee members.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants