Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for hash, content type, and multiple URLs as service endpoints #235

Merged
merged 20 commits into from
Aug 6, 2021

Conversation

ntn-x2
Copy link
Member

@ntn-x2 ntn-x2 commented Aug 5, 2021

fixes KILTProtocol/ticket#1505

This PR modifies the service endpoints of a DID by adding the possibility to specify a hash of the pointed service endpoints document, a content type to allow other entities to correctly parse the document, and up to three URLs to fetch the document from. The logic about what URL to use in case there are more than ones for a given resource are out of the scope of the chain code.

To do so, the PR updates the DidVerificationKeyUpdateAction and makes it more generic into a DidFragmentUpdateAction<FragmentType> that is then used for both verification key updates with DidFragmentUpdateAction<DidVerificationKey and for service endpoint updates with DidFragmentUpdateAction<ServiceEndpoints>, with ServiceEndpoints being:

pub struct ServiceEndpoints {
	pub content_hash: Hash,
	pub urls: Vec<Url>,
	pub content_type: ContentType,
}

I have not migrated this struct to use BoundedVec, but I hope that is fine. We have one more parameter in the trait's Config called MaxEndpointUrlsCount which specifies how many URLs can be included in a DidFragmentUpdateAction for the service endpoints. We decided to go with three which seems like a reasonable number to provide good redundancy to the data.

How to test:

cargo test --all --all-features --all-targets or simply cargo test -p did

Custom types

JS-Types
{
  "AccountInfo": "AccountInfoWithTripleRefCount",
  "Address": "MultiAddress",
  "AmountOf": "i128",
  "Balance": "u128",
  "BlockNumber": "u64",
  "Index": "u64",
  "LookupSource": "MultiAddress",
  "CtypeCreatorOf": "AccountId",
  "CtypeHashOf": "Hash",
  "ClaimHashOf": "Hash",
  "AttesterOf": "AccountId",
  "AttestationDetails": {
      "ctypeHash": "CtypeHashOf",
      "attester": "AttesterOf",
      "delegationId": "Option<DelegationNodeIdOf>",
      "revoked": "bool"
  },
  "Permissions": "u32",
  "DelegationNodeIdOf": "Hash",
  "DelegatorIdOf": "AccountId",
  "DelegateSignatureTypeOf": "Vec<u8>",
  "DelegationRoot": {
      "ctypeHash": "CtypeHashOf",
      "owner": "DelegatorIdOf",
      "revoked": "bool"
  },
  "DelegationNode": {
      "hierarchyRootId": "DelegationNodeIdOf",
      "parent": "Option<DelegationNodeIdOf>",
      "children": "BTreeSet<DelegationNodeIdOf>",
      "details": "DelegationDetails"
  },
  "DelegationDetails": {
      "owner": "DelegatorIdOf",
      "revoked": "bool",
      "permissions": "Permissions"
  },
  "DelegationHierarchyDetails": {
      "ctypeHash": "CtypeHashOf"
  },
  "KeyIdOf": "Hash",
  "DidIdentifierOf": "AccountId",
  "AccountIdentifierOf": "AccountId",
  "BlockNumberOf": "BlockNumber",
  "DidCallableOf": "Call",
  "DidVerificationKey": {
      "_enum": {
          "Ed25519": "[u8; 32]",
          "Sr25519": "[u8; 32]",
          "Ecdsa": "[u8; 33]"
      }
  },
  "DidEncryptionKey": {
      "_enum": {
          "X25519": "[u8; 32]"
      }
  },
  "DidPublicKey": {
      "_enum": {
          "PublicVerificationKey": "DidVerificationKey",
          "PublicEncryptionKey": "DidEncryptionKey"
      }
  },
  "DidVerificationKeyRelationship": {
      "_enum": [
          "Authentication",
          "CapabilityDelegation",
          "CapabilityInvocation",
          "AssertionMethod"
      ]
  },
  "DidSignature": {
      "_enum": {
          "Ed25519": "Ed25519Signature",
          "Sr25519": "Sr25519Signature",
          "Ecdsa": "EcdsaSignature"
      }
  },
  "DidError": {
      "_enum": {
          "StorageError": "StorageError",
          "SignatureError": "SignatureError",
          "UrlError": "UrlError",
          "InputError": "InputError",
          "InternalError": "Null"
      }
  },
  "StorageError": {
      "_enum": {
          "DidAlreadyPresent": "Null",
          "DidNotPresent": "Null",
          "DidKeyNotPresent": "DidVerificationKeyRelationship",
          "VerificationKeyNotPresent": "Null",
          "CurrentlyActiveKey": "Null",
          "MaxTxCounterValue": "Null"
      }
  },
  "SignatureError": {
      "_enum": [
          "InvalidSignatureFormat",
          "InvalidSignature",
          "InvalidNonce"
      ]
  },
  "KeyError": {
      "_enum": [
          "InvalidVerificationKeyFormat",
          "InvalidEncryptionKeyFormat"
      ]
  },
  "UrlError": {
      "_enum": [
          "InvalidUrlEncoding",
          "InvalidUrlScheme"
      ]
  },
  "InputError": {
      "_enum": [
          "MaxKeyAgreementKeysLimitExceeded",
          "MaxVerificationKeysToRemoveLimitExceeded",
          "MaxUrlLengthExceeded"
      ]
  },
  "DidPublicKeyDetails": {
      "key": "DidPublicKey",
      "blockNumber": "BlockNumberOf"
  },
  "DidDetails": {
      "authenticationKey": "KeyIdOf",
      "keyAgreementKeys": "BTreeSet<KeyIdOf>",
      "delegationKey": "Option<KeyIdOf>",
      "attestationKey": "Option<KeyIdOf>",
      "publicKeys": "BTreeMap<KeyIdOf, DidPublicKeyDetails>",
      "serviceEndpoints": "Option<ServiceEndpoints>",
      "lastTxCounter": "u64"
  },
  "DidCreationDetails": {
      "did": "DidIdentifierOf",
      "newKeyAgreementKeys": "BTreeSet<DidEncryptionKey>",
      "newAttestationKey": "Option<DidVerificationKey>",
      "newDelegationKey": "Option<DidVerificationKey>",
      "newServiceEndpoints": "Option<ServiceEndpoints>"
  },
  "DidUpdateDetails": {
      "newAuthenticationKey": "Option<DidVerificationKey>",
      "newKeyAgreementKeys": "BTreeSet<DidEncryptionKey>",
      "attestationKeyUpdate": "DidFragmentUpdateAction<DidVerificationKey>",
      "delegationKeyUpdate": "DidFragmentUpdateAction<DidVerificationKey>",
      "publicKeysToRemove": "BTreeSet<KeyIdOf>",
      "serviceEndpointsUpdate": "DidFragmentUpdateAction<ServiceEndpoints>"
  },
  "ServiceEndpoints": {
      "contentHash": "Hash",
      "urls": "Vec<Url>",
      "contentType": "ContentType"
  },
  "DidFragmentUpdateAction": {
      "_enum": {
          "Ignore": "Null",
          "Change": {
              "_enum": [
                  "DidVerificationKey",
                  "ServiceEndPoints"
              ]
          },
          "Delete": "Null"
      }
  },
  "ContentType": {
      "_enum": [
          "ApplicationJson",
          "ApplicationJsonLd"
      ]
  },
  "DidAuthorizedCallOperation": {
      "did": "DidIdentifierOf",
      "txCounter": "u64",
      "call": "DidCallableOf"
  },
  "HttpUrl": {
      "payload": "Text"
  },
  "FtpUrl": {
      "payload": "Text"
  },
  "IpfsUrl": {
      "payload": "Text"
  },
  "Url": {
      "_enum": {
          "Http": "HttpUrl",
          "Ftp": "FtpUrl",
          "Ipfs": "IpfsUrl"
      }
  },
  "LockedBalance": {
      "block": "BlockNumber",
      "amount": "Balance"
  },
  "BalanceOf": "Balance",
  "RoundInfo": {
      "current": "SessionIndex",
      "first": "BlockNumber",
      "length": "BlockNumber"
  },
  "OrderedSet": "Vec<Stake>",
  "Stake": {
      "owner": "AccountId",
      "amount": "Balance"
  },
  "TotalStake": {
      "collators": "Balance",
      "delegators": "Balance"
  },
  "InflationInfo": {
      "collator": "StakingInfo",
      "delegator": "StakingInfo"
  },
  "StakingInfo": {
      "maxRate": "Perquintill",
      "rewardRate": "RewardRate"
  },
  "RewardRate": {
      "annual": "Perquintill",
      "perBlock": "Perquintill"
  },
  "Delegator": {
      "delegations": "Vec<Stake>",
      "total": "Balance"
  },
  "CollatorSnapshot": {
      "stake": "Balance",
      "delegators": "Vec<Stake>",
      "total": "Balance"
  },
  "Collator": {
      "id": "AccountId",
      "stake": "Balance",
      "delegators": "Vec<Stake>",
      "total": "Balance",
      "state": "CollatorStatus"
  },
  "CollatorStatus": {
      "_enum": {
          "Active": "Null",
          "Leaving": "SessionIndex"
      }
  },
  "DelegationCounter": {
      "round": "SessionIndex",
      "counter": "u32"
  },
  "DidStorageVersion": {
      "_enum": ["V1", "V2"]
  }
}

Checklist:

  • I have verified that the code works
    • No panics! (checked arithmetic ops, no indexing array[3] use get(3), ...)
  • I have verified that the code is easy to understand
  • This PR does not introduce new custom types
  • I have left the code in a better state
  • I have documented the changes (where applicable)

@ntn-x2 ntn-x2 added ✨ new feature feature: new feature Breaking JS Types This PR will break the JS types. labels Aug 5, 2021
@ntn-x2 ntn-x2 self-assigned this Aug 5, 2021
@ntn-x2 ntn-x2 marked this pull request as ready for review August 5, 2021 15:30
@ntn-x2 ntn-x2 requested review from wischli and weichweich August 5, 2021 15:30
Copy link
Contributor

@wischli wischli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would move the URL checks to an implementation for ServiceEndPoints, see below.

I am also unsure, whether the VersionMigratorTrait should be in primitives. It probably should be moved to a new pallet Trait instead. We definitely need to make the migration template more generic and move most of the code to a Trait pallet eventually. Of course, this is out of scope for this PR.

Apart from that, LGTM.

pallets/did/src/did_details.rs Outdated Show resolved Hide resolved
pallets/did/src/lib.rs Show resolved Hide resolved
}
}

impl<T: Config> VersionMigratorTrait<T> for DidStorageVersion {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try to make this version migration more generic in the next week or future. But since we should wait for paritytech/substrate#9165 to be available in a release, it might not make sense to start with this before.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please, I think that would be great as we will eventually need it for other pallets as well. I can also take care of this if you don't manage to do it before you leave.

total_weight
}

fn old_to_new_did_details<T: Config>(old: deprecated::v1::DidDetails<T>) -> DidDetails<T> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we might not even need a migration since we just remove one field and initiate a new one with default None. But better to be safe.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need a migration here in case there are entries with a ServiceEndpointUrl. The new code would try to parse the URL into the new Struct.

///
/// In this way, the migrator can access the pallet's storage and the pallet's
/// types directly.
pub trait VersionMigratorTrait<T> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure whether we want to introduce a new Trait pallet for this instead.

Copy link
Member Author

@ntn-x2 ntn-x2 Aug 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it would make sense. But at the moment kilt_primitives was the only place I could put it in without creating any dependencies between KILT pallets. I guess also the storage migration stuff deserves to be put in a different crate, perhaps the same as the one for traits.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I am fine leaving it be for this PR and then we handle it correctly in another PR.

Copy link
Contributor

@wischli wischli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for resolving the issue around cargo test. LGTM now!

Copy link
Contributor

@weichweich weichweich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

  • The standard error in the benchmarks is suspiciously high.
  • a few opinions on naming

pallets/did/src/default_weights.rs Show resolved Hide resolved
pallets/did/src/did_details.rs Show resolved Hide resolved
pallets/did/src/did_details.rs Show resolved Hide resolved
total_weight
}

fn old_to_new_did_details<T: Config>(old: deprecated::v1::DidDetails<T>) -> DidDetails<T> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need a migration here in case there are entries with a ServiceEndpointUrl. The new code would try to parse the URL into the new Struct.

pallets/did/src/did_details.rs Show resolved Hide resolved
@ntn-x2 ntn-x2 enabled auto-merge (squash) August 6, 2021 14:27
@ntn-x2 ntn-x2 merged commit e3f676f into develop Aug 6, 2021
@ntn-x2 ntn-x2 deleted the aa-did-services-refactor branch August 6, 2021 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Breaking JS Types This PR will break the JS types. ✨ new feature feature: new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants