Should the DID parameters be normative in the spec?

[iherman]

Per current editorial status the are, because it is part of the normative section 4.

However, if so, that means that the four explicit values in the table are also normative, meaning that, probably, the are REQUIRED by conform DID implementations and the way they are used is normatively specified. But that can only be done if all dependencies are also normative. I see two problems:

• hl depends on Hashlink, which is an Internet Draft, itself depending on other drafts.

  Of course, If hl can be driven through IETF as an internet standard by the end of the WG then we are fine. Alternatively, we change the editorial spec by making the necessary part of the hl specification part of this document.

• the text says:

  exact processing rules for these parameters are specified in DID-RESOLUTION

  which is a problem because DID-RESOLUTION is not planned to become a Recommendation (at least in this round).

I do not know whether it makes sense to declare these as informative, though.

(This is related to #53)

[msporny]

@iherman is correct, we need to get this sorted before CR.

[talltree]

Agreed, these were both citations that we were comfortable making at the CCG spec stage because we knew we'd get them figured out at the full WG stage.

[selfissued]

In my view, the working group should make every attempt to not introduce the matrix parameters syntax at all. There are already two mechanisms for passing parameters in URLs - query parameters and fragments. One or the other should suffice in all cases.

Even if it's necessary to do something like dedicate particular query and/or fragment names for DID purposes, that would arguably be preferable to introducing yet a third parameter passing mechanism that requires non-standard URL parsing to use.

[dlongley]

@selfissued, even though it's not as clean an approach, I'm somewhat persuaded by your argument, particularly that there would be no need to update existing URL parsers. It's also the case that servers that would be referenced via the service redirection feature would need to have some awareness that they are being used that way in order to have a stable path/query structure with similar servers.

[selfissued]

I'll note that on the 17-Dec-19 working group call, it was clear that there wasn't agreement on whether matrix parameters COULD/SHOULD/MAY/MUST/MUST NOT change the DID Subject. Unless the overall semantics are crystal clear, we should steer clear of using matrix parameters. This comment is pertinent to #144 as well.

[peacekeeper]

@selfissued

it was clear that there wasn't agreement on whether matrix parameters COULD/SHOULD/MAY/MUST/MUST NOT change the DID Subject

I think you may have misunderstood something. Nobody is saying that matrix parameters or anything else in a DID URL changes the DID Subject. All we said was that different URLs may dereference to different resources, which is simply how all URIs work.

[peacekeeper]

@dlongley

It's also the case that servers that would be referenced via the service redirection feature would need to have some awareness that they are being used that way in order to have a stable path/query structure with similar servers.

Could you elaborate a bit? I don't follow.. The whole point of the service redirection use case is to be able to have persistent URLs that can point to changing (unaware) servers with changing path/query structures.

E.g. did:ex:123;service=image could point to https://socialnetwork.com/user/123/profile?image today and could be changed to point to https://personalcloud.com/mypic.jpg tomorrow, and neither of the servers need to be aware of that.

@selfissued

Even if it's necessary to do something like dedicate particular query and/or fragment names for DID purposes, that would arguably be preferable

I suppose dedicated query parameters could be used instead of matrix parameters, e.g. did:ex:123?_did_service=image instead of did:ex:123;service=image, but then the same construct (query parameters) would be applied during different steps of the dereferencing process, and I worry that might break some things.

[iherman]

@peacekeeper,

I suppose dedicated query parameters could be used instead of matrix parameters, e.g. did:ex:123?_did_service=image instead of did:ex:123;service=image, but then the same construct (query parameters) would be applied during different steps of the dereferencing process, and I worry that might break some things.

could you elaborate on this?

[peacekeeper]

@iherman

I worry that might break some things.

could you elaborate on this?

Well, normally in URL dereferencing, each part of a URL is applied at a certain stage of the dereferencing process. E.g. in an HTTP URL, the "host" is used to look up the IP address of the server, the "port" is used to connect to that server, the "path" and "query" are used to identify a certain resource under the HTTP server's authority, and the "fragment" is used by the HTTP client used to identify a secondary resource that is part of or related to the primary one.

If we decide to use reserved query parameters instead of matrix parameters in DID URLs, then e.g. we might have DID URLs like this:

did:ex:123/photo?_did_service=profile&width=200&height=80

Then the /photo part as well as the width=200 and height=80 parts would be used after service selection, but the _did_service=profile part would have to be extracted during a much earlier stage of the DID URL dereferencing process (service selection). This is very unintuitive, since the path would actually be processed AFTER one of the query parameters, even though it appears first in the DID URL syntax.

Suggesting this would be a bit (but admittedly not quite) like suggesting to add DNS query parameters to HTTP URLs; this makes no sense, because DNS and HTTP are different layers.

Therefore I believe it is much cleaner and more secure to use matrix parameter which are applied on a different layer and don't interfere with query parameters; the equivalent to the above example would be as follows:

did:ex:123;service=profile/photo?width=200&height=80

[TallTed]

@peacekeeper - I'm just wondering if this might not serve as well as your matrix parameter --

did:ex:123?_did_service=profile&path=/photo&width=200&height=80

(The path query parameter could obviously be changed to element or part or whatever else might be preferred, as path is probably better considered to be overloaded and thus not used here ... but it is what you're specifying, so that seemed the clearest param name for the example.)

[iherman]

@peacekeeper thanks, I understand what you mean when you refer to HTTP land.

However, at least for me, it still remains a bit vague. What I do not really understand (and I do not think it is part of the current core document) is a clear model of what happens where, who does what, what is the responsibility of the various actors. In HTTP land we have a pretty good idea about client, server, DNS, HTTP messages, etc. Is there a clear model of the similar actors in DID land? Would it help to have a clear written model (a diagram?) that we all agree on for what steps should happen by which actor? Once we all agree in this, we may have a clearer description of which part of the URI is addressed by which actor and how we must separate the parameters syntactically.

Until then I am afraid that we are all influenced by what is happening in HTTP land, which may be the source of confusion.

WDYT?

[peacekeeper]

@TallTed

did:ex:123?_did_service=profile&path=/photo&width=200&height=80

This doesn't make much sense to me, since you would be inventing a new query parameter for a path, when path is already a standard construct in a URI?

[peacekeeper]

@iherman

However, at least for me, it still remains a bit vague. What I do not really understand (and I do not think it is part of the current core document) is a clear model of what happens where, who does what, what is the responsibility of the various actors. In HTTP land we have a pretty good idea about client, server, DNS, HTTP messages, etc. Is there a clear model of the similar actors in DID land? Would it help to have a clear written model (a diagram?) that we all agree on for what steps should happen by which actor?

This is where DID resolution and DID URL dereferencing come in, which are the topic of the work we have been doing for many months in the DID Resolution specification.

The most important point to realize is that DID resolution is not a client-server protocol (like HTTP). It is defined as an abstract function that can be invoked over different bindings. Different parts of the DID URL dereferencing process can then happen in different actors/components, e.g. resolving a DID to a DID document could happen in a hosted resolver service, and applying the fragment could happen in a client. But there are also other possible arrangements.

This DID resoution work is (currently) in the CCG, not in the WG, but 1. there is a discussion whether it should be moved into the WG since the topics are so closely related, and 2. even if it remains outside the WG, I believe it would still be valuable for WG members to follow that work in order to better understand DID resolution, matrix parameters, and other topics.

[iherman]

@peacekeeper right. I still have to read the DID resolution document, but it strikes me that until we have a common understanding (everyone, including the many new kids on the block:) the discussion on matrix parameters does not really make sense: it remains at a purely syntactic level with arguments coming from the HTTP world (does the term 'query parameter' make sense in the etymological sense in the DID world, for example)?

[iherman]

@peacekeeper,

thanks again for pointing me to the DID Resolution specification. I also understand now your earlier comment in #137 (comment):

did:ex:123/photo?_did_service=profile&width=200&height=80

Then the /photo part as well as the width=200 and height=80 parts would be used after service selection, but the _did_service=profile part would have to be extracted during a much earlier stage of the DID URL dereferencing process (service selection). This is very unintuitive, since the path would actually be processed AFTER one of the query parameters, even though it appears first in the DID URL syntax.

Two comments on this.

1. Yes, that URL a bit un-intuitive, but it can be specified. The question is how the theoretical purity weights against the practical problem that, if we use matrix parameters, all off-the-shelf URL handling libraries out there become unusable.

   Also, looking at the DID Resolution specification, and considering the analogy of HTTP, a DID Resolver pretty much plays the role of a server in HTTP land (from the URI handling point of view, that is). Query parameters are handled in an HTTP server, i.e., are handled in a DID resolver; it is up to separate specifications to define what the query parameters mean. So again, if it can be defined then, well, it is ok.

2. I also realized that I do have a possibly more fundamental problem; forgive me if it just reflects the limitation of my knowledge. Yes, the URI spec defines query parameters, paths, and fragments. Because a DID, as well as a DID URL, is a URI, they “inherit” those. However (matrix parameters put aside for a moment) I have not yet seen really convincing examples for the necessary usage of general query parameters or paths.

   My fundamental mental model is that resolving of a DID URL returns a DID Document, and this DID Document may have a bunch of other pointers, information, etc., about the DID subject that the client can use. That is it. If the client needs to get hold of the photo of the DID subject, then there should be a reference to that photo in the DID document (the reference can be a different DID, or course). What your example does (again, matrix parameters pushed aside) is some sort of a practical shortcut: I have a DID URL, but I want is the photo of the DID subject and not the DID Document that is intimately bound to the DID. For me, this shortcut distorts the very simple model of DID URL -> DID Document.

   What if we simplify things having three rules:

   1. DID URLs may include the DID, query parameters and fragments
   2. Query parameters are used exclusively for what matrix parameters are used now, i.e., influencing the DID resolver on “which” DID Document is returned (e.g., via version id)
   3. Fragments are used to access parts of information (on the “client” side) within the DID Document

   and that is it. Would that throw out lots of use cases? Would that create huge difficulties in implementations?

[peacekeeper]

@iherman

Yes, that URL a bit un-intuitive, but it can be specified.

Actually there are cases when this is not just un-intuitive, but actually won't work at all. The idea of "reserved query parameters" assumes that a query string in a DID URL must always be in the form of key-value pairs, which in generic URI syntax is often the case but not required. E.g. if I wanted to construct a DID URL like did:ex:123/mypath?myquery#myfragment, then as an app developer I should have the freedom to do so, but since I chose to not use a key-value query string, I now have no way to add a "reserved query parameter" such as _did_service.

all off-the-shelf URL handling libraries out there become unusable

I disagree with the entire class of arguments that matrix parameters introduce a new requirement to use non-standard parsing, since depending on the perspective, you could either argue that

1. DID URLs are completely standard anyway (whether or not we have matrix paraemters), since they are compliant with generic URI syntax, or
2. DID URLs require custom parsing anyway (whether or not we have matrix parameters), since they already have a custom syntax for the "authority" component anyway, including the "non-standard" method-name and method-specific-id components.

Therefore while yes admittedly matrix parameters introduce additional syntax (which I agree should be carefully considered against the use cases enabled by that syntax), I don't think they change whether or not DID URLs are "standard".

I have a DID URL, but I want is the photo of the DID subject and not the DID Document that is intimately bound to the DID. For me, this shortcut distorts the very simple model of DID URL -> DID Document.

The idea of DID URLs is to open up an information space rooted in a DID, just like HTTP URLs enable information spaces rooted in a domain name. A DID URL that is just a DID itself is dereferenced to the DID document (the most common case), but a DID URL with other syntax components like matrix parameters, path, or query, can potentially be dereferenced to any other resource. If we think we don't want to enable this, and all we want is an identifier that is dereferenced to a DID document and nothing else, then we don't need DID URLs at all. We only need DIDs. Do you think we should go in that direction?

What if we simplify things having three rules ... Would that throw out lots of use cases?

Yes it would break (or restrict) the service endpoint indirection use case (see above), and it would break (or restrict) the basic assumption that path, query, and fragment should be left completely available for application or method developers and not be constrained by the DID spec, just like the HTTP spec doesn't constrain a web developer how they want to use path, query, and fragment.

[TallTed]

This doesn't make much sense to me, since you would be inventing a new query parameter for a path, when path is already a standard construct in a URI?

Yes... So? We're already mixing "URI uses/structures that exist" with "URI uses/structures that don't exist".

Your complaint was that the sequence of action was not strictly left-to-right on the components of your URI --

did:ex:123/photo?_did_service=profile&width=200&height=80

-- which you suggested would be best resolved with a complex matrix parameter --

did:ex:123;service=profile/photo?width=200&height=80

My URI makes the action sequence strictly left-to-right and removes the semicolon -- i.e., there's no matrix parameter involved or required, which has been the main question of this issue. I've changed path to element below, because that was (as I said initially) an arbitrary parameter name

did:ex:123?_did_service=profile&element=/photo&width=200&height=80

[peacekeeper]

@TallTed I'm sorry but I didn't "complain" about anything, and I am not suggesting that anything needs to be "resolved". And I understand matrix parameters may seem "complex" when first learning about them, but I think the alternatives are either to drop certain use cases or to implement them in even more complex ways than with matrix parameters.

did:ex:123?_did_service=profile&element=/photo&width=200&height=80

I believe this is more "complex" than the matrix parameter approach, since the reserved query parameter _did_service would have to be extracted at a certain stage, and the element query parameter would also have to be handled using some custom processing logic. And I furthermore think this would actually limit the use case, since it would restrict DID URL query strings to a key=value form when in general query strings don't have this restriction.

But I do think it would be worthwhile to compare this example to the matrix parameter example and see how they would each work in detail.

How would your example URI be dereferenced, and what would be the result of that dereferencing process?

[TallTed]

@peacekeeper -

My example URI --

did:ex:123?_did_service=profile&element=/photo&width=200&height=80

-- would appear to break down into --

scheme	authority (DID Method)	path (DID ID)	query
did	ex	123	_did_service=profile&element=/photo&width=200&height=80

The DID method tells the dereferencer where/how to get the query processed. The query processor takes the query parameters, and delivers a representation of the identified resource -- presumably being a 200x80 scaling of the photo found in the profile of the DID subject... but I'm guessing at much of this, because there's no requirement that query parameters be any less opaque than URIs in general.

Now... If I'm looking at this all wrong, it seems to me that this split of DID Resolution (which decides how to resolve / dereference a DID URI) from DID Core (which decides how to structure a DID URI) may have been premature -- or at the least, that there should be much more frequent coordination between these two.

[peacekeeper]

@TallTed thanks for the additional thoughts on query processing. Also, big +1 to your last paragraph. The gap between DID Core and DID Resolution is probably the reason why it's really hard to discuss matrix parameters (or alternatives to them) as well as other topics, such as the one on metadata (#65).

[peacekeeper]

Dear all, here are some materials about why matrix parameters may be useful in DID URLs:

• Slides, log, and recording from last week's DID Resolution spec weekly meeting where we discussed matrix parameters:
  https://github.com/w3c-ccg/meetings/tree/gh-pages/2020-01-16-did-resolution
• My topic paper for Rebooting-the-Web-of-Trust 10:
  https://github.com/WebOfTrustInfo/rwot10-buenosaires/blob/master/topics-and-advance-readings/why-matrix-parameters.md

Sorry for cross-posting to this issue and the DID WG mailing list.

[philarcher]

I suggest this issue be closed as the discussion is playing out in issue #159 (with some repetition)

[OR13]

matrix params should be removed from the spec entirely, and then query params that are actually used should be added.... the spec should not define a bunch of hypothetical ways of doing things that nobody is using. agree with @philarcher ... this should be closed, in favor of #159

[peacekeeper]

@philarcher @OR13: #159 is about the syntax of DID parameters, whereas this is about whether the four concrete DID parameters in the spec are normative or not, so I believe this issue here is independent. I'm going to rename it to make that clearer.

[peacekeeper]

The following PR will add references for the DID Core parameters into the DID Spec Registries: w3c/did-extensions#61

[kdenhartog]

In that case, I think once they've been registered (and considering they're already in the spec) we've reached the conclusion that this issue can be closed. Is that correct @peacekeeper?

[msporny]

The current spec has been updated to carefully state normative assertions around DID Parameters such that the language should survive through to REC. We need folks in the group to review the new language and nitpick things they feel are not appropriate normative requirements (or cite new normative requirements that need to be made).

[peacekeeper]

There have been a few comments in this issue suggesting to close it. Looking back at the original two comments raised by @iherman:

1. The hl parameter references the Hashlink Internet Draft.

2. The text references [DID-RESOLUTION], which is not a Recommendation.

The reference to [DID-RESOLUTION] is informative, not normative, and the language around it has been weakened since this issue was created. Instead of

exact processing rules for these parameters are specified in [DID-RESOLUTION]

it currently says

Additional considerations for processing these parameters are discussed in [DID-RESOLUTION]

Overall, I think our approach is that we normatively define the DID parameter names, their meanings, and their data types, but we don't define their exact allowed values and processing rules, this happens elsewhere. Just like with DID Resolution, in DID Core we only specify the interfaces, but not the algorithms to implement them.

I think this is fine now the way it is? Or do we still have a problem? @iherman ?

[iherman]

I really do not want to sound difficult, but... while I agree with @peacekeeper that the reference to the DID-RESOLUTION is fine, and the table gives an adequate definition for service, version id and time, I still have a problem with hl. The reference is to an internet draft, dated 208 and actually expired a little more than a year ago.

Our document does not specify the behaviour of hl at all, meaning that the implementer must rely on the IETF document. Can we be sure that the document is stable? (Look at a normative references guidelines.) Can we be sure that implementers implementing hl will not have to change their implementation in a few years? We simply cannot be sure of that.

[peacekeeper]

@iherman I understand your concerns with hl, it clearly has a strong dependency on the Hashlink spec. In the worst case, we could remove it from DID Core, define it elsewhere, but still list it in the DID Spec Registries just like we have done it with other DID parameters such as initial-state and transform-keys, right?

[iherman]

@peacekeeper, correct. We can actually even leave in the core spec but very clearly mark it as non-normative.

[peacekeeper]

On today's DID WG call, there was a proposal to address this issue by marking hl as non-normative, and to potentially remove that comment again at a future time when we are able to make a normative reference to Hashlink.

[peacekeeper]

I believe this can be closed now that #378 has been merged, @iherman do you agree?

[iherman]

I believe this can be closed now that #378 has been merged, @iherman do you agree?

Yes.

[peacekeeper]

Closing since the problems mentioned in this issue have been fixed.