IETF did+ld+json media type registration

[msporny]

We have received a response from IANA wrt. did+ld+json as a media type:

The [IANA] reviewers have identified issues with the following statement in the optional [JSON-LD 1.1] media type parameters section [which DID Core has a dependency on]:

"Other specifications MAY create further structured subtypes by using +ld+json as a suffix for a new base subtype, as in application/example+ld+json. Unless defined otherwise, such subtypes use the same fragment identifier behavior as application/ld+json."

This paragraph effectively creates a new suffix called "+ld+json". The issues:

a) the suffix would need to be registered at https://www.iana.org/assignments/media-type-structured-suffix;

b) if the suffix registration request is submitted, it is likely to be denied at this time, because this requires a larger discussion in IETF;

c) the easiest way to approve the media type registration is to remove the above paragraph from the registration, then initiate discussion in IETF about nested suffixes (which one of the reviewers can take on). If the discussion results in "nested suffixes are OK", this registration can be updated to include the above paragraph.

The DID WG now needs to decide if we should pursue the +ld+json suffix at IETF (which is the right thing to do), or we just fall back to did+jsonld and state that the media type is identical to ld+json (which is a hack to get around the fact that IETF hasn't ruled on this particular thing).

Exciting to be on the bleeding edge, isn't it?! :)

[iherman]

The DID WG now needs to decide if we should pursue the +ld+json suffix at IETF (which is the right thing to do),

there is almost zero chance this would go through within a reasonable amount of time.

• the JSON-LD WG will probably remove the relevant entry in the media type section to get it through. Ie, we cannot rely on that suffix (otherwise the WG cannot finish its work on time). (See the (Comment out +ld+json mime extensions. json-ld-syntax#331 PR.)
• the DID WG shoiuld not submit the ld+json suffix; it does not 'control' the JSON-LD specification, ie, it would not be appropriate to do so.
• The DID WG has the possibility to submit a did+ld+json of course but that would create a precedence at IETF; based on the remark above ("it is likely to be denied at this time, because this requires a larger discussion in IETF") the chances to pass this is close to zero.

or we just fall back to did+jsonld and state that the media type is identical to ld+json (which is a hack to get around the fact that IETF hasn't ruled on this particular thing).

I do not think that would work either: the application/jsonld does not exist (see the current IANA list), ie, it is not a valid suffix. The DID WG would have to submit the application/jsonld media type first but, just as above, it would not be appropriate to submit a media type whose technical specification it does not control.

Sigh...

The third alternative, which is valid and easy: define a profile for the ld+json media type, see the json-ld iana section. The downside is, of course, that the media type is ugly and verbose:

application/ld+json;profile=http://www.w3.org/ns/did-profile

(I have just made up the URL for the profile; that would need some bikeshedding.)

in spite of its ugliness this is, In my view, the only valid, easy to administer solution. As for being ugly: I do not think these media types will be set by average users, only by server administrators and application developers. In other words, this media type will be machine generated most of the time, ie, it does not really matter.

[rhiaro]

For the record, ActivityStreams 2.0 ended up with application/activity+json. The ld+json and a profile was considered and rejected. AFAIK nobody liked it then and nobody likes it now. People running generic JSON-LD libraries were unwilling to update anything to consider activity+json 'equiavlent' to ld+json, but we also had to do the 'implied context' thing to go along with it which didn't help. Maybe we'd have better luck with DIDs. Doing it with ld+json and a profile seems more correct to me than did+json.

[msporny]

nobody liked it then and nobody likes it now.

Yeah, I'm a -1 to the profile approach... it's not something that developers are used to doing wrt. media types. I'd rather we define an entirely new media type: application/did+jsonld and then state that the processing rules for it are the same as the JSON-LD media type (if IANA says that we can't do application/did+ld+json).

[rhiaro]

Sorry @manu I edited my comment after you saw it I think because I misread Ivan's. AS2 didn't use profile in the end, but a custom +json media type, which also nobody liked.

[msporny]

I edited my comment after you saw it I think because I misread Ivan's. AS2 didn't use profile in the end, but a custom +json media type, which also nobody liked.

Haha, great, so we have two distasteful options to choose from. :)

Has the ActivityStreams community signaled what they would have liked to see instead?

[rhiaro]

I should look stuff up instead of relying on my memory. AS2 actually uses application/activity+json as the media type and say that implementers SHOULD treat application/ld+json; profile="http://www.w3.org/ns/activitystreams#" as equivalent.

I can't and shouldn't try to speak for the ActivityStreams community, but actually I think most of the frustration with the custom media type comes from people having to customise JSON-LD tools to parse it as JSON-LD, but anyone treating it as JSON-only I suspect is fine with it. To be honest I'd be quite surprised if JSON-only implementations were not igorning ld+json though. But I could be wrong!

[BigBlueHat]

@rhiaro iirc the custom media type was needed because JSON-LD 1.0 doesn't support "lists of lists" (which has thankfully been fixed in JSON-LD 1.1!!).

So, developer can (and should) treat the custom media type and the profile'd one as equivalent in so far as that contained JSON doesn't include GeoJSON...which uses lists of lists...

An upgrade to JSON-LD 1.1 would fix that, though. 😸

[BigBlueHat]

Here's @jasnell's comment about the compromise:
w3c/activitystreams#52 (comment)
which ultimately became a pull request:
w3c/activitystreams#243

[msporny]

I think most of the frustration with the custom media type comes from people having to customise JSON-LD tools to parse it as JSON-LD, but anyone treating it as JSON-only I suspect is fine with it.

Hmm... so if the JSON-LD community has to go through pain for this, that's probably ok (because that community is used to going through pain to address JSON-only developer concerns). So, this is good news.

We should also see if we can convince JSON-LD processor implementers to treat other known media types as JSON-LD. So all these media types would be processed as "application/ld+json": "application/did+ld+json", "application/activity+json", etc. /cc @davidlehn @dlongley @gkellogg

We might do that via a media type hook that fires and does pre-processing on the input document (such as injecting a known @context if one doesn't exist)?

[peacekeeper]

Just for reference, this was discussed during a meeting with the JSON-LD WG (see minutes).

[peacekeeper]

I wonder if IANA is only against considering +ld+json as a (nested) suffix, or if they are against application/did+ld+json in general. We can argue that +json is the suffix, and that the first plus in did+ld+json is opaque.

Could this be a temporary hack until perhaps later nested suffixes can be officially introduced?

[msporny]

Could this be a temporary hack until perhaps later nested suffixes can be officially introduced?

It's legal per the rules, AFAIK. Whether or not IANA would approve it is another matter. I think we have data to say that they wouldn't approve it. :)

In any case, we should ask as soon as possible.

[peacekeeper]

So to summarize..

1. We can't use application/did+jsonld, because +jsonld isn't a registered suffix.
2. We don't want to use profiles.
3. We (probably) can't use application/did+ld+json, or if we can use it, then the suffix would be only the +json part, not the +ld+json part.
4. We (probably) can't use application/did+json, because that will probably be used by the "pure JSON" representation.

[jasnell]

ah... always fun to see long standing issues still as yet unresolved ;-) ... This all, of course, brings back up all the issues why the current media-type mechanism is broken in some fundamental ways but none of that would help resolve the more immediate issue here. One approach you could take is simple application/didld+json or application/lddid+json. It's not ideal, but it would work and fits within what IANA would accept. It would mean, of course, that "We should also see if we can convince JSON-LD processor implementers to treat other known media types as JSON-LD" would be required. @rhiaro describes the history of the Activity Streams 2.0 discussions well :-)

[msporny]

I prefer the option where we push IANA to fix Media Types on the Internet and allow "application/did+ld+json".

The worst that could possibly happen is they say no and we implement one of the many hacks available to us. We have ~9 months to get a response from IANA, let's see if we can get them to do the right thing.

I think the next step here is to have @burnburn, @brentzundel, and @iherman make the request to IANA to look into this issue on behalf of the DID WG (since it's too late for the JSON-LD WG to do anything about it in this iteration of that standard).

[jasnell]

Fwiw, I'd be very much in favor of an IETF draft that added json/* as a new top level media type. e.g. json/ld, json/did, json/did-ld, json/activity, etc ... it would be in line with other top level types like multipart/* and would address a large number of these kinds of issues. But again, that's just me shouting from the sidelines :-)

[davidlehn]

@msporny Adding some sort of support to JSON-LD processors for other known media types sounds possible and reasonable. Perhaps various use cases (from here and elsewhere) and recommendations for implementations could go in the best practices document.

How flexible are people expecting clients and servers to be? Will servers also support Accept: application/ld+json? Or are people expecting endpoints to special case the DID media type differently than other JSON types? Based on our experience with application/ld+json, I imagine clients and servers will have to handle whatever application/did+ld+json turns into, application/ld+json, and application/json (And then json/* too? Getting complex.) Too many tools will just ask for application/json or just send application/json by default. If tools expect that behavior, then everything just fuzzy works anyway regardless of this media type discussion.

[msporny]

Fwiw, I'd be very much in favor of an IETF draft that added json/* as a new top level media type

That's really interesting, I hadn't considered that as an option before, makes sense, @jasnell, you rebel, you. :)

If tools expect that behavior, then everything just fuzzy works anyway regardless of this media type discussion.

Except for the fact that we just merged something into DID Core today that @jricher wrote that does not allow you to do "fuzzy works" solutions - no content sniffing or other fuzzy detection of content.

[iherman]

I think the next step here is to have @burnburn, @brentzundel, and @iherman make the request to IANA to look into this issue on behalf of the DID WG (since it's too late for the JSON-LD WG to do anything about it in this iteration of that standard).

In a parallel universe Working Group, namely the JSON-LD one, we have just discussed this as well. The JSON-LD WG will have to make a decision on removing the mention of xyz+ld+json in the media type, otherwise we cannot publish a Rec. I expect this decision to happen this week (our weekly call is on Fridays). But we also agreed that when I answer to my contact at IANA I will ask her for her advice on what the best way is to start the general discussion on application/A+B+C media types. (@peacekeeper, the answer I got was definitely not aimed at ld+json; it was referring to the fact that this should be discussed in general before agreeing with it).

I will probably send the reply this weekend, after our JSON-LD call. We may have an answer next week, and we can plan accordingly. We are not in an extreme urgency.

[iherman]

Just for info: I did ask my contacts at IANA on what the best way forward is for this issue, and she has forwarded my question to an (unnamed) IANA expert. The ball is in their courtyard on the matter.

[msporny]

We need to make a call here so we can get something consistent in the specification. I'm going to start enforcing the following logically consistent layout in the specification (I think this is what we would want in the ideal case):

• application/did+json (JSON-only DID Documents)
• application/did+ld+json (JSON-LD DID Documents)
• application/did+cbor (CBOR-only DID Documents)
• application/did+ld+cbor (CBOR-LD DID Documents)... and so on

This will give us a consistent IANA naming convention and push the issue at IANA. If they come back and tell us we can't do that, well, we can cross that bridge when we come to it. Changing it in the spec won't be difficult, it's the getting consensus at IETF/IANA/W3C that's going to be the challenge.

[iherman]

@msporny we can do that of course. Our submission to IANA for the registration of the mime types would happen at CR phase, which is still several months away, so having these in the current document as a, as you say, ideal case is fine.

However, I have a problem with the fourth one, ie, application/did+ld+cbor. This would rely on a application/ld+cbor media type but, alas!, the JSON-LD WG did not get enough inputs for the creation of a CBOR-LD note, and hence the registration of such a media type. That WG is now running out of time, so it is too late. I would think we should drop that one for now, and stick with the first three that reflect the serialization formats we agreed we would put into the document...

If the CBOR+LD happens at some point (eg, via the JSON-LD CG + the maintenance WG), we can add this later.

[BigBlueHat]

If the CBOR+LD happens at some point (eg, via the JSON-LD CG + the maintenance WG), we can add this later.

That's certainly something we could (and should 😉) pursue in the JSON-LD CG:
https://lists.w3.org/Archives/Public/public-linked-json/2020May/0009.html

[msporny]

We need to reach out to IANA to see what the status of this is...

[msporny]

... where "we" is @iherman @msporny @burnburn and @brentzundel. Ivan, who is the contact at IANA that said they'd look at this?

[iherman]

I have sent out a mail asking about the status, sorry I missed the question two weeks ago.

(The contact address is simply [email protected], b.t.w., but it is, I believe, better if I continue to be the go-between. Just do not kill the messenger:-)

[rhiaro]

I've added PR for the at risk language for did+ld+json.

I would like to see something about using regular ld+json I think... seems odd that we'd say JSON-LD DID docs can't be recognised by consumers if they come in as application/ld+json. Plus generic JSON-LD systems that might find themselves handed a DID doc with application/did+jsonld are liable to ignore it as unrecognisable when it's perfectly valid JSON-LD that they could parse.

[iherman]

I would suggest we had a separate topic call at some point to consider the media type issue and what our fallbacks are. I am not convinced about the application/jsonld route, for example.

@msporny @brentzundel @burnburn

[msporny]

This issue continues to be moved forward and discussed here:

https://mailarchive.ietf.org/arch/msg/media-types/CdoZdUVL_RMTYICkiJ7dcis2D8k/
https://mailarchive.ietf.org/arch/msg/media-types/W3T6OlQzMrjTUr1Q_qLrSN0iUjc/
https://mailarchive.ietf.org/arch/msg/media-types/Twhf5ni_UQdyol1FdUxVpst6Hsk/

[msporny]

@iherman wrote:

I am not convinced about the application/jsonld route, for example.

Agreed... I think it might be useful to get two things sorted:

1. Consensus on the WG on the only sane path that we can commit to.
2. Opinion from W3C Director on what we can do if the timing doesn't line up.

That is to say, the only sane path forward is to register "application/did+ld+json" for the JSON-LD representation. The only thing that seems like it would be a challenge at this point is the timing of the media type suffix clarification I-D becoming an IETF RFC. We are not getting strong push back on it, and we can note in the specification that the registration will happen only when the IETF Media Type folks are happy with the extension (and then publish an update to the REC when it happens using the maintenance WG). If it doesn't happen in a timely manner, then it just hangs out there until it does happen.

@iherman, I believe that we've done similar things in previous WGs -- you just effectively say: "This isn't normative yet, but is expected to become normative when X happens, and at that point, the REC will be updated." So, we manage expectations on a particular feature by marking it as "at risk", saying it's non-normative for now, and is expected to become normative at some time in the future.

[iherman]

That is to say, the only sane path forward is to register "application/did+ld+json" for the JSON-LD representation. The only thing that seems like it would be a challenge at this point is the timing of the media type suffix clarification I-D becoming an IETF RFC. We are not getting strong push back on it, and we can note in the specification that the registration will happen only when the IETF Media Type folks are happy with the extension (and then publish an update to the REC when it happens using the maintenance WG). If it doesn't happen in a timely manner, then it just hangs out there until it does happen.

While I fully agree that application/did+ld+json is the sane way to go, I admire your optimism :-)

Once we have a consensus on the approach I will have to ask our IETF experts within the team; I am certainly not one of them!

[rhiaro]

I've updated the draft again based on feedback on the ietf mailing list.

If we can't get application/did+ld+json, I think just using application/ld+json is more sensible and less confusing/disruptive than application/did+jsonld. JSON-LD consumers that are looking for DID docs in particular can soon figure out it's a DID doc with did:... in the @id field.. (that's not content sniffing to determine the representation, as you already know it's JSON-LD..)

[msporny]

I think just using application/ld+json is more sensible and less confusing/disruptive than application/did+jsonld.

Agree.

that's not content sniffing to determine the representation, as you already know it's JSON-LD

Also agree, and that you know it's JSON-LD, @context and type can get you the rest of the way to knowing exactly what type of data you're dealing with. There was a certain sub-group of standards folks that were up in arms about this notion years ago in the JSON-LD 1.0 days... I can't remember what their argument was at the time.

[TallTed]

@rhiaro --

Your multiple-suffix draft RFC looks great. I have one suggestion, which may be contradict feedback you've already received, in which case my suggestion should probably be ignored.

That suggestion is to reverse the order of the subtypes and media types listed in 2.1, such that the most-refined (and most-preferred) come first, and the least-refined (and least-preferred) come last.

(Some text about that order of preference could be added to the descriptive text whether or not the current ordering is changed as I suggest here.)

[iherman]

If we can't get application/did+ld+json, I think just using application/ld+json is more sensible and less confusing/disruptive than application/did+jsonld.

Fully agree.

JSON-LD consumers that are looking for DID docs in particular can soon figure out it's a DID doc with did:... in the @id field.. (that's not content sniffing to determine the representation, as you already know it's JSON-LD..)

Just an additional data point that we should not forget about: the JSON-LD WG discussed this issue back in the days, but that WG did not have the time, nor really the possibility to introduce a radical change. However, the JSON-LD 1.1. IANA Considerations section already includes a series of predefined profile parameters for JSON-LD use. That section also includes the statement:

Other specifications may publish additional profile parameter URIs with their own defined semantics. This includes the ability to associate a file extension with a profile parameter.

I know that using such profile URI-s is a suboptimal solution at best, and I am not advocating it as the solution. But, nevertheless, we may want to consider the route of using application/ld+json with a profile value of, say https://www.w3.org/ns/did/json-ld-profile as a fallback position, in case we cannot use the application/did-ld-json.

[rhiaro]

That suggestion is to reverse the order of the subtypes and media types listed in 2.1, such that the most-refined (and most-preferred) come first, and the least-refined (and least-preferred) come last.

by "first" do you mean left-most? If so I think what you're asking is already the case. If you mean "first" from right to left, then I think it's too late to change that given ld+json is already registered.

I like the "most/least-preferred" language though. I'll see if I can work that in.

[TallTed]

@rhiaro - By "reverse the order", I mean, change this presentation --

json
ld+json
did+ld+json

-- to this --

did+ld+json
ld+json
json

-- and change this presentation --

application/json
application/ld+json
application/did+ld+json

-- to this --

application/did+ld+json
application/ld+json
application/json

In other words, not to change any of the specific texts (not changing anything about left-most vs right-most); just to change the vertical presentation order of the three items in each of these lists. Top is most refined media type and most preferred interpretation; bottom is least refined media type and least preferred interpretation.

[rhiaro]

I mean, change this presentation

ohhh sure, that works for me!

[TallTed]

I think all of the above means that, regardless of IANA/IETF, we need to change the current "at risk" declarations (1, 2), which speak of the undesirable for several reasons application/did+jsonld as our fallback.

[msporny]

Created PR #524 to modify the at risk text by using @iherman's updated alternative.

[jonnycrunch]

I'm curious as to why this work is prioritizing only one representation, i.e. json and not did in general? This would also affect did+dag+cbor as well.

[rhiaro]

@jonnycrunch the IETF spec I drafted solves the general problem of media types with more than one suffix (aka more than one + sign). We don't have this problem for did+json or did+cbor because they have only one + so are a normal case. There is no "did in general" because did must always be attached to another suffix for a specific representation. This work would simultaneously solve did+ld+json and did+dag+cbor. The reason the discussion was oriented around did+ld+json is because that is one of the core representations in the spec, alongside did+json and did+cbor.

[TallTed]

@rhiaro -- It appears that updating the draft from 00 to 01 (or maybe 02) awaits some confirmation action on your part --

Date	Rev.	By	Action
2020-12-29	01	(System)	Request for posting confirmation emailed to previous authors: Amy Guy [email protected]
2020-12-29	01	Amy Guy	Uploaded new revision
2020-12-28	01	(System)	Request for posting confirmation emailed to previous authors: Amy Guy [email protected]
2020-12-28	01	Amy Guy	Uploaded new revision
2020-10-24	00	Amy Guy	New version available: draft-w3cdidwg-media-types-with-multiple-suffixes-00.txt
2020-10-24	00	(System)	New version approved
2020-10-24	00	Amy Guy	Request for posting confirmation emailed  to submitter and authors: Amy Guy [email protected], Amy Guy [email protected]
2020-10-24	00	Amy Guy	Uploaded new revision

[rhiaro]

@TallTed yes, but I'm not getting any of the confirmation emails from the IETF system. Trying to figure out how to debug..

[TallTed]

@rhiaro - Ah! Bugs do bite everywhere...

Something seems to have changed, as the -01 version (now dated today) is now live on the status page.

@jonnycrunch - You may want to take a look at the above.

[msporny]

PR #524 has been merged addressing the original issue and sub-issues. Closing.