Document the semantics of our XML schema

[wking]

Spun off from #391, which has some links to current external-to-this-repo docs. I think we want to bring those docs into this repo so we only have one location to handle as we iterate on the schema. One thing that could use more detailed docs are the semantics of <alt> attributes, like the regexp dialect used by match and whether or not name needs to be unique within a license.

[goneall]

Agree as long as we don't duplicate information in the License Matching Guidelines Appendix.

I would prefer to cross-reference information in the spec so we do not have to maintain the information in 2 places.

[wking]

On Sun, Nov 26, 2017 at 07:51:25PM +0000, goneall wrote: Agree as long as we don't duplicate information in the [License Matching Guidelines Appendix](https://github.com/spdx/spdx-spec/blob/master/chapters/appendix-II-license-matching-guidelines-and-templates.md).

I'd also prefer to keep this DRY. But why does the spec care about the format? I'd rather *move* those docs over to this repository (updated to the new XML format), and have the spec linking a specific version of a completely external license list [1]. That way a schema change in this repository can be a single PR, and we can avoid doc-lag like we currently have, where the docs haven't been bumped to discuss the XML format at all. [1]: spdx/spdx-spec#46 (comment)

[goneall]

But why does the spec care about the format?

Format - I though we were talking about semantics ;) Agree the XML format should be in the repo. The semantics for <optional> and <alt> are the same as the <<beginOptional>> <<endOptional>> and <<var ...>> in the templates. Those semantics should be retained in the spec since they are broadly used outside the SPDX internal maintenance of the license list. The documentation in the repo can make reference to this specific section of the spec.

[wking]

On Mon, Nov 27, 2017 at 07:40:56PM +0000, goneall wrote: Those semantics should be retained in the spec since they are broadly used outside the SPDX internal maintenance of the license list.

Can you link a few examples? Are you thinking about external license lists? If so, I'd expect them to be interested in the syntax as well as the semantics. I'd recommend we have one schema repo with ListedLicense.xsd and semantic docs for the elements and attributes it defines and a namespace spec. Then license-list-XML can link a version of that schema and use the versioned namespace URL in it's XML where it currently uses the dummy http://www.spdx.org/license: $ curl -sLI http://www.spdx.org/license | grep '^HTTP\|^Location' HTTP/1.1 301 Moved Permanently Location: https://spdx.org/license HTTP/1.1 404 Not Found External XML license lists can reference the same schema/namespace repo. And the SPDX spec can point to the external license list, so the reference chain would be: spdx/spdx-spec → spdx/license-list-XML → spdx/spdx-XML-namespace instead of: spdx/spdx-spec ↔ spdx/license-list-XML

[goneall]

Can you link a few examples? Are you thinking about external license lists?

Just the license matching guidelines appendix (see link above). Note that it defines the regexp specifics which should be the same as the <alt> regexp description.

[wking]

On Tue, Nov 28, 2017 at 05:59:40AM +0000, goneall wrote: > Can you link a few examples? Are you thinking about external license lists? Just the license matching guidelines appendix (see link above).

That's what I'm interested in moving. It's not an “outside the SPDX internal maintenance” user. Perhaps you mean folks consuming translations of this repository (e.g. spdx/license-list-data)? I agree that license-list-data consumers will need docs for the formats there, but don't think spdx-spec is a good place for those docs either because that introduces dependency cycles. And currently the field docs are split between [1] and [2]. I'd like to have that all collected together.

Note that it defines the regexp specifics which should be the same as the ```<alt>``` regexp description.

That connection is currently not very discoverable. And some output forms remove or replace some of the metadata. For example, the HTML output currently only contains <<var;…>> for i2p-gpl-java-exception [3]. Was that really the only case when 2.6 was cut? Is license-list-data going to continue to provide alternatives with the <<var;…>> syntax? Or will it shift to using <span>, <var>, etc.? The preview page [4] and spdx/tools#121 suggest to me that license-list-data will be dropping <<var;…>>. But they could keep it if they want, and either way I don't think the license-list-XML docs should be discussing <<var;…>>. I'd like to see: * This repo get some docs for its format, along the lines of what's currently out in [1,2] and possibly more of [5]. * spdx/tools get some docs for its output formats, which can link to specific versions of the license-list-XML docs where syntax or semantics are recycled. * spdx/license-list-data link to the spdx/tools docs for the version of spdx/tools used to build them. * spdx/spdx-spec link to a version of license-list-data, and drop its current appendix I and II. Or it can give a floating link to license-list-data, and folks can use LicenseListVersion [6] or similar external constraints to select their license list. That will fix a number of confusing points, such as the current situation where it's not clear how LicenseListVersion interacts with SPDX License List v2.5 mentioned in [7] and the license-expression requirement that license IDs come from appendix I [8]. If someone sets LicenseListVersion to 2.6, can they use ‘Net-SNMP’ in their license expressions or not? And currently if someone sets LicenseListVersion to 2.0, where can we go to see what the valid license IDs are (license-list-data tags only go back to 2.4)? [1]: https://spdx.org/spdx-license-list/license-list-overview#fields [2]: https://github.com/spdx/spdx-spec/blob/cfa1b9d08903befdf03e669da6472707b7b60cb9/chapters/appendix-II-license-matching-guidelines-and-templates.md [3]: https://github.com/spdx/license-list-data/blob/v2.6/html/AAL.html#L5 [4]: https://spdx.org/licenses/preview/i2p-gpl-java-exception.html [5]: https://spdx.org/license-list [6]: https://github.com/spdx/spdx-spec/blob/cfa1b9d08903befdf03e669da6472707b7b60cb9/chapters/2-document-creation-information.md#2.7 [7]: https://github.com/spdx/spdx-spec/blob/cfa1b9d08903befdf03e669da6472707b7b60cb9/chapters/appendix-I-SPDX-license-list.md [8]: https://github.com/spdx/spdx-spec/blame/cfa1b9d08903befdf03e669da6472707b7b60cb9/chapters/appendix-IV-SPDX-license-expressions.md#L13

[goneall]

@wking I agree we need to straighten out the documentation but I disagree on removing the license template documentation from the spec. Currently, we have agreed that the XML format for licenses is used internally by the legal team. License templates are used externally. I don't think we want to move information needed by the external community to utilize license templates to a repository that is intended to be used internally by the legal team.

[wking]

On Tue, Nov 28, 2017 at 05:41:50PM +0000, goneall wrote: I don't think we want to move information needed by the external community to utilize license templates to a repository that is intended to be used internally by the legal team.

Which is why I think the docs for license templates exposed by license-list-data should be documented in spdx/tools and linked or duplicated in license-list-data. There's no reason to assume a tight coupling between the license-list-data formats and the XML in this repository.