Initial version of glTF spec in asciidoctor format

[oddhack]

For Working Group review. Specs built from the latest commit can be viewed by:

• Go to https://github.com/KhronosGroup/glTF/actions
• Click through to latest 'CI' workflow under 'All workflows'
• Click on 'spec-outputs'. This downloads a ZIP file with PDF and HTML outputs.

Building the spec yourself

To generate the specs locally, we recommend using a Khronos-supported Docker image as follows:

• Install docker (on Windows, you would probably need to do this under a WSL2 Linux environment)
• Execute 'docker run -it --rm -v full-path-to-your-local-repo-clone:/glTF khronosgroup/docker-images:vulkan-docs /bin/bash'
• 'cd /glTF/specification/2.0'
• 'make README.html README.pdf'

If you want to install the toolchain components yourself, refer to the vulkan-docs-base Dockerfile, which shows the steps needed to layer the toolchain components onto a base Ruby installation in a Debian or Ubuntu environment. We do not provide instructions for installing the components natively on other OS platforms.

Open issues

• Decided: spec will be published separately from the repository (probably in a glTF/3dformats registry), so there's no need to support github's asciidoctor renderer limitations.
  • To decide: now that the properties reference is being generated from the schema, it raises an issue with github's asciidoctor renderer. They do not support the asciidoctor include:: directive and aren't likely to. So when viewing README.adoc via github, the properties reference shows up as an external link, rather than expanding inline. There are a couple of possible workarounds:
    • Publish the spec in this two-part form.
    • Build the spec offline and publish the output HTML / PDF separately, in a registry repository, as many other Khronos specs do.
    • Check in a single-file spec source built by generating the properties reference, then editing that into the README. That's what the Makefile does with the SingleSpec.* targets.
• To decide: the generated properties reference comes with its own TOC. As a standalone document that is appropriate, but the automatically generated TOC of the asciidoctor outputs already includes every section in this chapter, so the extra one may be redundant. (Ed has added an option to wetzel to remove the extra TOC).
• To do: there are some minor issues with the generated properties reference (it should be using asciidoctor xrefs, rather than link: directives) - should be straightforward to correct and mostly affects just the PDF output.
• Outbound links to gltfOverview-2.0.0b.png and to the extensions registry are now to the master branch in github, rather than to versions in the local repository. This allows building a meaningful document for offline local consumption outside a local repository and seems appropriate for canonical external references, as well.
• To be fixed: the Burley2012 link in the References section is dead.
• To be fixed: the Pharr2018 reference text says "2016" when referring to the 3rd edition, which is actually 2018 AFAICT (edit: the 2018 date was wrong, 3rd ed. published in 2016).
• The 'MIME Type' and 'File Extension' sections under 'GLB File Format Specification' appear to be one level too deep (skipped from # to ### Markdown level). I moved them up in the asciidoctor document.
• Get checkmarks (from wetzel-generated content) rendering properly, which see Adding support for more math characters asciidoctor/asciidoctor-pdf#1832 - there are several options to pursue.
• There's a spurious page break generated in the PDF output after the Appendix A.21 title, presumably related to the long schema that followed, but other sections with multipage schemas don't seem to do this.
• Consider migrating to builtin stem math processing instead of images - comparison available in Compare latexmath to image math rendering in asciidoctor branch #1909 along with some pros/cons of converting.

[oddhack]

For reference, here are a couple ways of viewing the document:

(edit 2020-11-20: latest PDF/HTML are in CI artifacts, see top of PR comments above for how to access).

[emackey]

@oddhack If you get a chance can you regenerate the PDF and HTML zips here?

Also are there remaining problems in the wetzel-generated output that you can see? Thanks!

[oddhack]

Regenerated PDF and HTML (zipped) outputs. Latest updates are now linked from the top of the issue description.

Appendices are all now "numbered" as appendices.

Now that wetzel is in npm, uses 'npx wetzel' rather than needing the wetzel repo installed (thanks @donmccurdy @emackey !)

[oddhack]

@emackey should wetzel be running against the canonical schema in master branch on github, or against the locally checked-out schema? It's doing the former in the Makefile now but that seems suboptimal if anyone plans to evolve the schema :-)

[oddhack]

@emackey you asked in email if the Makefile can be run on Windows. TBH I have no idea. It will certainly run in Debian installed under WSL, though. If people want to experiment building the spec themselves, I can also provide instructions on using our Docker image with asciidoctor toolchain pre-installed. I don't know how running Docker images under Windows works since I can't nest a Docker container runtime inside a Windows 10 VM on my Linux desktop, but I expect it works when running under native Windows.

[oddhack]

Notes on running our Docker image and building the spec now in the PR header.

[oddhack]

@oddhack If you get a chance can you regenerate the PDF and HTML zips here?

Also are there remaining problems in the wetzel-generated output that you can see? Thanks!

It looks fine to me from a markup POV, so if the formatting is as you wish it, we should be good now.

[emackey]

should wetzel be running against the canonical schema in master branch on github, or against the locally checked-out schema? It's doing the former in the Makefile now but that seems suboptimal if anyone plans to evolve the schema :-)

That's not quite what wetzel is doing. From the Makefile, wetzel's arguments look something like this: (abbreviated)

$(WETZEL) ... -p "$(SCHEMA)" ... schema/glTF.schema.json > $@

That last argument schema/glTF.schema.json is a relative reference to the top-level glTF schema file within the branch. All of the branch's other schema files are pulled in by some chain of references initiated from the root schema.

The -p "$(SCHEMA)" portion tells wetzel that it should embed links back to the source schema for each section that it finds. In response to this argument, wetzel inserts links like these:

* **JSON schema**: link:https://github.com/KhronosGroup/glTF/tree/master/specification/2.0/schema/accessor.schema.json[accessor.schema.json]

Of course, it's possible for a branch to differ from the master schema. This would result in a document that showed the branch's reference material, while linking back to GitHub master as the source. Worst case, a whole new section could be added on a branch, and wetzel would write a broken link to something that doesn't exist in master. This won't happen within glTF 2.0's timeframe of course, but could happen on the road to some future version.

Two thoughts:

1. We could rename $(SCHEMA) to something like $(SCHEMALINK) or some such, to try to clarify that the actual schemas are not being read from there.

2. Since ISO may want a PDF with the schema contents included verbatim, we could have an option to inject the full schema there rather than just linking back to it. Or, we could bulk-include the schemas in some separate section of the PDF. Can AsciiDoctor "include" an external file that's code-formatted plain text like a schema?

[emackey]

One more thought: -p could be given an AsciiDoctor variable, like -p '{schemaurl}' and might produce something like:

* **JSON schema**: link:{schemaurl}/accessor.schema.json[accessor.schema.json]

I didn't test this but I think it would work. Of course one would need to define {schemaurl} in AsciiDoctor, and I don't know if that can be in the parent document or needs to be in the same document as the links.

[oddhack]

That's not quite what wetzel is doing. From the Makefile, wetzel's arguments look something like this: (abbreviated)

Got it, thanks. I just saw the URL and assumed it was pulling from there, my bad.

We could rename $(SCHEMA) to something like $(SCHEMALINK) or some such, to try to clarify that the actual schemas are not being read from there.

For my confusion, I think a clarifying comment in the Makefile where the variable is set might have been more useful - something like "# Links to schema are generated to these URLs, which may not match the ones imbedded in the document"

Can AsciiDoctor "include" an external file that's code-formatted plain text like a schema?

Yes.

Of course one would need to define {schemaurl} in AsciiDoctor, and I don't know if that can be in the parent document or needs to be in the same document as the links.

It can - in this case it would probably make sense to put it on the Makefile command line for asciidoctor as well as wetzel:

asciidoctor ... -a schemaurl="$(SCHEMAURL)"

[oddhack]

Can AsciiDoctor "include" an external file that's code-formatted plain text like a schema?

Following up a bit - for imbedding JSON in an appendix, perhaps wetzel could be invoked with different options that expand the schema name into the corresponding include directive rather than generating the properties descriptions, then that second generated file be included as an appendix? So output could look something like this:

[[json-accessor]]
== JSON for accessor schema

[source,json]
----
include::schema/accessor.schema.json[]
----

and so on for the other classes. The Properties Reference section could then potentially link to the json-schema anchors, instead of or in addition to the repository URLs, to make the document more self-contained for ISO.

[emackey]

I switched the checkmarks to ✓ but the PDF version doesn't recognize that at all. I got most of AsciiDoctor installed locally on my home Windows machine, except for asciidoctor-mathematical which won't build on Windows 10 on account of looking for a 1990's era copy of Bison for Unix. So for now I can only build HTML locally, and I rely on CI for the PDF.

[emackey]

I found one that works: icon:check[]. I'll update later today. It's pulling from the FontAwesome set which was already being used for the "i" next to implementation notes.

[emackey]

Checkmarks are working now. The whole document is starting to look very presentable with all of the recent changes.

[emackey]

@lexaknyazev When this is ready, can we do a full merge instead of squash? jon-adoc has a lot of history showing how the adoc version developed and evolved, and it could be valuable reference for future folks looking back on how this came to be.