Support accepting multiple files as CLI arguments

[craigfe]

The CLI subcommands (output, pp, rule, test) currently all accept only a single file as input; it might be nice if they could accept more than one at a time.

This would be helpful when verifying a set of markdown files, as in the Irmin tutorials. Currently, I think this requires an unsightly bash action to do it in a single Dune action:

  (action (with-stdout-to %{targets}
            (bash "echo %{tutorials} | xargs --max-args=1 ocaml-mdx pp")))

Whereas it would be nice to get away with simply:

  (action (with-stdout-to %{targets}
            (ocaml-mdx pp %{tutorials})))

What do you think?

[samoht]

that sounds like a good idea!

[NathanReb]

That would also be good for the dune integration I think!

[NathanReb]

It's actually better to invoke mdx once per file in dune so nevermind my previous comment.

[gpetiot]

Does the recent mdx dune stanza makes it easier or is this issue still relevant?

[craigfe]

I haven't ported the Irmin tutorials to use the mdx stanza yet: it's waiting for the release of the HTML-syntax for MDX attributes (also currently blocking mirage/irmin#991), and is a non-trivial change due to needing the workaround described in #158.

If the pp command is going to be deprecated in favour of the mdx stanza, this issue can probably be closed. Having said that, I still find pp a better literate programming workflow than the stanza + explicit file/part labels, so would only switch grudgingly 😉

[shonfeder]

Also worth noting that supporting multiple files is a nice feature for using this in tests outside of dune.

[NathanReb]

Also worth noting that supporting multiple files is a nice feature for using this in tests outside of dune.

I think at the moment the dune stanza is the recommended way to use MDX and we're more headed towards dropping the mdx binary than the other way around.

If the pp command is going to be deprecated in favour of the mdx stanza, this issue can probably be closed. Having said that, I still find pp a better literate programming workflow than the stanza + explicit file/part labels, so would only switch grudgingly 

Coming back to this comment, I think litterate programming as proposed by MDX is a very simple feature, likely one that should be extracted to a different, more focused tool and turned into a dune dialect or something similar. This feature probably don't belong in MDX anymore as it does not use any of the actual MDX features which are to actually interpret blocks. The pp command is just an alternative parser for OCaml basically.

Might be worth looking into how to provide a better experience with that workflow and cut this out of the tool.

[Leonidas-from-XIV]

Maybe then we should open some tickets to plan and schedule removal of features that have since been superseded by other tools? Kinda similar with how for cram tests it is better to just use dune directly…

[shonfeder]

I see. Thanks for the clarifying the direction.

So if testing generally, and cram-style tests in particular, and literate programming in markdown, and the mdx binary in general are no longer on the roadmap or prioritized, I'd be interested to learn the main focus of the project going forward (I don't mean that sarcastically!).

[NathanReb]

The dune stanza uses MDX to run the equivalent of mdx-test. What we mean here is that users don't have to right their own rules anymore to benefit from the mdx test promotion workflow and simply have to write a dune stanza.

MDX still does all the work here.

I'm not sure I understand what you're implying. The focus of MDX is the maintenance of code examples and fragments in documentation (markdown, mli, mld). You can still use it as a test tool too, the MDX stanza attaches the targets to the @runtest alias.

[NathanReb]

We merely deprecated an old and tedious workflow where users had to write their own rules, or use MDX to generate dune files.

Indeed we're dropping cram testing (as in support for cram files, you can still run shell examples in your markdown for example) as dune provides a better integrated workflow for it.

[NathanReb]

I was wondering: are you unhappy with the features provided by the dune stanza? Do you feel like some features of the mdx test command are not covered? If so, please don't hesitate to fill in an issue and we'll be happy to look into it.

At the moment none of the features you mentioned have been removed, we're just trying to write the right tools to provide them.

For example, I think the pp command could be extracted to a separate, simpler tool, with a smaller set of dependencies and better integrated with dune so users don't have to write custom rules to benefit from it.
In the same way, MDX initially provided a cram test feature because no tools was doing so back then. It was useful at the time but now we feel like dune does a better job at it and it makes more sense to use the dune feature directly rather than using MDX for it.