v2: Add Math extension

[Ambrevar]

It enables direct math support in LaTeX, as well as HTML with MathJax.

I believe this feature to be a popular demand. See #256 and #220.

This is a request for comments, as I am quite new to Blackfriday.
I haven't written many tests yet, I'm waiting for your comments.

The main point of contention would be the math delimiter, being '$' and '$$' at the moment. This obviously creates a conflict with a regular use of the dollar signe. Both MathJax and LaTeX use '(' and '['. I am not sure how we can use those in Markdown because of the conflict with the 'escape' rules.

[Ambrevar]

Any comment on this?

[rtfb]

Yeah... My elsewhere stated position that I'm very reluctant to accept any changes that affect interpretation of source document still holds. I would love to at least postpone any such changes until we support CommonMark, when we might have some better functional cross-checking with the rest of the world.

That said, I had an idea which might save MathJax. The block statement is easy: just write a code block and use mathjax as a language specifier, then apply custom logic in a custom code block renderer.

For an inline math, we could choose a combination of a backtick and dollar sign to punch through an untouched math equation to a custom inline code renderer.

Ideally, I would like our API to expose some mechanism to add hooks for these kinds of custom behavior, akin to HTTP handler middleware common in Go world. This way, people could find ways to implement extensions for their peculiar needs without asking us to do it and clutter an API for every case. Some of our own extensions could be reimplemented in these terms as well.

[Ambrevar]

Interesting. I like the idea. I also think it feels natural to embed math with code: in some ways, math is a sort of code.

CommonMark makes no provision for math support, so I guess your way would be the leanest.
Is there anything done regarding CommonMark validation?

Could you be more specific regarding the hooks? You mean on the renderer side? This is what I think of at the moment: add some BlockHooks, InlineHooks map[string]func to the renderer parameters, the string would be the language descriptor (in case of inline, the first characters), and the hook function would be of type func ([]byte) []byte or something.

Edit: should those hooks be reflecter in the Renderer interface? I would say no.

[Ambrevar]

I understand your point regarding the interpretation of the source, but what about the 'table' extension then? It is the same as for my current math extension. Or should we move tables in code blocks as well?

[Ambrevar]

What about #262 ?

[rtfb]

Could you be more specific regarding the hooks? You mean on the renderer side? This is what I think of at the moment: add some BlockHooks, InlineHooks map[string]func to the renderer parameters, the string would be the language descriptor (in case of inline, the first characters), and the hook function would be of type func ([]byte) []byte or something.

My initial idea is a sort of a callback chain for every Node type. Kinda like this:

walker := blackfriday.NewNodeWalker()
walker.addCallback(blackfriday.CodeBlock, mathCodeBlockFunc)

and addCallback would add it to a queue of callbacks which can alter the Node and potentially stop node's propagation to other callbacks. I don't like this particular idea too much, because it's very limited and smells of C code too much, but I'm not sure anything substantially better is possible.

Edit: should those hooks be reflecter in the Renderer interface? I would say no.

No, not in Renderer, but a compliant renderer should respect the callbacks.

Side note:

Cha! I do expose NewNodeWalker, but Node.Walk privately calls it and there's no way to customize the walker ATM. That definitely needs to be fixed if we want to properly expose Walker :-)

[rtfb]

What about #262 ?

And yes, I'd also like to postpone things like #262 as well.

[Ambrevar]

I'm not sure I understand the purpose of your hooks: we've already got a visitor function to pass to the walker which can "hook" on the node types as the user wishes.

Am I missing something?

But this actually gave me an idea: I used the visitor to hook Code and CodeBlock to implement the math support:

ast.Walk(func(node *bf.Node, entering bool) bf.WalkStatus {
    switch node.Type {
    case bf.Code:
        if bytes.HasPrefix(node.Literal, []byte("$$ ")) {
            output.Write([]byte("\\("))
            output.Write(escCode(node.Literal[3:]))
            output.Write([]byte("\\)"))
            return bf.GoToNext
        }
    case bf.CodeBlock:
        infoWords := bytes.Split(node.Info, []byte("\t "))
        if len(infoWords) > 0 && bytes.Equal(infoWords[0], []byte("math")) {
            output.Write([]byte("\\[\n"))
            output.Write(escCode(node.Literal))
            output.Write([]byte("\\]\n\n"))
            return bf.GoToNext
        }
    }
    return renderer.RenderNode(&output, node, entering)
})

In my proposal, math blocks are defined by the "math" keyword and inline math is inline code that starts with "$$ ". Not too sure about this last one, it certainly requires some discussion.

Anyways, it works beautifully, it does not change the parser, it is fully compatible with any existing input (except for the "$$ " prefix).

Comments?

[rubik]

Any news about this one?

[Ambrevar]

In the meantime, here is my homepage generator:
https://bitbucket.org/ambrevar/ambrevar.bitbucket.org/src/master/generate.go

It has math support, it hacks around the AST and support LaTeX output.