fix: MathML combines multidigit numbers with sup/subscript, comma separators, and multicharacter text when outputting to DOM

[edemaine]

What is the previous behavior before this PR?

When rendering MathML directly to the DOM (as opposed to a string), such as on the katex.org front-page demo, consecutive digits and text characters get rendered as multiple text children of an <mn> or <mtext> elements:

What is the new behavior after this PR?

When rendering MathML directly to the DOM, we combine consecutive text children into one:

Fixes #3995 which reports that this causes screen-reading issues.

---

Merging of consecutive <mtext> and <mn> elements is already done here:

KaTeX/src/buildMathML.js

Lines 157 to 167 in 2d1fec9

	// Concatenate adjacent <mtext>s
	if (group.type === 'mtext' && lastGroup.type === 'mtext'
	&& group.getAttribute('mathvariant') ===
	lastGroup.getAttribute('mathvariant')) {
	lastGroup.children.push(...group.children);
	continue;
	// Concatenate adjacent <mn>s
	} else if (group.type === 'mn' && lastGroup.type === 'mn') {
	lastGroup.children.push(...group.children);
	continue;
	// Concatenate <mn>...</mn> followed by <mi>.</mi>

And when rendering to a string, this was enough. Indeed, we don't have a great way of testing this, without a virtual DOM setup. But the inspect screenshots above confirm that this PR fixes the issue.

[ronkok]

The code written above will split a number like "4,200.30" into two elements, split at the comma. It will also mangle an expression like 24^3, writing the "4" and the "3" inside a <msup>, but leaving the leading "2" outside the <msup>.

In Temml, I struggled a bit and ended up with a more complex solution. My approach modified only the buildMathML.js file. After line 152, I inserted a function call:

    consolidateNumbers(expression);

Elsewhere in the same file, I added this code:

const numberRegEx = /^[0-9]$/
const isDotOrComma = (node, followingNode) => {
  return ((node.type === "textord" && node.text === ".") ||
    (node.type === "atom" && node.text === ",")) &&
    // Don't consolidate if there is a space after the comma.
    node.loc && followingNode.loc && node.loc.end === followingNode.loc.start
}
const consolidateNumbers = expression => {
  // Consolidate adjacent numbers. We want to return <mn>1,506.3</mn>,
  // not <mn>1</mn><mo>,</mo><mn>5</mn><mn>0</mn><mn>6</mn><mi>.</mi><mn>3</mn>
  if (expression.length < 2) { return }
  const nums = [];
  let inNum = false
  // Find adjacent numerals
  for (let i = 0; i < expression.length; i++) {
    const node = expression[i];
    if (node.type === "textord" && numberRegEx.test(node.text)) {
      if (!inNum) { nums.push({ start: i }) }
      inNum = true
    } else {
      if (inNum) { nums[nums.length - 1].end = i - 1 }
      inNum = false
    }
  }
  if (inNum) { nums[nums.length - 1].end = expression.length - 1 }

  // Determine if numeral groups are separated by a comma or dot.
  for (let i = nums.length - 1; i > 0; i--) {
    if (nums[i - 1].end === nums[i].start - 2 &&
      isDotOrComma(expression[nums[i].start - 1], expression[nums[i].start])) {
      // Merge the two groups.
      nums[i - 1].end = nums[i].end
      nums.splice(i, 1)
    }
  }

  // Consolidate the number nodes
  for (let i = nums.length - 1; i >= 0; i--) {
    for (let j = nums[i].start + 1; j <= nums[i].end; j++) {
      expression[nums[i].start].text += expression[j].text
    }
    expression.splice(nums[i].start + 1, nums[i].end - nums[i].start)
    // Check if the <mn> is followed by a numeric base in a supsub, e.g. the "3" in 123^4
    // If so, merge the first <mn> into the base.
    if (expression.length > nums[i].start + 1) {
      const nextTerm = expression[nums[i].start + 1];
      if (nextTerm.type === "supsub" && nextTerm.base && nextTerm.base.type === "textord" &&
          numberRegEx.test(nextTerm.base.text)) {
        nextTerm.base.text = expression[nums[i].start].text + nextTerm.base.text
        expression.splice(nums[i].start, 1)
      }
    }
  }
}

It's quite possible that parts of both solutions could be combined to form a solution that is less verbose than the Temml version, but still is correct.

[mbourne]

@ronkok Wondering if your solution caters for European number style, with periods and commas reversed, so 1.234,5, or with spaces, like 10 000,45?

[edemaine]

I haven't grokked your code yet, but I'm a little confused about the goal. In LaTeX you need to write 1{,}000.00. If you write 1,000 you get rendering that looks like a list of two numbers. Is a braced comma what you mean by a comma?

These heuristics feel a little messy, but it seems they're necessary for going from presentation to semantics.

I also don't think they're especially related to this PR. Do you agree that the change in this PR is useful, independent of how the nodes get combined?

[ronkok]

@mbourne The European number style was a major motivation for how I wrote this code.

@edemaine I take your point about TeX requiring a 1{,}23 to get a separator without a space. I think Knuth made a mistake. A majority of the world uses a comma for their decimal separator. It should be a first-class option, not relegated to a workaround with extra braces. Having taken your point, I intend to modify Temml so that 1{,}23 will be written to a single <mn> element.

[ronkok]

These heuristics feel a little messy

I agree. I'd like to find code that is less verbose. I haven't found it yet.

Do you agree that the change in this PR is useful

It is certainly better than the status quo. I would suggest at least examining the following atom to see if it a msup or msub element. This PR, as written will not recognize the "24" in 24^2 as a single number. That seems like a pretty big semantic bust.

[edemaine]

@ronkok I've taken a stab at handling ., {,}, and sup/subscripts. Let me know what you think! (You can try it out live in the preview.)

[ronkok]

Good stuff! I haven't looked at the code yet, but the preview shows me that you have:

• handled a dot as a decimal separator well.
• handled {,} as a decimal separator well.
• handled a space as a thousands separator well.
• handled a following sup or sub well.
• handled a plain comma as I expected you to handle it. That's okay. I would have handled it differently, but I understand your reasons. You show more fidelity to TeX than I do, and given your goals, your code works as it should.

I hate to move the goal posts at this late date, but I have just now realized there is another problem. In the expression x^2.2, the "2.2" is not treated as a single <mn> element. The code needs to check after a sup or sub for a following number.

[edemaine]

Thanks for reviewing!

• I would have handled it differently, but I understand your reasons. You show more fidelity to TeX than I do, and given your goals, your code works as it should.

Yep, exactly.

I hate to move the goal posts at this late date, but I have just now realized there is another problem. In the expression x^2.2, the "2.2" is not treated as a single <mn> element.

I believe that that is actually correct: here's how it renders in HTML.

So the MathML correctly reflects the same.

x^{2.2} correctly produces a single <mn>2.2</mn> for the exponent.

[ronkok]

Ah yes, you're correct. Which means that all the behavior I see from this code is good.

I don't plan to do a detailed review of code style, but I think it looks good in general.

[KaTeX-bot]

🎉 This PR is included in version 0.16.17 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀