upgrade gatsby 2.22.15 to 2.24.62 breaks comment syntax support in mdx

[aharbis]

Description

We are working on package update to pull in some bug fixes across our dependencies. I ran an npm update and gatsby updated from 2.22.15 to 2.24.62, and the plugin in question (gatsby-plugin-mdx) updated from 1.2.13 to 1.2.40.

After this update, our build failed with the following error:

error "gatsby-plugin-mdx" threw an error while running the onCreateNode lifecycle:

Unexpected character `!` (U+0021) before name, expected a character that can start a name, such as a letter, `$`, or `_`/home/travis/build/datapower/datapower-operator-doc/src/pages/getting-started/index.mdx: Unexpected character `!` (U+0021) before name, expected a character that can start a name, such as a letter, `$`, or `_`

Looking at this particular index.mdx file, the only existence of a ! character that I can find is within a comment block, i.e.:

<!-- comment -->

I'm guessing that this is what's triggering the failure, but I don't understand why this would be invalid. Checking current Gatsby MDX documentation there are examples of this same comment syntax.

Steps to reproduce

I tried to reproduce this using the How to Make a Minimal Reproduction repo, but I was not able to reproduce the issue. Here's my commit showing the diff of what I added (and this works as far as I can tell):

aharbis/gatsby-bug-repro@26b5eb8

npm run build passes on this repo, and I've confirmed the package-lock.json shows 1.2.40 for the gatsby-plugin-mdx package.

So, since the minimal repro isn't showing the issue, I might as well share the actual code that is? 🤷‍♂️

Since it's MDX, kind of hard to permalink the line, so here's a blame link at the line that I suspect is causing the issue:

https://github.com/IBM/datapower-operator-doc/blame/master/src/pages/getting-started/index.mdx#L19

Expected result

Comment syntax <!-- comment --> works in MDX.

Actual result

Comment syntax <!-- comment --> breaks in MDX.

Environment

$ gatsby info --clipboard

  System:
    OS: macOS 10.15.6
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 12.18.3 - ~/.nvm/versions/node/v12.18.3/bin/node
    npm: 6.14.6 - ~/.nvm/versions/node/v12.18.3/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Firefox: 80.0.1
    Safari: 13.1.2
  npmPackages:
    gatsby: ^2.24.62 => 2.24.62
    gatsby-theme-carbon: ^1.26.10 => 1.26.10
  npmGlobalPackages:
    gatsby-cli: 2.12.99

[aharbis]

I was able to resolve this build issue by taking the following steps:

1. Revert changes to package.json and package-lock.json (essentially reverting my previous npm update)
2. rm -rf node_modules
3. npm ci
4. Upgrade gatsby-theme-carbon to 1.26.10 (this step also updated gatsby-plugin-mdx to 1.2.40)
5. npm run build (logs from gatsby build showed that plugin changes were detected, and caches were cleaned to rebuild)
6. npm update (this updated gatsby to 2.24.63)
7. npm run build

I'm not quite sure why the order of operations here mattered, but my issue is resolved and this does not appear to be a bug in the gatsby-plugin-mdx plugin itself. Closing.

[axe312ger]

I found this issue and similar ones with gatsby-plugin-mdx when upgrading, especially with yarn upgrade-interactive.

If you end up with multiple version of the plugin, you might have multiple versions of @mdx-js/react as well. As the React implementation uses a global context, you might end up doubling it. This can happen in combination with theme-ui and react-i18next as well.

My fix:

• Reduce dependency mess from yarn 1.0 via https://github.com/atlassian/yarn-deduplicate (it claims yarn 2.0 does not have the problems, did not try yarn 2.0 yet)
• if it does not help: rm -rf yarn.lock & node_modules, then install all dependencies via yarn again

Hope this helps :)