feat: Title support via front matter for ER, state, class, git, and graph diagrams

[MasonM]

📑 Summary

This adds title support for all diagram types currently lacking support for titles (excluding "info" and "error") using Jekyll-style YAML frontmatter blocks. C4C diagrams, pie charts, gantt charts, requirement diagrams, sequence diagrams, and user journey diagrams already have title support.

Resolves #1433

📏 Design Decisions

1. This increases the size of mermaid.min.js by 33KB and mermaid.esm.min.js by 42KB:

   Before:
   $ ls packages/mermaid/dist/mermaid*min*js -la
   -rw-rw-r-- 1 masonm masonm 1591427 Nov 19 10:27 packages/mermaid/dist/mermaid.esm.min.mjs
   -rw-rw-r-- 1 masonm masonm 1176130 Nov 19 10:27 packages/mermaid/dist/mermaid.min.js
   
   After:
   $ ls packages/mermaid/dist/mermaid*min*js -la
   -rw-rw-r-- 1 masonm masonm 1633782 Nov 19 10:25 packages/mermaid/dist/mermaid.esm.min.mjs
   -rw-rw-r-- 1 masonm masonm 1208723 Nov 19 10:25 packages/mermaid/dist/mermaid.min.js
   

   I avoided using gray-matter or similar packages to minimize the bundle size increase and avoid the maintenance hassles associated with new dependencies.
2. The parser uses the failsafe schema to keep things simple and avoid the Norway problem.
3. Only support for titles in frontmatter have been added. Support for other types of metadata should be done after this is merged IMO, since that will require more discussion (as noted by @sidharthv96).
4. I didn't deprecate the existing title syntax for those diagram types that already support titles, since I'm not sure which docs to update, and that would make this PR too big IMO. I think that should be done in a follow-up PR.
5. I added a titleTopMargin configuration option for every diagram type for consistency. Making it a global option might make more sense, but that could raise backwards-compatibility concerns (e.g. what if both gantt.titleTopMargin and the global titleTopMargin are set).

📋 Tasks

Make sure you

• 📖 have read the contribution guidelines
• 💻 have added unit/e2e tests (if appropriate)
• 🔖 targeted develop branch

[MasonM]

The failure in the "E2E / build (18.x, 1) (pull_request) " check appears unrelated:

Warning: Failed to download action 'https://api.github.com/repos/actions/setup-node/tarball/8c91899e586c5b171469028077307d293428b516'. Error: Response status code does not indicate success: 500 (Internal Server Error).

[aloisklink]

The failure in the "E2E / build (18.x, 1) (pull_request) " check appears unrelated

Yep! Just seems to be GitHub's servers being flakey.

This isn't a proper review (I'm not an expert on the rendering code, so will leave that to other reviewers), but from one I've seen, it all looks good! Thanks for adding tests and demo examples!

Potential improvements:

• Can you add a title example to the packages/mermaid/src/docs/classDiagram.md?

[MasonM]

Thanks for the review @aloisklink! Extracting the rendering logic to a helper method sounds reasonable, so I went ahead and did that and added tests for it. I also updated the docs for packages/mermaid/src/docs/classDiagram.md

[aloisklink]

Looks good to me!

Again, I'm not super experienced with SVG/d3/rendering code, so another reviewer will need to double-check that.

As a quick example, this is what the new title in the http://localhost:9000/git.html demo looks like:

[knsv]

Wow Mason! A big job, I appreciate changes that align diagrams. One of the challenges going forward is keeping the diagrams in line. Will do a deep dive here as it affect many diagrams.

[knsv]

Hi @MasonM!

Great job but we are not there yet! I have reviewed you changes and found some issues. When you introduced the title keyword you also unwittingly made it impossible to create a state named title, a flowchart with an id of title/usage of title in a subgraph title or an er diagram with an entity title.

We are very restrictive with changes that would break existing diagrams. It is one thing to introduce a change leading to a developer having update the integration of mermaid on a site but whole other thing with an update that is forcing authors to go in and update the already written documentation.

On possible fix for the issue would be, to be more restrictive with scoping out what a title is, in the lexer. The solutions will probably differ between the different diagrams though. Another avenue could be to use the different states in the parser to know if a title could be expected.

One example of a breaking diagram:

erDiagram
    title ||--o{ ORDER : places
    CUSTOMER {
        string title
        string custNumber
        string sector
    }
    ORDER ||--|{ LINE-ITEM : title
    ORDER {
        int orderNumber
        string deliveryAddress
    }
    LINE-ITEM {
        string productCode
        int quantity
        float pricePerUnit
    }

Good luck,
Knut

[MasonM]

@knsv Thanks for the review! I understand the important of retaining backwards-compatibility, but I'd assumed there was some leeway here, since the changes that introduced accTitle in #3008 were also breaking changes, for the same reason as these changes (e.g. it broke state diagrams with a state named accTitle:). It's unlikely those changes affected anyone, since accTitle is an uncommon phrase, but it's still technically possible.

If the policy is that backwards-incompatible changes are strictly forbidden, then the only option I can see is using a directive, e.g.:

graph TD
    %%{ title: 'testing' }%%
    foo --> bar

That's cumbersome, but it's guaranteed to be backwards-compatible, and all the diagrams already support directives. I can go ahead and implement that if you want, but if there is some leeway here, then changing title to title: or using slightly different syntax would probably be preferable. Let me know what you'd like.

[knsv]

Hi! Yeah, thats the price of an organically growing syntax. If I started today I would make the syntax a little different so that states in the parser can be applied easier but it is as it is.

I am thinking that restricting the lexer regarding the title could be a good place to start. Right now you use "title"\s[^#\n;]+ for flowcharts. So this makes the parser interpret title followed by space and anything except #, ; or new-line as being a title. Here you could be more strict and force a space and a quote. You could also require an end quote before new line before classifying the string a title.

Hope this helps!

[MasonM]

I am thinking that restricting the lexer regarding the title could be a good place to start. Right now you use "title"\s[^#\n;]+ for flowcharts. So this makes the parser interpret title followed by space and anything except #, ; or new-line as being a title.

That's still a breaking change for some diagram types. Consider this state diagram, which currently renders as a diagram with the states title and "foo", but would instead just show the title "foo" after such a change.

It sounds like you're okay with "minor" breaking changes, but the definition of "minor" is quite subjective. That's perfectly reasonable, but I can't read your mind, and I'd prefer not to guess and spend a lot of time implementing a new syntax, only to be told it won't be accepted because it's too big of a breaking change. Could you please tell me the exact grammar you'd like to see for each diagram type?

[knsv]

You are right, that solution will only work for flowcharts and er diagrams. We need a different solution for state diagrams.

I would to start with proposing to delay State diagrams a bit so that we can get this done in October as a part of Hacktoberfest.

[MasonM]

@knsv It's going to break class diagrams too. Example.

As I see it, there's four options here:

1. Accept there's going to be some breakage and stick with the current syntax, as was done with accTitle in 2732 accesible charts #3008
2. Go with directives, as I suggested earlier. This has zero chance of breaking anything but it's cumbersome.
3. Go with the the title "test" syntax you suggested, but only for flowcharts and ER diagrams, and revert all the changes in this PR for all the other diagram types. This means Mermaid will end up having at least three different syntaxes for defining titles: title test, title "test", and whatever ends up being chosen for the other diagram types. That seems awfully confusing to me.
4. Close this PR and abandon the whole thing

Which of these 4 options would you prefer?

[knsv]

Thanks for your analysis. Some feedback:
1: This would be nice but title is a much to common word. Many diagrams will break. I don't think this option is realistic. We cant do this.
2: I agree that this one is safe but I also agree with you that it is cumbersome. I think that no one will understand how to use it.
3. The point is to make the syntax standardised between the diagrams. Then this is not suitable either.
4. I can see that this is growing a bit larger then you perhaps knew when starting out on this journey. I understand if you think this is too complicated in all the wrong ways but the essence of this issue is sound and I think we should solve it. Your help would be appreciated.

Some broader ideas:

• Let's deprecate the current way to set the title in the diagrams that can set the title. In practice, keep it in the parsers indefinitely in the old way but remove it from the documentation.
• These standardised functions should use the common db instead of putting the same function in all diagrams databases.
• Perhaps the syntax could be generic to be utilised for future properties as well, something like this perhaps:
  • _set: title "The title"
  • _set: otherProperty "value"

What do you think?

[MasonM]

Perhaps the syntax could be generic to be utilised for future properties as well, something like this perhaps:

That's a good idea. Having special syntax for distinguishing metadata (like the title) from the actual diagram data is likely be useful in the future. Implementing it in a backwards-compatible way will be tricky, but it's only going to get more challenging as Mermaid continues to grow and adds more diagrams and syntax.

State diagrams are particularly challenging due to the lax grammar, and _set: title "foo" won't work since it's currently parsed as states _set and title (example). A possible alternative is just prefixing it with a hyphen (i.e. -title foo), since hyphens aren't allowed in state names. From a quick look, it doesn't appear that syntax breaks any of the other diagram types either, but I'll need to spend more time this weekend to check. If it does work, then it could be generalized to the grammar "-" metadataName "\s+" metadataValue for representing arbitrary metadata.

[aloisklink]

That's a good idea. Having special syntax for distinguishing metadata (like the title) from the actual diagram data is likely be useful in the future. Implementing it in a backwards-compatible way will be tricky, but it's only going to get more challenging as Mermaid continues to grow and adds more diagrams and syntax.

Hmmmm, what about having an optional YAML header in a mermaid definition?

In many markdown software, you can optionally add a YAML header to add Markdown metadata, e.g. something like:

---
title: My title
date: 1970-01-01
author: FirstName LastName
---

# Hello world

The rules would be the same as in Markdown, it must be the first thing in a Mermaid definition, it might be valid YAML, and it must be between --- lines, e.g.

Here is my mermaid definition:

```mermaid
---
title: my graph
author: FirstName LastName
---
flowchart LR
    A[Hard] -->|Text| B(Round)
    B --> C{Decision}
```

Benefits:

• YAML is well known to most people that write Markdown
• YAML has a bunch of helpful features, e.g. the ability to write very long strings, comments, nested arrays/objects
• If we stick it before the graph definition, we could have a separate pre-processor that runs before the graphs, which would be easier to code.

[weedySeaDragon]

Jumping in (since I'm currently working with the state diagram parser) --

Why is title: not acceptable? I've read through the comments but have somehow missed why that isn't acceptable.

Is it only the problem with the state diagrams? If so, it's not hard to write a rule that looks for not just for : but for the entire string title: .

[aloisklink]

Why is title: not acceptable? I've read through the comments but have somehow missed why that isn't acceptable.

I believe it's because of backwards compatibility, there's a pretty good chance somebody has already created a mermaid state diagram that uses title: in it, and so changing the behavior of title: might break their diagrams.

E.g. something like:

```mermaid
stateDiagram
  title: Example state diagram
  my2ndState: My second state
  title --> my2ndState
```

stateDiagram
  title
  title: Example state diagram
  my2ndState: My second state
  title --> my2ndState

Loading

Or

```mermaid
stateDiagram
  title
  description
  title --> description
```

stateDiagram
  title
  description
  title --> description

Loading

[weedySeaDragon]

Ah Makes total sense. Thanks for spelling that out for me @aloisklink .

[MasonM]

Okay, I couldn't find anything that would break with the -title foo syntax, so I went ahead and pushed a commit to switch to that, but I do like @aloisklink's suggestion of using a YAML front matter block. I believe that was pioneered by Jekyll, and is also supported by Pandoc. It has a few disadvantages, however:

1. The syntax is significantly more verbose than the "-" metadataName "\s+" metadataValue syntax, but it's definitely more readable, which is more important IMO.

2. We'd need an extra package to handle parsing the front matter block, which has both performance and security ramifications. There is an NPM module called front-matter, but it's 171KB minified according to Bundlephobia, which is probably unacceptable. The js-yaml package is ~40KB minified, and is already an indirect dev dependency of Mermaid:

   mermaid/pnpm-lock.yaml

   Line 1650 in 77782fd

   js-yaml: 4.1.0

   
   40KB is an upper-bound estimate for the bundle size increase. A good portion of js-yaml is devoted to the serialization logic, which we obviously don't need, and Vite should be able to tree-shake that out, reducing the final size significantly.

   There was an arbitrary code execution vulnerability in js-yaml v3 a few years ago (advisory), which is worrisome, but it appears v4 made a lot of changes to improve security.

3. Directives are in JSON format, so having front matter in YAML could be a little confusing

Personally, I think YAML is probably the right way to go. If @knsv agrees, then I can go ahead and implement that using js-yaml.

[knsv]

I am all for it!

Further down the line if things work out I could see directives pulled in to this syntax. The old syntax could remain for backwards compatibility, this would be the documented one. The size increase is worry-some as mermaid is on the heavy side already.

I am convinced that having a preprocessing step is the method to use. This will eliminate conflicts with the diagrams.

[AssassinLV]

Why not then just:

```mermaid
title: my graph
author: FirstName LastName
flowchart LR
    A[Hard] -->|Text| B(Round)
    B --> C{Decision}
```

without encapsulating in --- as it would still be above the actual diagram definition.
This would work even if you create diagram type 'title', because of the column. Also only requirement is for the properties to be above the diagram definition - so you could do formatting like:

```mermaid
%% properties
title: my graph
author: FirstName LastName

flowchart LR
    A[Hard] -->|Text| B(Round)
    B --> C{Decision}
```

[aloisklink]

Directives are in JSON format, so having front matter in YAML could be a little confusing

JSON is valid YAML, so it should be easy to migrate directives to YAML, if we want to support that.

without encapsulating in --- as it would still be above the actual diagram definition

It makes sense if we're using YAML for a few reasons:

• Everybody else does it for YAML headers (e.g. https://docusaurus.io/docs/markdown-features#front-matter)
• --- is part of the YAML and RFC 2046 spec, and I believe you can tell a YAML parser to just scan the bit between ---s (not 100% sure).
• It's easier to code that way, we can just say that if a mermaid definition starts with a ---, run the YAML pre-parser, else continue.
  • It will also make syntax highlighting for mermaid code easier!

Basically, using --- and YAML should hopefully mean less code to write!

[sidharthv96]

+1 for yaml header.
We'll have to draft a spec for what goes into the yaml.

[aloisklink]

We'll have to draft a spec for what goes into the yaml.

It might be worth specifying this using https://json-schema.org/, since YAML can almost always be converted to JSON (I think the only issue is with circular references, but those are very rare).

We could then use https://www.npmjs.com/package/json-schema-to-typescript to easily convert these into TypeScript types.

And we could use those JSON Schemas for input validation, and print good errors if something is invalid (e.g. what Webpack does).

If I have time, I'm considering moving the Mermaid config to JSON Schema too, since the way it is currently documented is incompatible with the new Vitepress website.

[aloisklink]

Looks amazing! And you've made this PR pretty small too, which means it's a pretty good change!

Some minor nit-picks from me, but they're mostly just nit-picks.

The only required change is to add a test for a YAML that contains --- on it's own line, e.g.:

title: |
    My multi-line string
    ---
    ^^^ Has an end-document string in it in it's own line

It's a rare edge-case, but we might run into it if we make a flowchart mermaid and we want to add a YAML front-matter example to a description :)

We should also change the PR title to saying something about YAML frontmatters. E.g. something like: Add YAML frontmatter to set titles in diagrams. It might also be worth copying some of #3706 (comment) to the main PR description with a quick example.

[aloisklink]

Can you add a test for a string with a --- in it on it's own line, e.g like the following YAML:

title: |
    My multi-line string
    ---
    ^^^ Has an end-document string in it in it's own line
description: Some other junk here
# but what about a comment that has a --- in it?

It might be that the REGEX gets confused by it.

If it's too difficult to handle, it might just be easier to use a library to parse the front-matter like https://github.com/jonschlinkert/gray-matter, although that project hasn't been updated for a while, so 🤷

[MasonM]

I don't see any way of supporting that syntax without serious compromises:

1. Disallowing leading whitespace before the delimiters would work (that's what gray-matter does), but leading whitespace before a diagram is very common, and disallowing it means you can't indent a diagram. e.g.

   mermaid/demos/flowchart.html

   Lines 19 to 24 in 70f0247

   	<pre class="mermaid">
   	---
   	title: This is a complicated flow
   	---
   	graph LR
   	accTitle: This is a complicated flow

2. Changing the (.*?) to (.*) isn't an option because that will break compatibility with diagrams containing ---
3. Trying to match the indent level of the closing delimiter with that of the opening delimiter won't work because leading whitespace is stripped in

   mermaid/packages/mermaid/src/mermaid.ts

   Lines 149 to 153 in 70f0247

   	// transforms the html to pure text
   	txt = utils
   	.entityDecode(txt)
   	.trim()
   	.replace(/<br\s*\/?>/gi, '<br/>');

Do you have any other ideas? If not, I think we should consider this an edge case and not support it

[aloisklink]

Disallowing leading whitespace before the delimiters would work

My gut feeling is that this is the approach we should go for (as that's that is what YAML spec v1.0 section 4.3.1 says), but you are right that many people do indent the diagrams.

We could add a dedent to https://github.com/mermaid-js/mermaid/blob/70f024735b57ac5b29b5c339ac871078ab46fa5a/packages/mermaid/src/mermaid.ts#LL149-L153. It would only apply to HTML however.

We could also add dedent inside parse(str) somewhere (or other functions that takes the Mermaid string directly), e.g. right before:

mermaid/packages/mermaid/src/mermaidAPI.ts

Line 81 in 09ed41b

const diagram = new Diagram(text, parseError);

But that's not super required. People calling the JavaScript API can use dedent themselves if they want to use indented strings and the YAML frontmatter, e.g.

import { dedent } from 'ts-dedent';
const myDiagram = dedent`
  ---
  title: This YAML is indented by two spaces.
  ---
  flowchart
    "This diagram is indented by two spaces."
`;

const svg = mermaid.render("mermaid-render-element", myDiagram);

[MasonM]

Well, alright, if you're sure that's the way to go, I implemented that in a11ab3d

[aloisklink]

if you're sure that's the way to go

I'm not fully sure! But I'd rather have the parser be too strict at the the beginning, since we can always loosen things later without it being a breaking change. It's much harder to make something stricter later on, as that would be a breaking change.

But I'd be be happy for people to disagree or if they think there's a better way to do things!

I am slightly leaning towards also adding a dedent before parsing the Mermaid HTML in mermaid.init/mermaid.initThrowsErrors so that Mermaid diagrams indented in HTML work, but not for JS code in mermaid.parse. Edit PR created for this in #3859

But that could be a decision for a future PR :)

[knsv]

When adding diagrams in html you inherit the indentation from the tag. This looks better in HTML:

    <pre id="diagram" class="mermaid">
      ---
      title: Example Git diagram
      ---
      gitGraph
        commit
        commit
    </pre>

than:

    <pre id="diagram" class="mermaid">
---
title: Example Git diagram
---
gitGraph
  commit
  commit
    </pre>

Even though it is less compliant with the standard.

I don't see the multi-line issue as a big problem though. My guess is that the user impact of indenting will affect far, far more users then the lack of ability to use --- in the beginning of a a multiline row.

I lean towards the more loose application, better to let the very few users that wants to have the --- in the multiline comment fail instead of irritating the users that want to indent.

But Alois has a fair point. Once we let the cat out of the box, it is out. Lets start with this but I suspect that we need to discuss this more and that we might might want to reverse the a11ab3d commit. That can be outside this PR though. Good job @MasonM Solid!

[aloisklink]

👍 This is a good idea!

[aloisklink]

I'm happy with this!

For other reviewers, currently, the YAML must not be indented, e.g.:

<!-- This is fine, no indentation on the YAML --!>
<pre class="mermaid">
---
title: My title
---
flowchart
  my flowchart
</pre>

<!-- This is not fine, there is indentation on the YAML --!>
<pre class="mermaid">
    ---
    title: My title
    ---
    flowchart
        my flowchart
</pre>

It may be worth using dedent to automatically remove the indentation from Mermaid code in HTML, but I'm happy for that to be added in a different PR. See discussion in #3706 (comment) for more info.

[MasonM]

@knsv Could you please review this? Unfortunately, I have very limited additional free time to spend on this, so if this isn't merged soon, I'll have to close it.

[knsv]

Will review today

[knsv]

Approving this but lets add dedent. I like that idea. Solves the indentation issue that will annoy some users!