Merge pull request #338 from marp-team/slug

Make slug for each headings and assign to `id` attribute

CHANGELOG.md

@@ -2,6 +2,11 @@
 
 ## [Unreleased]
 
+### Added
+
+- Assign auto-generated slug to `id` attribute of each headings ([#299](https://github.com/marp-team/marp-core/issues/299), [#338](https://github.com/marp-team/marp-core/pull/338))
+- `slug` constructor option ([#338](https://github.com/marp-team/marp-core/pull/338))
+
 ### Changed
 
 - Upgrade Node.js and dependent packages ([#336](https://github.com/marp-team/marp-core/pull/336))

README.md

@@ -47,9 +47,10 @@ _We will only explain features extended in marp-core._ Please refer to [Marpit f
   - Enabled [inline SVG slide](https://marpit.marp.app/inline-svg) and [loose YAML parsing](https://marpit-api.marp.app/marpit#Marpit) by default.
 
 * **CommonMark**
-  - For making secure, we will deny most of HTML tags used in Markdown (`<br>` is only allowed by default).
+  - For making secure, we will deny most of HTML tags used in Markdown by default. Allowed HTML tags are `<br>` only for now.
   - Support [table](https://github.github.com/gfm/#tables-extension-) and [strikethrough](https://github.github.com/gfm/#strikethrough-extension-) syntax, based on [GitHub Flavored Markdown](https://github.github.com/gfm/).
   - Line breaks in paragraph will convert to `<br>` tag.
+  - Slugification for headings (assining auto-generated `id` attribute for `<h1>` - `<h6>`) is enabled by default.
 
 ---
 
@@ -229,6 +230,7 @@ const marp = new Marp({
     source: 'cdn',
     nonce: 'xxxxxxxxxxxxxxx',
   },
+  slug: false,
 
   // It can be included Marpit constructor options
   looseYAML: false,
@@ -264,7 +266,7 @@ By passing `object`, you can set the allowlist to specify allowed tags and attri
 }
 ```
 
-Marp core allows only `<br>` tag by default, that is defined in [`Marp.html`](https://github.com/marp-team/marp-core/blob/5c3593320f1c1234f3b2556ecd1ff1f91d69c77a/src/marp.ts#L45).
+Marp core allows only `<br>` tag by default. That is defined in [a readonly `html` member in `Marp` class](https://github.com/marp-team/marp-core/blob/38fb33680c5837f9c48d8a88ac94b9f0862ab6c7/src/marp.ts#L34).
 
 > Whatever any option is selected, `<!-- HTML comment -->` and `<style>` tags are always parsed by Marpit for directives / tweaking style.
 
@@ -310,8 +312,8 @@ Enable or disable minification for rendered CSS. `true` by default.
 
 Setting about an injected helper script for the browser context. This script is necessary for applying [WebKit polyfill](https://github.com/marp-team/marpit-svg-polyfill) and rendering [auto-scaled elements](#auto-scaling-features) correctly.
 
-- **`true`**: Inject the inline helper script into after the last of slides. (default)
-- **`false`**: Not inject helper script. Developer must execute a helper script manually, exported in [`@marp-team/marp-core/browser`](src/browser.ts). Requires bundler such as [webpack](https://webpack.js.org/). It's suitable to the fully-controlled tool such as [Marp Web](https://github.com/marp-team/marp-web).
+- **`true` (default)**: Inject the inline helper script into after the last of slides.
+- **`false`**: Don't inject helper script. Developer must execute a helper script manually, exported in [`@marp-team/marp-core/browser`](src/browser.ts). Requires bundler such as [webpack](https://webpack.js.org/). It's suitable to the fully-controlled tool such as [Marp Web](https://github.com/marp-team/marp-web).
 
 You can control details of behavior by passing `object`.
 
@@ -321,6 +323,27 @@ You can control details of behavior by passing `object`.
 
 * **`nonce`**: _`string`_ - Set [`nonce` attribute](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script#attr-nonce) of `<script>`.
 
+### `slug`: _`boolean` | `function` | `object`_
+
+Configure slugification for headings. By default, Marp Core tries to make the slug by the similar way to GitHub. It should be compatible with [Markdown Language Server](https://code.visualstudio.com/blogs/2022/08/16/markdown-language-server).
+
+- **`true` (default)**: Assign auto-generated `id` attribute from the contents of `<h1>`-`<h6>` headings.
+- **`false`**: Disable auto-assigning slug to headings.
+- _`function`_: Set the custom slugifier function, that takes one argument: the content of the heading. It must return a generated slug string.
+
+You can control details of behavior by passing `object`.
+
+- **`slugifier`**: _`function`_ - Set the custom slugifier function.
+- **`postSlugify`**: _`function`_ - Set the post-process function after generated a slug. The function takes 2 arguments, the string of generated slug and the index of the same slug, and must return a string for assigning to `id` attribute of the heading.
+
+  By default, Marp Core applies the post-process to avoid assigning duplicated `id`s in the document: `` (slug, index) => (index > 0 ? `${slug}-${index}` : slug) ``
+
+  Assigning the custom post-process function is also helpful to append the custom prefix and suffix to the generated slug: `` (slug, i) => `prefix:${slug}:${i}` ``
+
+> Take care not to confuse Marp Core's `slug` option and [Marpit's `anchor` option](https://marpit-api.marp.app/marpit#:~:text=Description-,anchor,-boolean%20%7C%20Marpit). `slug` is for the Markdown headings, and `anchor` is for the slide elements.
+>
+> `Marp` class is extended from `Marpit` class so you can customize both options in the constructor. To fully disable auto-generated `id` attribute, set both options as `false`. (This is important to avoid breaking your Web application by user's Markdown contents)
+
 ## Contributing
 
 Are you interested in contributing? Please see [CONTRIBUTING.md](.github/CONTRIBUTING.md) and [the common contributing guideline for Marp team](https://github.com/marp-team/.github/blob/master/CONTRIBUTING.md).

src/marp.ts

@@ -12,6 +12,7 @@ import * as mathPlugin from './math/math'
 import minifyPlugins from './prebundles/postcss-minify-plugins'
 import * as scriptPlugin from './script/script'
 import * as sizePlugin from './size/size'
+import * as slugPlugin from './slug/slug'
 
 export interface MarpOptions extends Options {
   emoji?: emojiPlugin.EmojiOptions
@@ -26,6 +27,7 @@ export interface MarpOptions extends Options {
   math?: mathPlugin.MathOptions
   minifyCSS?: boolean
   script?: boolean | scriptPlugin.ScriptOptions
+  slug?: slugPlugin.SlugOptions
 }
 
 export class Marp extends Marpit {
@@ -48,6 +50,7 @@ export class Marp extends Marpit {
       math: true,
       minifyCSS: true,
       script: true,
+      slug: true,
       ...opts,
       emoji: {
         shortcode: 'twemoji',
@@ -84,6 +87,7 @@ export class Marp extends Marpit {
       .use(autoScalingPlugin.markdown)
       .use(sizePlugin.markdown)
       .use(scriptPlugin.markdown)
+      .use(slugPlugin.markdown)
   }
 
   // eslint-disable-next-line @typescript-eslint/no-unused-vars

src/slug/slug.ts

@@ -0,0 +1,119 @@
+import marpitPlugin from '@marp-team/marpit/plugin'
+import type { Marp } from '../marp'
+
+export type Slugifier = (text: string) => string
+export type PostSlugify = (slug: string, index: number) => string
+
+export type SlugOptions = boolean | Slugifier | SlugOptionsObject
+
+type SlugOptionsObject = {
+  slugifier?: Slugifier
+  postSlugify?: PostSlugify
+}
+
+const textTokenTypes = [
+  'text',
+  'code_inline',
+  'image',
+  'html_inline',
+  'marp_emoji',
+  'marp_unicode_emoji',
+]
+
+const defaultPostSlugify: PostSlugify = (slug, index) =>
+  index > 0 ? `${slug}-${index}` : slug
+
+const parseSlugOptions = (
+  options: SlugOptions
+): false | Required<SlugOptionsObject> => {
+  if (options === false) return false
+
+  if (typeof options === 'function') {
+    return { slugifier: options, postSlugify: defaultPostSlugify }
+  }
+
+  const defaultSlugOptions: Required<SlugOptionsObject> = {
+    slugifier: githubSlugify,
+    postSlugify: defaultPostSlugify,
+  }
+
+  return options === true
+    ? defaultSlugOptions
+    : { ...defaultSlugOptions, ...options }
+}
+
+export const markdown = marpitPlugin((md) => {
+  const marp: Marp = md.marpit
+
+  md.core.ruler.push('marp_slug', (state) => {
+    const opts = parseSlugOptions(marp.options.slug ?? true)
+    if (!opts) return
+
+    const slugs = new Map<string, number>()
+
+    for (const token of state.tokens) {
+      if (token.type === 'marpit_slide_open') {
+        const tokenId = token.attrGet('id')
+        if (tokenId != null) slugs.set(tokenId, 0)
+      }
+    }
+
+    let targetHeading
+    let targetHeadingContents: any[] = []
+
+    for (const token of state.tokens) {
+      if (!targetHeading && token.type === 'heading_open') {
+        targetHeading = token
+        targetHeadingContents = []
+      } else if (targetHeading) {
+        if (token.type === 'heading_close') {
+          let slug = token.attrGet('id')
+
+          if (slug == null) {
+            slug = opts.slugifier(
+              targetHeadingContents
+                .map((contentToken) => {
+                  if (contentToken.type === 'inline') {
+                    return contentToken.children
+                      .map((t) => {
+                        if (t.hidden) return ''
+                        if (textTokenTypes.includes(t.type)) return t.content
+
+                        return ''
+                      })
+                      .join('')
+                  }
+
+                  return ''
+                })
+                .join('')
+            )
+          }
+
+          // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
+          const index = slugs.has(slug) ? slugs.get(slug)! + 1 : 0
+          targetHeading.attrSet('id', opts.postSlugify(slug, index))
+
+          slugs.set(slug, index)
+          targetHeading = undefined
+        } else if (!token.hidden) {
+          targetHeadingContents.push(token)
+        }
+      }
+    }
+  })
+})
+
+// Convert given text to GitHub-style slug. This is compatible with Markdown language service.
+export const githubSlugify: Slugifier = (text: string): string =>
+  encodeURI(
+    text
+      .trim()
+      .toLowerCase()
+      .replace(/\s+/g, '-')
+      .replace(
+        /[\][!/'"#$%&()*+,./:;<=>?@\\^{|}~`。，、；：？！…—·ˉ¨‘’“”々～‖∶＂＇｀｜〃〔〕〈〉《》「」『』．〖〗【】（）［］｛｝]/g,
+        ''
+      )
+      .replace(/(?:^-+|-+$)/, '')
+  )

test/marp.ts

@@ -133,7 +133,7 @@ describe('Marp', () => {
 
       describe('with false', () => {
         const emoji: EmojiOptions = { unicode: false }
-        const instance = marp({ emoji })
+        const instance = marp({ emoji, slug: false })
 
         it("does not inject Marp's unicode emoji renderer", () =>
           expect(
@@ -146,7 +146,7 @@ describe('Marp', () => {
 
       describe('with true', () => {
         const emoji: EmojiOptions = { unicode: true }
-        const instance = marp({ emoji })
+        const instance = marp({ emoji, slug: false })
 
         it("injects Marp's unicode emoji renderer", () =>
           expect(
@@ -706,6 +706,117 @@ function matchwo(a,b)
     })
   })
 
+  describe('slug option', () => {
+    it('makes slugs for headings by default', () => {
+      const { html } = marp().render('# a\n\n---\n\n## b\n\n---\n\n### a')
+      const $ = load(html)
+
+      expect($('h1').attr('id')).toBe('a')
+      expect($('h2').attr('id')).toBe('b')
+      expect($('h3').attr('id')).toBe('a-1')
+    })
+
+    describe('with undefined (default)', () => {
+      it('makes slugs for headings', () => {
+        const { html } = marp({ slug: undefined }).render('# a\n\n---\n\n## b')
+        const $ = load(html)
+
+        expect($('h1').attr('id')).toBe('a')
+        expect($('h2').attr('id')).toBe('b')
+      })
+    })
+
+    describe('with false', () => {
+      it('does not make slugs for headings', () => {
+        const { html } = marp({ slug: false }).render('# a\n\n---\n\n## b')
+        const $ = load(html)
+
+        expect($('h1').attr('id')).toBeUndefined()
+        expect($('h2').attr('id')).toBeUndefined()
+      })
+    })
+
+    describe('with custom slugifier', () => {
+      it('makes slugs for headings by custom slugifier', () => {
+        const slugifier = (s: string) => `custom:${s}`
+        const { html } = marp({ slug: slugifier }).render('# abc')
+        const $ = load(html)
+
+        expect($('h1').attr('id')).toBe('custom:abc')
+      })
+    })
+
+    describe('with option object', () => {
+      it('allows slugifier option', () => {
+        const slugifier = (s: string) => `custom:${s}`
+
+        expect(marp({ slug: { slugifier } }).render('# abc').html).toBe(
+          marp({ slug: slugifier }).render('# abc').html
+        )
+      })
+
+      it('allows postSlugify option, to deal with duplicate slugs', () => {
+        const postSlugify = (s: string, i: number) => `${'-'.repeat(i)}${s}`
+        const { html } = marp({ slug: { postSlugify } }).render(
+          '# abc\n\n---\n\n## abc\n\n---\n\n### abc'
+        )
+        const $ = load(html)
+
+        expect($('h1').attr('id')).toBe('abc')
+        expect($('h2').attr('id')).toBe('-abc')
+        expect($('h3').attr('id')).toBe('--abc')
+      })
+    })
+
+    describe('with duplicated slug with slide anchor', () => {
+      it('adds index to duplicated slug', () => {
+        const { html } = marp().render('# 1')
+        const $ = load(html)
+
+        expect($('h1').attr('id')).toBe('1-1')
+      })
+
+      it('recongizes custom anchor generation', () => {
+        const { html } = marp({ anchor: (i) => `slide-${i + 1}` }).render(
+          '# Slide 1'
+        )
+        const $ = load(html)
+
+        expect($('h1').attr('id')).toBe('slide-1-1')
+      })
+    })
+
+    describe('with <!--fit--> annotation', () => {
+      it('ignores the annotation comment in the slug', () => {
+        const { html } = marp().render('# <!--fit--> a')
+        const $ = load(html)
+
+        expect($('h1').attr('id')).toBe('a')
+      })
+    })
+
+    describe('when the heading tokens has surrounded a non inline token', () => {
+      it('ignores non inline elements in the slug', () => {
+        const { html } = marp()
+          .use((md) => {
+            md.core.ruler.before('marp_slug', 'marp_test', (state) => {
+              for (let i = 0; i < state.tokens.length; i += 1) {
+                if (state.tokens[i].type === 'heading_open') {
+                  const token = new state.Token('test', '', 0)
+                  token.content = 'test'
+                  state.tokens.splice(i + 1, 0, token)
+                }
+              }
+            })
+          })
+          .render('# abc')
+
+        const $ = load(html)
+        expect($('h1').attr('id')).toBe('abc')
+      })
+    })
+  })
+
   describe('Auto scaling', () => {
     describe('when fit comment keyword contains in heading (Fitting header)', () => {
       const baseMd = '# <!--fit--> fitting'