Single quotes in content following bold text get curled the wrong way

[dknauss]

Apostrophes in content get curled in the wrong direction (to the right) in the screen output (not within the editor) when used after bold text. E.g., He's will be rendered like this:

Probably related to ye olde wptexturize filter: #37754, #2199.

[kathrynwp]

Hi @dknauss - thanks for your report. So far I haven't been able to replicate this. Here's a front-end screenshot:

My test setup:

• WordPress 6.0
• Gutenberg 13.6.0 - active or inactive
• No other plugins active
• Twenty Twenty-Two
• Paragraph block

Could you please provide detailed steps to replicate, as well as your WP and Gutenberg version, and theme name/version? Could you provide a link to a page or post where the problem is visible? Thanks!

[dknauss]

Hi @kathrynwp, the output you are showing is actually (typographically) incorrect and exactly the same as my example. Yours is just much more subtle due to the font used by Twenty Twenty-two. I was fooled by this as well when I first replied. This comment has been heavily edited. :-)

I also realized "apostrophe" is not exactly the right term to use, so I updated the issue title as well.

The issue I've identified has to do with single quotation marks — Unicode Character “'” (U+0027) being potentially transformed into Unicode Character “‘” (U+2018) or Unicode Character “’” (U+2019).

(U+0027) is identified as an "apostrophe" in PostScript corresponding to the Unicode HTML entity ', ' '. That character is not appearing in the test cases I've shared.

Using the same test setup as you (a paragraph block in WP 6.0 with no other plugins, with or without Gutenberg 13.6.0 active), this is the content I started with in the editor — in the first screenshot below. The Avant-Garde theme is active there but just in the first example. The text, formatting, and ' characters are the same in the editor (and the rendered page source) in all themes tested here.

Underlying markup in the editor:

Post output with the Avant Garde theme showing misdirected single quotation marks:

Post output with the Twenty Twenty theme showing misdirected single quotation marks:

Rendered HTML source with the Twenty Twenty theme activated — single quotation marks are bad, as shown above:

Rendered HTML source and post output with the Twenty Twenty-One theme activated — single quotation marks are bad — but it's much harder to detect visually. It fooled me at first:

Rendered HTML source with the Twenty Twenty-Two theme activated — again, the wrong output but masked by the subtle differences of the font in use:

In all these themes the character encoding is UTF-8, and the ‘ and ’ HTML entities are created in the same pattern, presumably by wptexturize(), under which ’ always seems to follow the </strong> tag.

I think that should be/is? considered a bug in wptexturize.

Discriminating accurately between ''' characters intended as opening or closing single quotes in all possible cases (which vary by language) isn't likely to be an option, but if they're immediately adjacent to an inline HTML tag like <strong></strong> with no space between them, it would make sense to have wptexturize ignore the ' by default.

Is there any chance of core making text filtering optional (there's a 12-year-old plugin for that) and/or simpler for users to modify with exceptions? :-)

There's some neat history behind all this... The infelicities of wptexturize are almost as old as WordPress itself. Its source, Texturize, was one of Matt's early projects for b2, later forked into WordPress... Here's the whole fascinating thread.

[kathrynwp]

@dknauss Thank you for the additional clarifications, super helpful – and tricky indeed! I was able to see the issue clearly in Twenty Twenty, with or without Gutenberg 13.6.0 active.

I added a label requesting technical feedback from a developer to see what they have to say about your suggestions. Thanks again!

[dknauss]

Thank you, @kathrynwp — you made me think it through more deeply and carefully. 👍

[dknauss]

Just dropping this here for the #WP20 historical interest: Matt talking about Texturize in WordPress 0.72.

[ellatrix]

This is a super old bug for wptexturize in core: https://core.trac.wordpress.org/ticket/18549

Unfortunately there's nothing we can do here in the Gutenberg repo, this must be fixed in the core repo. I'd recommend following that ticket and bringing it back up.