Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unreadable diff for large HTML snapshot #7393

Open
6 tasks done
rChaoz opened this issue Jan 31, 2025 · 1 comment · May be fixed by #7400
Open
6 tasks done

Unreadable diff for large HTML snapshot #7393

rChaoz opened this issue Jan 31, 2025 · 1 comment · May be fixed by #7400
Labels
p3-minor-bug An edge case that only affects very specific usage (priority)

Comments

@rChaoz
Copy link

rChaoz commented Jan 31, 2025 •
edited
Loading

Describe the bug

In some cases, Vitest produces a huge, unreadable diff.

Reproduction

My test is:

// generate a large HTML file
const html = generateEmail(...)
expect(html).toMatchSnapshot()

If the generated email does not change, the test passes. But, if the HTML changes even by a tiny bit (a single character), the Vitest diff says every single file is wrong:

...
- <tr>
+       <tr>
-   <td>&nbsp;</td>
+         <td>&nbsp;</td>
-   <td class="container">
+         <td class="container">
-     <div class="content">
+           <div class="content">
-       {{#if preheader}}
+             {{#if preheader}}
-         <span class="preheader">{{{preheader}}}</span>
+               <span class="preheader">{{{preheader}}}</span>
-       {{/if}}
+             {{/if}}
...

So, it's useless in finding the diff. This happens for any kind of snapshots (separate file, inline). Additionally, it took me 5 hours to realize that the diffs it does find (indentation) don't actually matter and, in fact, there was a single line wrong, somewhere in the 300-line long output.

Full test - copy & paste, then run with vitest

POV: You are me. Try to figure out why the test fails.

import { expect, test } from "vitest"

test("test", async () => {
    const email = "<!--suppress HtmlDeprecatedAttribute -->\n<html lang=\"en\">\n  <!--suppress HtmlRequiredTitleElement -->\n  <head>\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\" />\n    <meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\n    <!--suppress CssUnusedSymbol -->\n    {{! prettier-ignore }}\n    <style media=\"all\">\n      /* Global styles & resets */\n      body {\n        font-family: Helvetica, sans-serif;\n        -webkit-font-smoothing: antialiased;\n        font-size: 16px;\n        line-height: 1.3;\n        -ms-text-size-adjust: 100%;\n        -webkit-text-size-adjust: 100%;\n      }\n\n      table {\n        border-collapse: separate;\n        mso-table-lspace: 0;\n        mso-table-rspace: 0;\n        width: 100%;\n      }\n\n      table td {\n        font-family: Helvetica, sans-serif;\n        font-size: 16px;\n        vertical-align: top;\n      }\n\n      p {\n        font-family: Helvetica, sans-serif;\n        font-size: 16px;\n        font-weight: normal;\n        margin: 0 0 16px 0;\n      }\n\n      a {\n        color: #0867ec;\n        text-decoration: underline;\n      }\n\n      /* Body & container */\n\n      body {\n        background-color: #f4f5f6;\n        margin: 0;\n        padding: 0;\n      }\n\n      .body {\n        background-color: #f4f5f6;\n        width: 100%;\n      }\n\n      .container {\n        margin: 0 auto !important;\n        max-width: 720px;\n        padding: 24px 0 0 0;\n        width: 600px;\n      }\n\n      .content {\n        box-sizing: border-box;\n        display: block;\n        margin: 0 auto;\n        max-width: 600px;\n        padding: 0;\n      }\n\n      /* Header, footer, main */\n\n      .preheader {\n        color: transparent;\n        display: none;\n        height: 0;\n        max-height: 0;\n        max-width: 0;\n        opacity: 0;\n        overflow: hidden;\n        mso-hide: all;\n        visibility: hidden;\n        width: 0;\n      }\n\n      .main {\n        background: #ffffff;\n        border: 1px solid #eaebed;\n        border-radius: 16px;\n        width: 100%;\n      }\n\n      .wrapper {\n        box-sizing: border-box;\n        padding: 24px;\n      }\n\n      .footer {\n        clear: both;\n        padding-block: 16px;\n        text-align: center;\n        width: 100%;\n      }\n\n      .footer td,\n      .footer p,\n      .footer span,\n      .footer a {\n        color: #9a9ea6;\n        font-size: 16px;\n        text-align: center;\n      }\n\n      /* Buttons */\n      .btn {\n        box-sizing: border-box;\n        min-width: 100% !important;\n        width: 100%;\n      }\n\n      .btn > tbody > tr > td {\n        padding-bottom: 16px;\n      }\n\n      .btn table {\n        width: auto;\n      }\n\n      .btn table td {\n        background-color: #ffffff;\n        border-radius: 4px;\n        text-align: center;\n      }\n\n      .btn a {\n        background-color: #ffffff;\n        border: solid 2px #0867ec;\n        border-radius: 4px;\n        box-sizing: border-box;\n        color: #0867ec;\n        cursor: pointer;\n        display: inline-block;\n        font-size: 16px;\n        font-weight: bold;\n        margin: 0;\n        padding: 12px 24px;\n        text-decoration: none;\n        text-transform: capitalize;\n      }\n\n      .btn-primary table td {\n        background-color: #0867ec;\n      }\n\n      .btn-primary a {\n        background-color: #0867ec;\n        border-color: #0867ec;\n        color: #ffffff;\n      }\n\n      @media all {\n        .btn-primary table td:hover {\n          background-color: #217fff !important;\n        }\n        .btn-primary a:hover {\n          background-color: #217fff !important;\n          border-color: #217fff !important;\n        }\n      }\n\n      /* Responsive styles */\n\n      @media only screen and (max-width: 720px) {\n        .main p,\n        .main td,\n        .main span {\n          font-size: 16px !important;\n        }\n        .wrapper {\n          padding: 8px !important;\n        }\n        .content {\n          padding: 0 !important;\n        }\n        .container {\n          padding: 0 !important;\n          padding-top: 8px !important;\n          width: 100% !important;\n        }\n        .main {\n          border-left-width: 0 !important;\n          border-radius: 0 !important;\n          border-right-width: 0 !important;\n        }\n        .btn table {\n          max-width: 100% !important;\n          width: 100% !important;\n        }\n        .btn a {\n          font-size: 16px !important;\n          max-width: 100% !important;\n          width: 100% !important;\n        }\n      }\n\n       /* Don't inline these styles, fixes for various e-mail viewers */\n      @media all {\n        .ExternalClass {\n          width: 100%;\n        }\n        .ExternalClass,\n        .ExternalClass p,\n        .ExternalClass span,\n        .ExternalClass font,\n        .ExternalClass td,\n        .ExternalClass div {\n          line-height: 100%;\n        }\n        #MessageViewBody a {\n          color: inherit;\n          text-decoration: none;\n          font-size: inherit;\n          font-family: inherit;\n          font-weight: inherit;\n          line-height: inherit;\n        }\n      }\n    </style>\n  </head>\n  <body>\n    <table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"body\">\n      <tr>\n        <td>&nbsp;</td>\n        <td class=\"container\">\n          <div class=\"content\">\n            {{#if preheader}}\n              <span class=\"preheader\">{{{preheader}}}</span>\n            {{/if}}\n\n            <table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"main\">\n              <tr>\n                <td class=\"wrapper\">\n                  {{{content}}}\n                  {{#if button}}\n                    <table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\" class=\"btn-primary btn\">\n                      <tbody>\n                        <tr>\n                          <td align=\"left\">\n                            <table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\">\n                              <tbody>\n                                <tr>\n                                  <td><a href=\"{{button.url}}\" target=\"_blank\">{{button.text}}</a></td>\n                                </tr>\n                              </tbody>\n                            </table>\n                          </td>\n                        </tr>\n                      </tbody>\n                    </table>\n                  {{/if}}\n                  {{{postContent}}}\n                </td>\n              </tr>\n            </table>\n\n            <div class=\"footer\">\n              <table role=\"presentation\" border=\"0\" cellpadding=\"0\" cellspacing=\"0\">\n                <tr>\n                  <td class=\"content-block\">\n                    {{#if baseUrl}}\n                      <a href=\"{{baseUrl}}\">{{baseUrl}}</a>\n                    {{/if}}\n                    {{#if unsubscribeUrl}}\n                      <br />\n                      Don't like receiving these e-mails?\n                      <a href=\"{{unsubscribeUrl}}\" target=\"_blank\">Change notification preferences</a>.\n                    {{/if}}\n                    {{#if contactEmail}}\n                      <br />\n                      Contact us at\n                      <a href=\"mailto:{{contactEmail}}\">{{contactEmail}}</a>\n                    {{/if}}\n                  </td>\n                </tr>\n              </table>\n            </div>\n          </div>\n        </td>\n        <td>&nbsp;</td>\n      </tr>\n    </table>\n  </body>\n</html>";
    expect(email).toMatchInlineSnapshot(`
      "<!--suppress HtmlDeprecatedAttribute -->
      <html lang="en">
        <!--suppress HtmlRequiredTitleElement -->
        <head>
          <meta name="viewport" content="width=device-width, initial-scale=1.0" />
          <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
          <!--suppress CssUnusedSymbol -->
          {{! prettier-ignore }}
          <style media="all">
            /* Global styles & resets */
            body {
              font-family: Helvetica, sans-serif;
              -webkit-font-smoothing: antialiased;
              font-size: 16px;
              line-height: 1.3;
              -ms-text-size-adjust: 100%;
              -webkit-text-size-adjust: 100%;
            }

            table {
              border-collapse: separate;
              mso-table-lspace: 0;
              mso-table-rspace: 0;
              width: 100%;
            }

            table td {
              font-family: Helvetica, sans-serif;
              font-size: 16px;
              vertical-align: top;
            }

            p {
              font-family: Helvetica, sans-serif;
              font-size: 16px;
              font-weight: normal;
              margin: 0 0 16px 0;
            }

            a {
              color: #0867ec;
              text-decoration: underline;
            }

            /* Body & container */

            body {
              background-color: #f4f5f6;
              margin: 0;
              padding: 0;
            }

            .body {
              background-color: #f4f5f6;
              width: 100%;
            }

            .container {
              margin: 0 auto !important;
              max-width: 720px;
              padding: 24px 0 0 0;
              width: 600px;
            }

            .content {
              box-sizing: border-box;
              display: block;
              margin: 0 auto;
              max-width: 600px;
              padding: 0;
            }

            /* Header, footer, main */

            .preheader {
              color: transparent;
              display: none;
              height: 0;
              max-height: 0;
              max-width: 0;
              opacity: 0;
              overflow: hidden;
              mso-hide: all;
              visibility: hidden;
              width: 0;
            }

            .main {
              background: #ffffff;
              border: 1px solid #eaebed;
              border-radius: 16px;
              width: 100%;
            }

            .wrapper {
              box-sizing: border-box;
              padding: 24px;
            }

            .footer {
              clear: both;
              padding-block: 16px;
              text-align: center;
              width: 100%;
            }

            .footer td,
            .footer p,
            .footer span,
            .footer a {
              color: #9a9ea6;
              font-size: 16px;
              text-align: center;
            }

            /* Buttons */
            .btn {
              box-sizing: border-box;
              min-width: 100% !important;
              width: 100%;
            }

            .btn > tbody > tr > td {
              padding-bottom: 16px;
            }

            .btn table {
              width: auto;
            }

            .btn table td {
              background-color: #ffffff;
              border-radius: 4px;
              text-align: center;
            }

            .btn a {
              background-color: #ffffff;
              border: solid 2px #0867ec;
              border-radius: 4px;
              box-sizing: border-box;
              color: #0867ec;
              cursor: pointer;
              display: inline-block;
              font-size: 16px;
              font-weight: bold;
              margin: 0;
              padding: 12px 24px;
              text-decoration: none;
              text-transform: capitalize;
            }

            .btn-primary table td {
              background-color: #0867ec;
            }

            .btn-primary a {
              background-color: #0867ec;
              border-color: #0867ec;
              color: #ffffff;
            }

            @media all {
              .btn-primary table td:hover {
                background-color: #217fff !important;
              }
              .btn-primary a:hover {
                background-color: #217fff !important;
                border-color: #217fff !important;
              }
            }

            /* Responsive styles */

            @media only screen and (max-width: 720px) {
              .main p,
              .main td,
              .main span {
                font-size: 16px !important;
              }
              .wrapper {
                padding: 8px !important;
              }
              .content {
                padding: 0 !important;
              }
              .container {
                padding: 0 !important;
                padding-top: 8px !important;
                width: 100% !important;
              }
              .main {
                border-left-width: 0 !important;
                border-radius: 0 !important;
                border-right-width: 0 !important;
              }
              .btn table {
                max-width: 100% !important;
                width: 100% !important;
              }
              .btn a {
                font-size: 16px !important;
                max-width: 100% !important;
                width: 100% !important;
              }
            }

            /* Don't inline these styles, fixes for various e-mail viewers */
            @media all {
              .ExternalClass {
                width: 100%;
              }
              .ExternalClass,
              .ExternalClass p,
              .ExternalClass span,
              .ExternalClass font,
              .ExternalClass td,
              .ExternalClass div {
                line-height: 100%;
              }
              #MessageViewBody a {
                color: inherit;
                text-decoration: none;
                font-size: inherit;
                font-family: inherit;
                font-weight: inherit;
                line-height: inherit;
              }
            }
          </style>
        </head>
        <body>
          <table role="presentation" border="0" cellpadding="0" cellspacing="0" class="body">
            <tr>
              <td>&nbsp;</td>
              <td class="container">
                <div class="content">
                  {{#if preheader}}
                    <span class="preheader">{{{preheader}}}</span>
                  {{/if}}

                  <table role="presentation" border="0" cellpadding="0" cellspacing="0" class="main">
                    <tr>
                      <td class="wrapper">
                        {{{content}}}
                        {{#if button}}
                          <table role="presentation" border="0" cellpadding="0" cellspacing="0" class="btn-primary btn">
                            <tbody>
                              <tr>
                                <td align="left">
                                  <table role="presentation" border="0" cellpadding="0" cellspacing="0">
                                    <tbody>
                                      <tr>
                                        <td><a href="{{button.url}}" target="_blank">{{button.text}}</a></td>
                                      </tr>
                                    </tbody>
                                  </table>
                                </td>
                              </tr>
                            </tbody>
                          </table>
                        {{/if}}
                        {{{postContent}}}
                      </td>
                    </tr>
                  </table>

                  <div class="footer">
                    <table role="presentation" border="0" cellpadding="0" cellspacing="0">
                      <tr>
                        <td class="content-block">
                          {{#if baseUrl}}
                            <a href="{{baseUrl}}">{{baseUrl}}</a>
                          {{/if}}
                          {{#if unsubscribeUrl}}
                            <br />
                            Don't like receiving these e-mails?
                            <a href="{{unsubscribeUrl}}" target="_blank">Change notification preferences</a>.
                          {{/if}}
                          {{#if contactEmail}}
                            <br />
                            Contact us at
                            <a href="mailto:{{contactEmail}}">{{contactEmail}}</a>
                          {{/if}}
                        </td>
                      </tr>
                    </table>
                  </div>
                </div>
              </td>
              <td>&nbsp;</td>
            </tr>
          </table>
        </body>
      </html>"
    `)
})

Used Package Manager

yarn

Validations
@rChaoz rChaoz changed the title Unreadable diff for large snapshots Unreadable diff for large HTML snapshot Jan 31, 2025
@hi-ogawa
Copy link
Contributor

hi-ogawa commented Feb 1, 2025 •
edited
Loading

Moving the repro on stackblitz https://stackblitz.com/edit/vitest-dev-vitest-xauauggn?file=test%2Frepro.test.ts

Probably the issue is in this heuristics for object snapshot indent detection:

vitest/packages/snapshot/src/port/utils.ts

Lines 178 to 183 in 6082d7f

export function prepareExpected(expected?: string): string | undefined {
function findStartIndent() {
// Attempts to find indentation for objects.
// Matches the ending tag of the object.
const matchObject = /^( +)\}\s+$/m.exec(expected || '')
const objectIndent = matchObject?.[1]?.length

Simpler repro is here https://stackblitz.com/edit/vitest-dev-vitest-va4arntu?file=test%2Frepro.test.ts

import { expect, test } from 'vitest';

test('test', async () => {
  const email = `\
aaaaaa
    xxxxxx {
    }

    yyyyyy {
    }
`;
  // see error diff after prepending 'x'
  expect('#' + email).matchSnapshot();
});

Image

It looks like there is another code handling indentation (for inline snapshot). I'm not sure which is for what purpose.

vitest/packages/snapshot/src/port/inlineSnapshot.ts

Line 186 in c82387d

export function stripSnapshotIndentation(inlineSnapshot: string): string {

@hi-ogawa hi-ogawa added p3-minor-bug An edge case that only affects very specific usage (priority) and removed pending triage labels Feb 1, 2025
@hi-ogawa hi-ogawa linked a pull request Feb 1, 2025 that will close this issue
Draft
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
p3-minor-bug An edge case that only affects very specific usage (priority)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(snapshot): tweak indent normalization hi-ogawa/vitest
2 participants
@hi-ogawa @rChaoz