Most XML entities not being decoded by reader

[kernwig]

Describe the bug
New XML entity decoding in v3.0 only works on & < and >, but not others. Interestingly, the XML builder does encode a few other characters like '.

To Reproduce

   const xmlResponse = create('<ssid>Me &amp; Myself&apos;s WiFi &copy;&#x1D306;</ssid>');
   const networkResponse = xmlResponse.end({format: 'object'});
   return networkResponse.ssid;

The output string is Me & Myself's WiFi ©𝌆

The first two entities were not touched, while the last two got double-encoded.

Expected behavior
Expected result to be Me & Myself's WiFi ©𝌆.

Version:

• node.js: 12.16.1
• xmlbuilder2: 3.0.1

Additional context
Related to #88

[oozcitak]

I'll take a look at ' it should be decoded as '. Also 𝌆 should be decoded as 𝌆. But & needs to be escaped when serialized as & per spec. © is not an XML predefined entity, so it should throw a parser exception. Browsers throw on undefined entities, e.g. with Chrome:

const str = `<ssid>Me &amp; Myself&apos;s WiFi &copy;&#x1D306;</ssid>`;
const p = new DOMParser();
const s = new XMLSerializer();
s.serializeToString(p.parseFromString(str, "application/xml"));

<ssid>
  <parsererror xmlns=\"http://www.w3.org/1999/xhtml\" style=\"display: block; white-space: pre; border: 2px solid #c77; padding: 0 1em 0 1em; margin: 1em; background-color: #fdd; color: black\">
    <h3>This page contains the following errors:</h3>
      <div style=\"font-family:monospace;font-size:12px\">
         error on line 1 at column 41: Entity 'copy' not defined\n
      </div>
    <h3>Below is a rendering of the page up to the first error.</h3>
  </parsererror>
</ssid>

without ©:

const str = `<ssid>Me &amp; Myself&apos;s WiFi &#x1D306;</ssid>`;
const p = new DOMParser();
const s = new XMLSerializer();
s.serializeToString(p.parseFromString(str, "application/xml"));

<ssid>Me &amp; Myself's WiFi 𝌆</ssid>

[oozcitak]

Instead of throwing on unknown entities, I opted to silently ignore them. Your code should return:

Me & Myself's WiFi ©𝌆

[kernwig]

Thank you. My driving bug had to do with '. When I was working around the bug in xmlbuilder2 v2.x, I used the he library and tried this larger variety of characters just to be thorough. Looks like you are now covering the "predefined entities" of XML, not the massive list of "standard" entities in HTML 5. Reference

[kernwig]

@oozcitak When might I expect the next release with this fix?

[oozcitak]

Oh shoot sorry. Forgot to release. I'll prepare one tomorrow. Thanks for reminding.

[oozcitak]

Released v3.0.2 with the fix.

[kernwig]

Tried it. Looked at the code. It looks like the code should encode/decode &, but your unit test is not expecting to decode and I'm seeing that too.

& is the one character that absolutely must be encoded, since it's the start of the escape sequence. When decoding into a Javascript object, shouldn't all signs of XML encoding be gone so that it can be used in other contexts?

[kernwig]

I see that < and > also get re-encoded, but not '.

   const xmlResponse = create('<ssid>Me &amp; Myself&apos;s &lt;WiFi&gt;&#x1D306;</ssid>');
   const networkResponse = xmlResponse.end({format: 'object'});
   return networkResponse.ssid;

Expected result: Me & Myself's <WiFi>𝌆

Actual result with v3.0.2: Me & Myself's <WiFi>𝌆