result is garbled after format

[Miaoxingren]

my code:

  var diffpatcher = jsondiffpatch.create({ textDiff: { minLength: 10 } })

  var diffData = diffpatcher.diff('喵星人最可爱最可爱最可爱', '喵星人meow最可爱最可爱最可爱')

  var diffView = jsondiffpatch.formatters.html.format(diffData, '喵星人最可爱最可爱最可爱')

and I'm getting this:

  <div class="jsondiffpatch-delta jsondiffpatch-textdiff"><div class="jsondiffpatch-value"><ul class="jsondiffpatch-textdiff"><li><div class="jsondiffpatch-textdiff-location"><span class="jsondiffpatch-textdiff-line-number">1</span><span class="jsondiffpatch-textdiff-char">11</span></div><div class="jsondiffpatch-textdiff-line"><span class="jsondiffpatch-textdiff-context">��人</span><span class="jsondiffpatch-textdiff-added">meow</span><span class="jsondiffpatch-textdiff-context">��������</span></div></li></ul></div></div>

[benjamine]

thanks, this would make a nice unit test.
text diffs are done using neil fraser's google diff match patch lib, I'm not sure it has good support for unicode, but it might be good to test if the problem is there: https://www.npmjs.com/package/diff-match-patch

[geyang]

unicode handling is notoriously bad in javascript. https://mathiasbynens.be/notes/javascript-unicode

…

[Miaoxingren]

It works fine using diff-match-patch:

var dmp = new diff_match_patch();
var diff = dmp.diff_main('喵星人最可爱最可爱最可爱.', '喵星人meow最可爱最可爱最可爱.');
dmp.diff_prettyHtml(diff);

result:

<span>喵星人</span><ins style="background:#e6ffe6;">meow</ins><span>最可爱最可爱最可爱.</span>

[geyang]

I think for CKJ it is fine, but not some emojis and glyphs of some other languages.

[Miaoxingren]

I found google diff match patch lib use decodeURI and encodeURI to process data. so I change unescape to decodeURI and it seems to work.

   /* global unescape */
    let piece = pieces[pieceIndex];
    context.out(
      `<span class="jsondiffpatch-textdiff-${piece.type}">${htmlEscape(
        decodeURI(piece.text)
      )}</span>`
    );

code:

var diffpatcher = jsondiffpatch.create({ textDiff: { minLength: 10 } })
var diffData = diffpatcher.diff('喵星人最可爱最可爱最可爱', '喵星人meow最可爱最可爱最可爱')
var diffView = jsondiffpatch.formatters.html.format(diffData, '喵星人最可爱最可爱最可爱')

result:

<div class="jsondiffpatch-delta jsondiffpatch-textdiff"><div class="jsondiffpatch-value"><ul class="jsondiffpatch-textdiff"><li><div class="jsondiffpatch-textdiff-location"><span class="jsondiffpatch-textdiff-line-number">1</span><span class="jsondiffpatch-textdiff-char">11</span></div><div class="jsondiffpatch-textdiff-line"><span class="jsondiffpatch-textdiff-context">喵星人</span><span class="jsondiffpatch-textdiff-added">meow</span><span class="jsondiffpatch-textdiff-context">最可爱最可爱最可</span></div></li></ul></div></div>

Is it ok to make this cahnge?

[benjamine]

on the html formatter? ok, seems to make sense, would you be interested in a PR (including a test case like that?). that would be ok, otherwise I can take a stab at the fix.

[Miaoxingren]

#217
：）