Problem with encoding still exists on Mac OS X and Debian

[jonas]

From user: I still have this issue. Installed on mac with brew, apt-get on deb stable. Screenshot along.

References #72

[jonas]

@jaseemabid Hi, I think it was you who mentioned this. If you can provide a script to create this repo that would be really useful. I'd also like to know what version of tig you are using and against which ncurses library you link.

[jaseemabid]

@jonas The screenshot is from debian. tig installed from source 'coz the package was really old.

$ ncurses5-config --version => 5.9.20110404
$ tig --version => tig version 2.0.3-10-gb0e2c2d

Test script:

cd /tmp
mkdir foo
cd foo
touch init
git init
git add init
git commit -m  "αβψδεφγηιξκλμνοπρστθςχυζ"
echo " " > init
git commit init -m "😊 "

[jaseemabid]

Same thing on osx.

[jaseemabid]

The smiley character is something iterm can render when its typed in, but git or tig won't show it again. See output of the script when run.

[jonas]

Apart from the smiley character which I don't think it is possible to get ncurses to draw correctly, it looks like the OS X version is OK. This suggests that the version you compiled on Debian is not linked with ncursesw. As mentioned in the INSTALL file Ncurses with wide character support (ncursesw) is required to properly handle UTF-8 encoded strings.

The ncursesw package needs to be installed separately on Debian if I remember correctly. To configure the build to use the ncursesw package you can create a config.make file (see contrib/config.make for an example).

LDLIBS = -lncursesw
CPPFLAGS = -DHAVE_NCURSESW_CURSES_H

[jonas]

@jaseemabid Did you compile tig with ncursesw or not? I am asking because I cannot reproduce this locally. See my previous comment for more information.

[jaseemabid]

@jonas I think characters which cannot be rendered like the smiley should be replaced with a ? so that the alignment stays correct. Showing the unicode representation <U+12EF> might also be fine in most cases. Not displaying anything is confusing.

[jaseemabid]

The default git behaviour might be the best to copy.

$ git commit -m "😊 " 

commit 5110ce6fa48a15d161151fc51d5c876fb9f6ba4f
Author: Jaseem Abid <[email protected]>
Date:   Tue Nov 11 12:39:34 2014 +0530

    <U+1F60A>

[jaseemabid]

I dont see any change after compiling with nursesw. Can I verify if it was build with the correct headers? Maybe along with tig --version or tig --info?

[edi9999]

Using

LDLIBS = -lncursesw
CPPFLAGS = -DHAVE_NCURSESW_CURSES_H

in my config.make solves that issue. Great :-)

[guyzmo]

I have to add that the issue is still there with both v2.2 and -git versions on archlinux. When setting up the utf-8 formatting option in the tigrc it's working fine, with the neat ●─╮ or ●─┴─╯.

So ncursew is definitely there and working (even though it's being installed simply as ncurses). And even an ldd proves it:

ldd /usr/bin/tig
	linux-vdso.so.1 (0x00007ffe99de4000)
	libreadline.so.7 => /usr/lib/libreadline.so.7 (0x00007ff2c9c6d000)
	libncursesw.so.6 => /usr/lib/libncursesw.so.6 (0x00007ff2c9a01000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007ff2c9663000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff2c9ebb000)

To reproduce

% git init test
% cd test
% touch fubar
% git add fubar
% git ci -m "this part is visible 😀🍻 this part is not visible ☹🔥"
% git log --oneline
9b66f84 this part is visible 😀🍻 this part is not visible ☹🔥
% tig
…

Related

• in the archlinux tracker there's an issue about ncursesw
  • maybe is it a matter that there needs to be HAVE_NCURSESW_CURSES_H enabled with ncursesw/curses.h available on the system for the unicode log lines to be parsed correctly?
• there's been no report of unicode issues otherwise.

[jonas]

Fixed by @rolandwalker in #644

[guyzmo]

yaaay 🙌 thanks ♥ and cheers 🍻