enable --ansi by default on Windows

[ericfrederich]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Feature Request

The --ansi option should be default on Windows.

On a fresh install of Windows 10 with a fresh install of the Windows version of Python 3.9.0
Install Poetry via:

py -m pip install --user pipx
py -m pipx ensurepath
# need to get a new console now for pipx to be on PATH
pipx install poetry

Now in both PowerShell and in cmd.exe running pipx --ansi produces nice colors.

Is there a reason not to default --ansi on Windows?

[TBBle]

This is actually decided by an underlying library (clikit), and is probably due to a bug there: sdispater/clikit#35.

As a workaround, you can set the ANSICON env-var to something non-empty to force ANSI support.

[passionsfrucht]

I hope I'm not totally wrong in this thread, but I have actually seen different behaviours between ANSICON and --ansi, as seen here:

Hunting down the problem made me a bit crazy as it seems to be an endless loop about conemu, windows, python, git-bash for windows, termcap and numerous other potential culprits.

To break the loop I just made the comment here to show the interesting different behaviour between ANSICON and --ansi.

My environment surely is interesting as it is but in a strange circle of modern Windows environment, too:

• Windows Terminal Preview
  Version: 1.6.10272.0
• WSL2 with fish as log-in shell
• tmux
• new fish shell in tmux
• in there git-bash for windows is started

$ poetry --version
Poetry version 1.1.4
lord@Bratwurst ~/configs/python windows *6 !1 ?1
$ which poetry
/c/Users/lord/.poetry/bin/poetry
lord@Bratwurst ~/configs/python windows *6 !1 ?1
$ env | rg TERM
TERM=screen-256color
WSLENV=WT_SESSION::WT_PROFILE_ID:TERM/w

But I can reproduce such behaviour in simpler set-ups, too.

If there is any more info I can provide or a pointer in the right direction I'd be glad to provide or take it.
Actually it is not only poetry showing such or similar errors, but so far only poetry did this difference between the environment setting and command line option. But ANSICON was new to me in any way.

So thanks for this new bit of knowledge and for the really great piece of software!

[TBBle]

I can't reproduce your result under PowerShell 7.1.1, under the same Windows Terminal Version:

I also couldn't reproduce it under git-bash git version 2.28.0.windows.1:

--ansi and ANSICON=1 should have the same effect, marking the output streams as decorated. Edit: I just noticed these are looking at cleo, not clikit, as I have looked at before. I've lost track of which code Poetry is hitting, as those two codebases have been refactored recently. I'm not aware that they behave differently yet though.

That said, I'm not totally clear from your comment... Are you testing this inside WSL? I assume so based having WSLENV set. That isn't Windows as far as Python is concerned, so ANSICON won't have any effect.

I'd suggest trying to get a simpler reproduction case, in terms of the applications involved in handling the output stream. Due to the recency of real PTY support in Windows, older Windows apps would often mangle ANSI codes as they passed through, and git-bash being Cygwin based has definitely been one of those in the past.

[passionsfrucht]

Hi again, I did some further tests, now with a bit of simplified setup.

As I said my setup is weird due to other codebases I'm usually working on. My typical shell setup is in WSL, but for different reasons I need to drop into msys bash from time to time.

That said I tried to simplify the test setup, with similar results in git-bash, but not in powershell.

powershell:

bash (msys):

This time git-bash was started directly from Windows Terminal, via:

            {
                "guid": "{f8346d0a-b08a-48db-a12a-abb322e5b278}",
                "commandline": "\"C:\\Program Files\\Git\\bin\\bash.exe\" -i -l",
                "historySize": 9001,
                "icon": "ms-appx:///ProfileIcons/{0caa0dad-35be-5f56-a8ff-afceeeaa6101}.png",
                "name": "Git-Bash",
                "snapOnInput": true,
                "startingDirectory": "%USERPROFILE%"
                //"background": "#332222"
            },

Same result when git-bash is started from the start menu, in the 'classic' terminal, with whatever settings they use there exactly:

I do not think that the newer git version is a culprit, I have seen similar behaviours long before 2.28.0, but only IIRC as haven't tested that right now.

[TBBle]

I had a quick check, and on a different machine (with git git version 2.29.2.windows.2 and Windows Terminal 1.5.10271.0) it does reproduce.

So I suspect this is a bug in git-bash (or rather, its Cygwin-based MSYS2 layer) introduced with 2.29. I would guess related to the Cygwin 3.1.7 upgrade mentioned there, since it seems something is corrupting U+001b into U+2190. It's weird that it only affects ANSICON but not --ansi, so I'm wondering if perhaps Cygwin is also changing behaviour based on the ANSICON env-var? I can't see ANSICON in the source though.

A quick test by piping to less shows that less sees the <ESC> correctly in both --ansi and ANSICON=1 cases, so unless clikit is doing something very weird in checking for tty output, it must be something in Cygwin. Piping through cat also works, so it's not less specifically.

Confounding the above, my MSYS2 installation with runtime msys2-runtime 3.1.5-3 (i.e. the same Cygwin base as git for windows 2.28) also reproduces this issue on this machine.

I'll have to check on my other machine, but now I'm wondering if there's something else system-wide which is changing the results. It's not Windows Terminal, since my home machine is running the same version you used.

And one last test, it's not ANSICON specifically, as I can use ConEmuANSI=ON or Term=xterm and see the same bug. Other random env-vars don't have this effect.

The only thing that comes to mind, is that by passing in these env-vars, the code that tries to activate VT processing is bypassed. I haven't checked, but perhaps when using --ansi, that code is called, but the result is ignored.

And I therefore guess that for Cygwin-based console apps (as opposed to Powershell or CMD), VT processing is not enabled by default? Although that would imply that detection should work, since the underlying bug that blocks that is that if VT processing is already enabled, clickit returns "ANSI not supported". I'm not sure what piping through less is changing.

This wouldn't technically be a bug, because we're setting env-vars that claim things that aren't actually true (i.e. that we're running under ansicon, ConEmu or xterm.

I'd have to do some more debugging to track this down, but I think getting sdispater/clikit#35 fixed first would be better, since it's possible that that will function correctly under Cygwin anyway, removing the need for the ANSICON workaround.

Edit: Recently, cleo was refactored to not depend on clikit, but the buggy code at fault in sdispater/clikit#35 was copied across. So this may now be a cleo bug to fix.

Edit: Back on my home machine, it definitely doesn't reproduce under Windows Terminal, or with MSYS2 under mintty. Git bash on cmd appears to show the right ANSI codes, but the terminal is not processing them:

[passionsfrucht]

I just want to say thanks for the effort, I did not have the time to look further into the issue right now, but FYI:

$ git --version
git version 2.30.1.windows.1

Did not bring any relief.

[johnjohnk2]

Hi @passionsfrucht @ericfrederich @TBBle. Thanks a lot for your work. I have this issue so far ->no colors at all in Arch, some colors in powershell, both started from windows terminal. Not sure about the best solution to implement at this step as I read many different bug fix on this issue...
Thanks a lot for your help

[TBBle]

@johnjohnk2 I don't see Poetry in your screenshots. Are you on the correct bug-tracker? I sounds like you are looking for Windows Terminal support?

[johnjohnk2]

@TBBle Hi Thanks a lot for your time. Not sure to understant as yes it is WSL2 but I came here from your 4 Jan commet where you said that the bug comes from clikit, isn' it ?

[TBBle]

The issue in Poetry, yes, appears to actually be a bug in clickit, a Python library used by Poetry.

You appear to just be having trouble with bash (under WSL2?) and PowerShell under Windows Terminal, so no connection to Poetry or clikit. You should follow up according to the Windows Terminal support instructions, as this is the wrong place to ask.

[johnjohnk2]

Ok OK sorry for my misunderstanding !

[Jhraumg]

I could reproduce it from my Windows Terminal + git bash and poetry 1.1.6 (while it worked flawlessly with the same git bash and cmder)

With poetry version 1.2.0a1, it seems all fixed (thanks to #3618 I guess):

• poetry displays color automaticaly on Windows Terminal
• poetry does accept global parameter before command (which could help aliasing poetry to poetry --ansi, but it is non necessary anyway 😉 )

[KuSi833]

I've installed Poetry version 1.2.0a1, but the color still doesn't display by default in the Windows Terminal

[passionsfrucht]

Hey again, unfortunately

Did not bring the fix for me either.

But by being able to alias poetry to poetry --ansi I was able to mitigate this for me in a manner that I can move on.

If I can help to debug some deeper causes, I'd be happy to help!

[TBBle]

I also see Poetry 1.2.0a2 failing to auto-detect ANSI support:

• Windows Terminal Preview 1.10.1933.0
• PowerShell 7.1.3
• Python 3.9.6
• pipx 0.16.4

The ANSICON env-var workaround still successfully enables ANSI.

(Ignore the warning about Poetry already existing, that's noticing my Poetry 1.1.7 installation)

I also checked pipx run --spec poetry==1.2.0a1 poetry just in case it was working in a1 but broken in a2, but the result was the same.

I just checked, and the faulty code that was in clickit (see sdispater/clikit#35 (comment)) is still unchanged in cleo 1.0.0a4, so the bug is still in the underlying library: Cleo only auto-detects ANSI for Windows if it can turn on ENABLE_VIRTUAL_TERMINAL_PROCESSING, but not if it is already turned on, as is the case for Windows Terminal or other ConPTY-supporting terminal emulators.

I raised a new issue at python-poetry/cleo#104

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.