Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nox -s vendoring -- --upgrade fails on Windows #9462

Closed
pfmoore opened this issue Jan 16, 2021 · 8 comments · Fixed by #9537
Closed

nox -s vendoring -- --upgrade fails on Windows #9462

pfmoore opened this issue Jan 16, 2021 · 8 comments · Fixed by #9537
Labels
type: bug A confirmed bug or unintended behavior type: maintenance Related to Development and Maintenance Processes

Comments

@pfmoore
Copy link
Member

pfmoore commented Jan 16, 2021

Environment

  • pip version: 20.3.3
  • Python version: 38.
  • OS: Windows

Description

Using nox -s vendoring -- --upgrade on Windows, causes extra newlines to be added to vendor.txt, resulting in the process failing.

Expected behavior
The process works on Windows.

How to Reproduce
nox -s vendoring -- --upgrade on a Windows system.

Output

>nox -s vendoring -- --upgrade
nox > Running session vendoring
nox > Re-using existing virtual environment at .nox\vendoring.
nox > python -m pip install vendoring>=0.3.0
nox > vendoring update . appdirs
Load configuration... Done!
Updating requirements... Done!
nox > vendoring update . CacheControl
Load configuration... Done!
Updating requirements...
  [(2, ''), (4, ''), (6, ''), (8, ''), (10, ''), (12, ''), (14, ''), (16, ''), (18, ''), (20, ''), (22, ''), (24, ''), (26, ''), (28, ''), (30, ''), (32, ''), (34, ''), (36, ''), (38, ''), (40, ''), (42, ''), (44, ''), (46, '')]
nox > Session vendoring raised exception ValueError('not enough values to unpack (expected 2, got 1)')
Traceback (most recent call last):
  File "c:\users\gustav\.local\pipx\venvs\nox\lib\site-packages\nox\sessions.py", line 549, in execute
    self.func(session)
  File "c:\users\gustav\.local\pipx\venvs\nox\lib\site-packages\nox\_decorators.py", line 53, in __call__
    return self.func(*args, **kwargs)
  File "D:\Work\Projects\pip\noxfile.py", line 182, in vendoring
    for inner_name, inner_version in pinned_requirements(vendor_txt):
  File "D:\Work\Projects\pip\noxfile.py", line 167, in pinned_requirements
    one, two = line.split("==", 1)
ValueError: not enough values to unpack (expected 2, got 1)
nox > Session vendoring failed.

I suspect the issue is two-fold: the nox session doesn't handle blank lines in vendor.txt, but also something (possibly the vendoring tool?) is writing unnecessary blank lines?
@pfmoore
Copy link
Member Author

pfmoore commented Jan 16, 2021

Ping @pradyunsg

@pradyunsg
Copy link
Member

pradyunsg commented Jan 16, 2021 •
edited
Loading

What's the value of line here?

@pradyunsg pradyunsg added type: bug A confirmed bug or unintended behavior type: maintenance Related to Development and Maintenance Processes labels Jan 16, 2021
@pradyunsg pradyunsg mentioned this issue Jan 16, 2021
Closed
@pfmoore
Copy link
Member Author

pfmoore commented Jan 16, 2021

"". Because we have extra blank lines in the file.

@pfmoore pfmoore mentioned this issue Jan 29, 2021
Closed
@henryiii
Copy link
Contributor

This works on macOS, with current versions of everything, so it could be Windows only, perhaps? (pipx run nox -s vendoring -- --upgrade)

@pfmoore
Copy link
Member Author

pfmoore commented Jan 29, 2021

It is Windows only, it's a line ending issue as far as I can tell.

henryiii added a commit to henryiii/vendoring that referenced this issue Jan 29, 2021
@henryiii
fix: Windows double line spacing fix
e565929 
The current version does not work on Windows, as reported by @pfmoore in pypa/pip#9462. Files opened in text mode should not have the native line ending applied - Python normalizes this for you. Quick demo of problem:

```python
from pathlib import Path
from os import linesep

txt = Path("tmp.txt")
with txt.open("w", encoding="utf-8") as f:
    f.writelines(f"{p}{linesep}" for p in range(3))
with txt.open(encoding="utf-8") as f:
    print(f.read())
```

Produces empty lines between outputs, but only Windows (due to the separator there).
@henryiii henryiii mentioned this issue Jan 29, 2021
Merged
@uranusjr uranusjr mentioned this issue Jan 30, 2021
Merged
@uranusjr
Copy link
Member

I submitted #9537 to make the parsing part more resilient on pip’s end.

@henryiii
Copy link
Contributor

The problem seen by @pfmoore should be fixed by pradyunsg/vendoring#27. Vendoring is using os.linesep on a file opened in text mode, which causes double line breaks on Windows. Do you want this to silently pass and succeed if there are extra blank lines? They shouldn't be there in the first place. :)

PS: That is a question, I don't know the correct answer. If it never gets saved with blank lines, then it likely doesn't matter.

@uranusjr
Copy link
Member

I do think the parser should be less brittle. Empty lines are not unreasonable to allow in a file listing requirements no matter what.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 1, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
type: bug A confirmed bug or unintended behavior type: maintenance Related to Development and Maintenance Processes
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make vendoring task respond well to empty lines uranusjr/pip
4 participants
@uranusjr @pfmoore @pradyunsg @henryiii