Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move unpack_file_url() out of download.py #6861

Closed
cjerdonek opened this issue Aug 11, 2019 · 4 comments
Closed

Move unpack_file_url() out of download.py #6861

cjerdonek opened this issue Aug 11, 2019 · 4 comments
Labels
auto-locked Outdated issues that have been locked by automation C: download About fetching data from PyPI and other sources help wanted For requesting inputs from other members of the community type: refactor Refactoring code

Comments

@cjerdonek
Copy link
Member

cjerdonek commented Aug 11, 2019

As a follow-on to PR #6859, unpack_file_url() and the functions it depends on should be moved out of download.py into a new module IMO since this function doesn't do any downloading or require network access -- just copying / unpacking on the file system:

def unpack_file_url(
link, # type: Link
location, # type: str
download_dir=None, # type: Optional[str]
hashes=None # type: Optional[Hashes]
):
# type: (...) -> None
"""Unpack link into location.
If download_dir is provided and link points to a file, make a copy
of the link file inside download_dir.
"""

Does anyone have any thoughts on the name of the new module? I think it can probably be inside utils. It is shutil-like (needing things like shutil.copytree()) and so feels higher-level than utils/filesystem.py.

Doing this will also let us remove wheel.py's import dependency on download.py.

This is also related to issue #6813 ("Breakout pip._internal.download to a pip._internal.network sub-package").

Posted in its original form by @cjerdonek in #6859 (comment)

@triage-new-issues triage-new-issues bot added the S: needs triage Issues/PRs that need to be triaged label Aug 11, 2019
@cjerdonek cjerdonek added C: download About fetching data from PyPI and other sources type: refactor Refactoring code labels Aug 11, 2019
@triage-new-issues triage-new-issues bot removed the S: needs triage Issues/PRs that need to be triaged label Aug 11, 2019
@cjerdonek cjerdonek added the help wanted For requesting inputs from other members of the community label Aug 11, 2019
@cjerdonek
Copy link
Member Author

Functions in misc.py like unpack_file(), untar_file(), and unzip_file() would probably be good moved to this new module, too:

def unpack_file(
filename, # type: str
location, # type: str
content_type, # type: Optional[str]
link # type: Optional[Link]
):

Maybe unpacking.py?

ivenk pushed a commit to ivenk/pip that referenced this issue Aug 11, 2019
@ivenk
Copy link

ivenk commented Aug 11, 2019

I will try and submit a PR later on.

@chrahunt
Copy link
Member

After looking at it more, I think it would be better to move the networking/http-specific stuff out of download and into something in utils (or another top-level folder under which we can split up networking). unpack_file_url and unpack_http_url are essentially the same function - moving them away from each other may make it harder to see that and refactor.

@cjerdonek
Copy link
Member Author

Since you were able to remove wheel.py's dependence on download.py without moving unpack_file_url(), I'm okay with closing this issue.

@lock lock bot added the auto-locked Outdated issues that have been locked by automation label Oct 21, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Oct 21, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
auto-locked Outdated issues that have been locked by automation C: download About fetching data from PyPI and other sources help wanted For requesting inputs from other members of the community type: refactor Refactoring code
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants