Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows use mkl static lib. #1790

Merged
merged 3 commits into from
Apr 24, 2024
Merged

Windows use mkl static lib. #1790

merged 3 commits into from
Apr 24, 2024

Conversation

xuhancn
Copy link
Contributor

@xuhancn xuhancn commented Apr 19, 2024

From pytorch issue: pytorch/pytorch#124009 I found libtorch seems use shared mkl lib and missing some mkl dll files.

  1. Currently pytorch Linux already use static mkl lib.
  2. Windows can also support static mkl lib, I have validated as update build guide to use mkl-static. pytorch#116946

So, this PR will switch pytorch to use static mkl lib.
I have tested PR on my local PC:
image

@seemethere seemethere requested a review from atalman April 22, 2024 19:24
Copy link
Contributor

@malfet malfet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@malfet malfet merged commit 9f33088 into pytorch:main Apr 24, 2024
2 checks passed
xuhancn added a commit to xuhancn/pytorch_builder that referenced this pull request Apr 24, 2024
@xuhancn
Copy link
Contributor Author

xuhancn commented Apr 24, 2024

@malfet Please merge this PR to fix pip confilct: #1797

malfet added a commit that referenced this pull request Apr 24, 2024
@malfet
Copy link
Contributor

malfet commented Apr 24, 2024

@xuhancn reverted, please do not hesitate to open new PR, but this time test it using ciflow/binaries against pytorch, similar to how pytorch/pytorch#124353 tests #1787

@xuhancn
Copy link
Contributor Author

xuhancn commented Apr 24, 2024

@xuhancn reverted, please do not hesitate to open new PR, but this time test it using ciflow/binaries against pytorch, similar to how pytorch/pytorch#124353 tests #1787

@malfet I don‘t have permittion to submit dummy builder tester PR.
image

@malfet
Copy link
Contributor

malfet commented Apr 24, 2024

@malfet I don‘t have permittion to submit dummy builder tester PR. image

@xuhancn Not sure what's your ask here. This is your own repo and your PAT, you'll need to tweak its scope to permit pushes or create a different token.

@xuhancn
Copy link
Contributor Author

xuhancn commented Apr 24, 2024

@malfet I don‘t have permittion to submit dummy builder tester PR. image

@xuhancn Not sure what's your ask here. This is your own repo and your PAT, you'll need to tweak its scope to permit pushes or create a different token.

Let me seek some solution.

malfet pushed a commit that referenced this pull request Apr 26, 2024
resubmit #1790 with fix PR #1797.

From pytorch issue: pytorch/pytorch#124009 I found libtorch seems use shared mkl lib and missing some mkl dll files.
1. Currently pytorch Linux already use static mkl lib.
2. Windows can also support static mkl lib, I have validated as pytorch/pytorch#116946

Tested in https://github.com/pytorch/pytorch/actions/runs/8836875904/job/24264643410
pytorchmergebot pushed a commit to pytorch/pytorch that referenced this pull request Jun 28, 2024
# Background
I have fixed pytorch Windows missing mkl shared library dependency issue: #124009
The solution is change torch_cpu module static link mkl library:
1. pytorch static link mkl PR: #124925
2. builder install mkl static library: pytorch/builder#1790

Double confirmed current build is using mkl static link: #124009 (comment)

# Goal
Remove setup.py `install_requires` will install mkl shared lib on pytorch Windows. It is not required now, due to we have static linked it.
It will reduce the pytorch install network traffic and avoid install useless mkl shared library package.

Pull Request resolved: #129493
Approved by: https://github.com/malfet
pytorchbot pushed a commit to pytorch/pytorch that referenced this pull request Jun 28, 2024
# Background
I have fixed pytorch Windows missing mkl shared library dependency issue: #124009
The solution is change torch_cpu module static link mkl library:
1. pytorch static link mkl PR: #124925
2. builder install mkl static library: pytorch/builder#1790

Double confirmed current build is using mkl static link: #124009 (comment)

# Goal
Remove setup.py `install_requires` will install mkl shared lib on pytorch Windows. It is not required now, due to we have static linked it.
It will reduce the pytorch install network traffic and avoid install useless mkl shared library package.

Pull Request resolved: #129493
Approved by: https://github.com/malfet

(cherry picked from commit 424068d)
atalman pushed a commit to pytorch/pytorch that referenced this pull request Jun 28, 2024
[Windows] remove mkl shared library dependency. (#129493)

# Background
I have fixed pytorch Windows missing mkl shared library dependency issue: #124009
The solution is change torch_cpu module static link mkl library:
1. pytorch static link mkl PR: #124925
2. builder install mkl static library: pytorch/builder#1790

Double confirmed current build is using mkl static link: #124009 (comment)

# Goal
Remove setup.py `install_requires` will install mkl shared lib on pytorch Windows. It is not required now, due to we have static linked it.
It will reduce the pytorch install network traffic and avoid install useless mkl shared library package.

Pull Request resolved: #129493
Approved by: https://github.com/malfet

(cherry picked from commit 424068d)

Co-authored-by: Xu Han <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants