Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make import symbol limit configurable #28

Closed
fpq473 opened this issue Apr 28, 2022 · 2 comments
Closed

Make import symbol limit configurable #28

fpq473 opened this issue Apr 28, 2022 · 2 comments

Comments

@fpq473
Copy link

fpq473 commented Apr 28, 2022

Hello, thanks for this useful tool.

I came across a scenario where delvewheel-repair does not include all necessary DLLs. This was caused by pefile returning a truncated list of imports due to a limit on the number of imports parsed. I observe in CadQuery/cadquery#1048 (comment) that increasing pefile.MAX_IMPORT_SYMBOLS at least partially resolves some issues.

My ideas for delvewheel are:

  • Allow MAX_IMPORT_SYMBOLS to be specified at the command-line, e.g. delvewheel repair --max_import_symbols=1000000 leads to pefile.MAX_IMPORT_SYMBOLS = 1000000 before parsing;

  • Display all pefile warnings to the user. Warning messages are accumulated on the PE object (example) and can be retrieved by PE.get_warnings().

adang1345 added a commit that referenced this issue Apr 28, 2022
When the -vv argument is specified, warnings from pefile will be
printed. This will help diagnose related to pefile such as #28.
@adang1345
Copy link
Owner

I have fixed this issue and #29 and in the newly released version 0.0.22.

I added an option to display warnings from pefile.

I fixed the issue of exceeding the number of imported symbols by using the import_dllnames_only parameter at https://github.com/erocarrera/pefile/blob/9addb08b9906266ad8835a5ac80c7ec623cff758/pefile.py#L3513. When this is set to True, we avoid parsing the symbols that are imported from each DLL and thus don't go over the pefile.MAX_IMPORT_SYMBOLS limit.

I ran a quick test against a copy of OCP.cp310-win_amd64.pyd from https://github.com/roipoussiere/OCP/actions/runs/2173754949, and it looks like delvewheel is able to list out all its dependencies now.

@fpq473
Copy link
Author

fpq473 commented Apr 28, 2022

Thank you very much for the exceedingly quick and helpful changes. Using import_dllnames_only brings the loading of data directories from around two minutes to under 10 ms. Also thanks for testing against OCP and making the new release - that will make our builds very convenient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants