Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean package names found in python catalogers #1417

Merged
merged 1 commit into from
Dec 21, 2022

Conversation

wagoodman
Copy link
Contributor

This PR fixes a couple of bugs:

  • When there are formatting directives in setup.py package names (for dynamically crafted require lines, such as %s or {}) we should skip these lines
  • When there are empty name or version fields for requirements.txt files (such as ==1.2.3) we should skip these lines

@wagoodman wagoodman added the bug Something isn't working label Dec 20, 2022
@wagoodman wagoodman self-assigned this Dec 20, 2022
@wagoodman wagoodman requested a review from a team December 20, 2022 15:46
syft/pkg/cataloger/python/parse_requirements.go Outdated Show resolved Hide resolved
@github-actions
Copy link

github-actions bot commented Dec 20, 2022

Benchmark Test Results

Benchmark results from the latest changes vs base branch
name                                                       old time/op    new time/op    delta
ImagePackageCatalogers/alpmdb-cataloger-2                    13.3ms ± 1%    13.6ms ± 2%    ~     (p=0.114 n=4+4)
ImagePackageCatalogers/ruby-gemspec-cataloger-2              1.55ms ± 6%    1.54ms ± 2%    ~     (p=0.841 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2            3.89ms ± 1%    3.86ms ± 0%    ~     (p=0.222 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2    1.28ms ± 1%    1.28ms ± 1%    ~     (p=0.690 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         888µs ± 1%     890µs ± 1%    ~     (p=0.421 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                    1.01ms ± 1%    1.03ms ± 1%  +2.04%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpm-db-cataloger-2                    1.50ms ± 1%    1.54ms ± 1%  +2.23%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      17.0ms ± 2%    17.4ms ± 2%    ~     (p=0.056 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.03ms ± 1%    1.04ms ± 2%    ~     (p=0.151 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2          7.44µs ± 2%    7.49µs ± 1%    ~     (p=0.421 n=5+5)
ImagePackageCatalogers/dotnet-deps-cataloger-2               1.60ms ± 1%    1.63ms ± 3%    ~     (p=0.421 n=5+5)
ImagePackageCatalogers/portage-cataloger-2                    848µs ± 1%     857µs ± 1%    ~     (p=0.095 n=5+5)
ImagePackageCatalogers/sbom-cataloger-2                      5.35ms ± 1%    5.43ms ± 1%  +1.47%  (p=0.016 n=5+5)
ImagePackageCatalogers/binary-cataloger-2                    4.74ms ± 1%    4.76ms ± 1%    ~     (p=0.151 n=5+5)

name                                                       old alloc/op   new alloc/op   delta
ImagePackageCatalogers/alpmdb-cataloger-2                    5.26MB ± 0%    5.26MB ± 0%    ~     (p=0.841 n=5+5)
ImagePackageCatalogers/ruby-gemspec-cataloger-2               205kB ± 0%     205kB ± 0%    ~     (p=0.548 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2             962kB ± 0%     962kB ± 0%    ~     (p=0.222 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     217kB ± 0%     218kB ± 0%    ~     (p=0.421 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         159kB ± 0%     159kB ± 0%    ~     (p=0.056 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                     199kB ± 0%     200kB ± 0%  +0.14%  (p=0.016 n=5+5)
ImagePackageCatalogers/rpm-db-cataloger-2                     303kB ± 0%     303kB ± 0%    ~     (p=0.063 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      3.49MB ± 0%    3.49MB ± 0%    ~     (p=0.841 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                      182kB ± 0%     182kB ± 0%    ~     (p=0.222 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2          1.12kB ± 0%    1.12kB ± 0%    ~     (all equal)
ImagePackageCatalogers/dotnet-deps-cataloger-2                375kB ± 0%     375kB ± 0%    ~     (p=0.841 n=5+5)
ImagePackageCatalogers/portage-cataloger-2                    139kB ± 0%     139kB ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/sbom-cataloger-2                       722kB ± 0%     722kB ± 0%  +0.01%  (p=0.008 n=5+5)
ImagePackageCatalogers/binary-cataloger-2                     656kB ± 0%     656kB ± 0%  +0.01%  (p=0.008 n=5+5)

name                                                       old allocs/op  new allocs/op  delta
ImagePackageCatalogers/alpmdb-cataloger-2                     85.7k ± 0%     85.7k ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/ruby-gemspec-cataloger-2               4.25k ± 0%     4.25k ± 0%    ~     (all equal)
ImagePackageCatalogers/python-package-cataloger-2             16.5k ± 0%     16.5k ± 0%    ~     (p=0.087 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     5.50k ± 0%     5.50k ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         3.33k ± 0%     3.33k ± 0%    ~     (all equal)
ImagePackageCatalogers/dpkgdb-cataloger-2                     4.47k ± 0%     4.47k ± 0%    ~     (all equal)
ImagePackageCatalogers/rpm-db-cataloger-2                     8.12k ± 0%     8.12k ± 0%    ~     (all equal)
ImagePackageCatalogers/java-cataloger-2                       57.5k ± 0%     57.5k ± 0%    ~     (p=0.325 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                      5.23k ± 0%     5.23k ± 0%    ~     (p=0.095 n=5+4)
ImagePackageCatalogers/go-module-binary-cataloger-2            38.0 ± 0%      38.0 ± 0%    ~     (all equal)
ImagePackageCatalogers/dotnet-deps-cataloger-2                7.12k ± 0%     7.12k ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/portage-cataloger-2                    3.58k ± 0%     3.58k ± 0%    ~     (all equal)
ImagePackageCatalogers/sbom-cataloger-2                       24.4k ± 0%     24.4k ± 0%    ~     (all equal)
ImagePackageCatalogers/binary-cataloger-2                     22.2k ± 0%     22.2k ± 0%    ~     (all equal)

@wagoodman wagoodman force-pushed the clean-python-package-parsers branch from f06afa1 to 8a36d2c Compare December 21, 2022 18:10
@wagoodman wagoodman enabled auto-merge (squash) December 21, 2022 18:11
@wagoodman wagoodman merged commit 5dd726f into main Dec 21, 2022
@wagoodman wagoodman deleted the clean-python-package-parsers branch December 21, 2022 18:31
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
Signed-off-by: Alex Goodman <[email protected]>

Signed-off-by: Alex Goodman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants