Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow user to override path assumption for system-provided modules #9

Merged

Conversation

priv-kweihmann
Copy link
Contributor

by optionally using environment variables

Copy link
Member

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you ++ this makes sense.
Do you think it might be best to have a set of dedicated environment variables such as EXTRACTCODE_7Z_PATH, EXTRACTCODE_LIBARCHIVE_PATH and TYPECODE_LIBMAGIC_PATH and TYPECODE_LIBMAGIC_DB_PATH instead?
May be it is more explicit than overloading LD_LIBRARY_PATH?

@priv-kweihmann
Copy link
Contributor Author

@pombredanne I'm actually a little torn about this - on the one hand the usage of commonly known variable is what I would like to see, on the other I'm not sure about any possible sideeffects, as esp. LD_LIBRARY_PATH is pretty invasive.

So I guess it's better to follow your proposal, even if it mean having a larger env set to mind when building - will push a v2 somewhat soon

@priv-kweihmann
Copy link
Contributor Author

BTW does that somehow need to be documented here? If yes, please point me to the right doc to edit

@pombredanne
Copy link
Member

@priv-kweihmann you wrote:

BTW does that somehow need to be documented here? If yes, please point me to the right doc to edit

We can start with a simple doc statement in the README of each plugin.

And beyond this we may want to put this in:

@priv-kweihmann priv-kweihmann force-pushed the system-provided-better-override branch from 9d45c19 to 402bf7b Compare April 8, 2021 16:34
@priv-kweihmann
Copy link
Contributor Author

priv-kweihmann commented Apr 8, 2021

New version pushed, including some brief documentation in the respective plugin dir - let me know if there is more to do from my side

@AyanSinhaMahapatra
Copy link
Member

AyanSinhaMahapatra commented Apr 9, 2021

@pombredanne I think there should be a list at https://github.com/nexB/scancode-toolkit/tree/develop/docs/source/plugins with all available ones, that would make sense. If that was what you're asking, should that get a separate issue?

let the user override assumed paths via environment.
EXTRACTCODE_7Z_PATH is used to determine the path of the 7z
executable.
If not specified it falls back to previously used probe of
distro data.

With this patch it possible to run scancode is buildsystems like YOCTO,
which heaviily rely on overriding paths via environment to pick the
correct implementation from the build workspace

Signed-off-by: Konrad Weihmann <[email protected]>
let the user override assumed paths via environment.
TYPECODE_LIBMAGIC_PATH environment variable can be used to explicitly
set the path to libmagic.so. If not specified it falls back to
previously used calculation from distro data.
Location of the magic database can be specified via
TYPECODE_LIBMAGIC_DB_PATH environment variable.
If not found it falls back to the previous bevahior.

With this patch it possible to run scancode is buildsystems like YOCTO,
which heaviily rely on overriding paths via environment to pick the
correct implementation from the build workspace

Signed-off-by: Konrad Weihmann <[email protected]>
let the user override assumed paths via environment.
Path to libarchive can be specified via
EXTRACTCODE_LIBARCHIVE_PATH environment variable.
If not found it falls back to previously calculation from
distro data.

With this patch it possible to run scancode is buildsystems like YOCTO,
which heaviily rely on overriding paths via environment to pick the
correct implementation from the build workspace

Signed-off-by: Konrad Weihmann <[email protected]>
@priv-kweihmann priv-kweihmann force-pushed the system-provided-better-override branch from 402bf7b to 627608d Compare April 12, 2021 19:03
@pombredanne
Copy link
Member

pombredanne commented Apr 19, 2021

@priv-kweihmann Thank you ++ let me merge all this as well as this other PR aboutcode-org/extractcode#18 by @tardyp which is closely related.
I will refactor the code to get this cascaded behaviour as a generally available and documented way to get pre-built binaries across all the tools and libraries:

  1. if there is an environment variable that points to a path use this
  2. otherwise try to get a plugin-provided path (which could be a bundled binary or a path provider)
  3. otherwise try to get a path from well-known locations (these may be OS-specific)
  4. otherwise try to get a path from the PATH (e.g. which)
  5. otherwise fail with a descriptive error message exposing remediation options

Copy link
Member

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you ++
I am also porting your changes upstream in https://github.com/nexB/extractcode and https://github.com/nexB/typecode directly, so that it can work even if so plugin is installed.

@pombredanne pombredanne merged commit a8e52b3 into aboutcode-org:main Apr 23, 2021
pombredanne added a commit to aboutcode-org/typecode that referenced this pull request May 30, 2021
We now load the libmagic native library and its database from paths found in:
1. environment variables,
2. OR a location provider plugin,
3. OR the system PATH,
4. OR we fail with an informative error message.

Based on original code contributed by @priv-kweihmann to
scancode-plugins in aboutcode-org/scancode-plugins#9 and
moved here and adapted for use in the core code rather than in a plugin.

Contributed-by: Konrad Weihmann <[email protected]>
Signed-off-by: Philippe Ombredanne <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants