Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make PATH scanning/filtering a native operation #10526

Open
stuhood opened this issue Jul 31, 2020 · 1 comment
Open

Make PATH scanning/filtering a native operation #10526

stuhood opened this issue Jul 31, 2020 · 1 comment

Comments

@stuhood
Copy link
Member

stuhood commented Jul 31, 2020

(broken out of #9760)

The ability to adjust the PATH (filtering) for a process is a fairly universal thing, and a global default for processes is likely. But it will likely also be necessary to adjust this on a Process by Process basis (a simpler form of what was described in #10156).

So, we should:

  1. Add a ProcessEnvironment subsystem with an option to override/filter the PATH.

    • And probably rename the SubprocessEnvironment subsystem (which is somewhat Python specific) to minimize confusion.
    • And lift the PATH off of PEXEnvironment onto the global ProcessEnvironment
    • Both of these will eventually become platform specific via Platform/Environment-specific subsystems #7735.
  2. Add Process(.., environment: Optional[ProcessEnvironment]), and adjust the Process intrinsic to depend on the global ProcessEnvironment if an environment is not specified.

    • Each rust CommandRunner will do "Something" to detect the PATH in the relevant environment, and will then filter it to provide to all processes:
      • For the local CommandRunner, this will mean adding a complete PATH global_option (marked daemon=True to restart pantsd, and with some deeper scanning), with a default of the entire PATH that pants is running with. That variable will exist for fingerprinting purposes, and is the thing that will then be filtered down on a process by process basis by the PATH filter option.
      • For the remote CommandRunner, this will involve running a specially formulated process lazily at startup time to detect the PATH in the remote environment, and then filtering that for each user process that runs afterward.
  3. Implement a syntax for declaring a scoped Subsystem dependency, so that different processes can choose to use different instances of the subsystem.

    • Most likely via memoization of an anonymous class:
    MyProcessEnvironment = ProcessEnvironment.scoped_to("my_scope")
    
    SubsystemRule(MyProcessEnvironment)
    
    • This is OK as a followup: if people can configure global PATH filters, we can migrate toward having scoped configuration later (similar to Platform/Environment-specific subsystems #7735 later introducing environment-specific filters).

#10489 did a portion of this by adding the PEXEnvironment Subsystem with a PATH-equivalent variable. But it has a caching bug: a particular PATH will be filtered exactly once, even though its content might change for each run.

@stuhood
Copy link
Member Author

stuhood commented Oct 28, 2020

cc @tdyas : This relates to the discussion of remote caching yesterday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant