New method of enabling PEP 582

README.md

@@ -74,9 +74,52 @@ Or you can install it under a user site:
 $ pip install --user pdm
 ```
 
-## Usage
+## Quickstart
 
-`python -m pdm --help` provides helpful guidance.
+**Initialize a new PDM project**
+
+```bash
+$ pdm init
+```
+
+Answer the questions following the guide, and a PDM project with a `pyproject.toml` file will be ready to use.
+
+**Install dependencies into the `__pypackages__` directory**
+
+```bash
+$ pdm add requests flask
+```
+
+You can add multiple dependencies in the same command. After a while, check the `pdm.lock` file to see what is locked for each package.
+
+**Run your script with PEP 582 support**
+
+Suppose you have a script `app.py` placed next to the `__pypackages__` directory with the following content(taken from Flask's website):
+
+```python
+from flask import Flask
+app = Flask(__name__)
+
+@app.route('/')
+def hello_world():
+    return 'Hello World!'
+
+if __name__ == '__main__':
+    app.run()
+```
+
+Set environment variable `export PYTHONPEP582=1`. Now you can run the app directly with your familiar **Python interpreter**:
+
+```bash
+$ python /home/frostming/workspace/flask_app/app.py
+ * Serving Flask app "app" (lazy loading)
+ ...
+ * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
+```
+
+Ta-da! You are running an app with its dependencies installed in an isolated place, while no virtualenv is involved.
+
+If you are curious about how this works, check [this doc section](https://pdm.fming.dev/project/#how-we-make-pep-582-packages-available-to-the-python-interpreter) for some explanation.
 
 ## Docker image
 

README_zh.md

@@ -69,9 +69,52 @@ $ pipx install pdm
 $ pip install --user pdm
 ```
 
-## 使用方法
+## 快速上手
 
-作者很懒，还没来得及写，先用 `python -m pdm --help` 查看帮助吧。
+**初始化一个新的 PDM 项目**
+
+```bash
+$ pdm init
+```
+
+按照指引回答提示的问题，一个 PDM 项目和对应的`pyproject.toml`文件就创建好了。
+
+**把依赖安装到 `__pypackages__` 文件夹中**
+
+```bash
+$ pdm add requests flask
+```
+
+你可以在同一条命令中添加多个依赖。稍等片刻完成之后，你可以查看`pdm.lock`文件看看有哪些依赖以及对应版本。
+
+**在 PEP 582 加持下运行你的脚本**
+
+假设你在`__pypackages__`同级的目录下有一个`app.py`脚本，内容如下（从 Flask 的官网例子复制而来）：
+
+```python
+from flask import Flask
+app = Flask(__name__)
+
+@app.route('/')
+def hello_world():
+    return 'Hello World!'
+
+if __name__ == '__main__':
+    app.run()
+```
+
+设置环境变量`export PYTHONPEP582=1`，现在你可以用你最熟悉的 **Python 解释器** 运行脚本：
+
+```bash
+$ python /home/frostming/workspace/flask_app/app.py
+ * Serving Flask app "app" (lazy loading)
+ ...
+ * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
+```
+
+当当当当！你已经把应用运行起来了，而它的依赖全被安装在一个项目独立的文件夹下，而我们完全没有创建虚拟环境。
+
+如果你好奇这是如何实现的，可以查看[文档](https://pdm.fming.dev/project/#how-we-make-pep-582-packages-available-to-the-python-interpreter)，有一个简短的解释。
 
 ## 常见问题
 

docs/docs/index.md

@@ -44,6 +44,17 @@ Install PDM into user site with `pip`:
 $ pip install --user pdm
 ```
 
+### Enable PEP 582 globally
+
+To make the Python interpreters aware of PEP 582 packages, set the environment variable `PYTHONPEP582` to `1`.
+You may want to write a line in your `.bash_profile`(or similar profiles) to make it effective when login:
+
+```bash
+export PYTHONPEP582=1
+```
+
+**This setup may become the default in the future.**
+
 ### Use the docker image
 
 PDM also provides a docker image to ease your deployment flow, to use it, write a Dockerfile with following content:

docs/docs/project.md

@@ -121,8 +121,7 @@ If you want global project to track another project file other than `~/.pdm/glob
 project path following `-g/--global`.
 
 !!! danger "NOTE"
-    Be careful with `remove` and `sync --clean` commands when global project is used. Because it may
-    remove packages installed in your system Python.
+    Be careful with `remove` and `sync --clean` commands when global project is used. Because it may remove packages installed in your system Python.
 
 ## Working with a virtualenv
 
@@ -238,7 +237,7 @@ The function can be supplied with literal arguments:
 foobar = {call = "foo_package.bar_module:main('dev')"}
 ```
 
-### Environment variables expansion
+### Environment variables support
 
 All environment variables set in the current shell can be seen by `pdm run` and will be expanded when executed.
 Besides, you can also define some fixed environment variables in your `pyproject.toml`:
@@ -277,3 +276,9 @@ test_shell  shell echo $FOO        shell command
 ```
 
 You can add an `help` option with the description of the script, and it will be displayed in the `Description` column in the above output.
+
+### How we make PEP 582 packages available to the Python interpreter
+
+Thanks to the [site packages loading](https://docs.python.org/3/library/site.html) on Python startup. It is possible to patch the `sys.path`
+by placing a `_pdm_pep582.pth` together with a small script under the `site-packages` directory. The interpreter can search the directories
+for the neareset `__pypackage__` folder and append it to the `sys.path` variable. This is totally done by PDM and users shouldn't be aware.

news/181.feature

@@ -0,0 +1 @@
+Now PEP 582 can be enabled in the Python interperter directly!

pdm/cli/actions.py

@@ -97,6 +97,7 @@ def do_sync(
         candidates.update(project.get_locked_candidates())
     handler = project.core.synchronizer_class(candidates, project.environment)
     handler.synchronize(clean=clean, dry_run=dry_run)
+    project.environment.install_pep582_launcher()
 
 
 def do_add(
@@ -387,7 +388,7 @@ def do_init(
         project._pyproject.setdefault("tool", {})["pdm"] = data["tool"]["pdm"]
         project._pyproject["build-system"] = data["build-system"]
     project.write_pyproject()
-    project.environment.write_site_py()
+    project.environment.install_pep582_launcher()
 
 
 def do_use(project: Project, python: str, first: bool = False) -> None:

pdm/cli/commands/run.py

@@ -11,7 +11,6 @@
 from pdm.exceptions import PdmUsageError
 from pdm.iostream import stream
 from pdm.project import Project
-from pdm.utils import find_project_root
 
 
 class Command(BaseCommand):
@@ -42,37 +41,35 @@ def _run_command(
         env: Optional[Dict[str, str]] = None,
         env_file: Optional[str] = None,
     ) -> None:
-        with project.environment.activate():
-            if env_file:
-                import dotenv
+        os.environ.update({"PYTHONPEP582": "1"})
+        if env_file:
+            import dotenv
 
-                stream.echo(f"Loading .env file: {stream.green(env_file)}", err=True)
-                dotenv.load_dotenv(
-                    project.root.joinpath(env_file).as_posix(), override=True
-                )
-            if env:
-                os.environ.update(env)
-            if shell:
-                sys.exit(subprocess.call(os.path.expandvars(args), shell=True))
+            stream.echo(f"Loading .env file: {stream.green(env_file)}", err=True)
+            dotenv.load_dotenv(
+                project.root.joinpath(env_file).as_posix(), override=True
+            )
+        if env:
+            os.environ.update(env)
+        if shell:
+            sys.exit(subprocess.call(os.path.expandvars(args), shell=True))
 
-            command, *args = args
-            expanded_command = project.environment.which(command)
-            if not expanded_command:
-                raise PdmUsageError(
-                    "Command {} is not found on your PATH.".format(
-                        stream.green(f"'{command}'")
-                    )
+        command, *args = args
+        expanded_command = project.environment.which(command)
+        if not expanded_command:
+            raise PdmUsageError(
+                "Command {} is not found on your PATH.".format(
+                    stream.green(f"'{command}'")
                 )
-            expanded_command = os.path.expanduser(os.path.expandvars(expanded_command))
-            expanded_args = [
-                os.path.expandvars(arg) for arg in [expanded_command] + args
-            ]
-            if os.name == "nt" or "CI" in os.environ:
-                # In order to make sure pytest is playing well,
-                # don't hand over the process under a testing environment.
-                sys.exit(subprocess.call(expanded_args))
-            else:
-                os.execv(expanded_command, expanded_args)
+            )
+        expanded_command = os.path.expanduser(os.path.expandvars(expanded_command))
+        expanded_args = [os.path.expandvars(arg) for arg in [expanded_command] + args]
+        if os.name == "nt" or "CI" in os.environ:
+            # In order to make sure pytest is playing well,
+            # don't hand over the process under a testing environment.
+            sys.exit(subprocess.call(expanded_args))
+        else:
+            os.execv(expanded_command, expanded_args)
 
     def _normalize_script(self, script):
         if not getattr(script, "items", None):
@@ -153,17 +150,6 @@ def handle(self, project: Project, options: argparse.Namespace) -> None:
         global_env_options = project.scripts.get("_", {}) if project.scripts else {}
         if project.scripts and options.command in project.scripts:
             self._run_script(project, options.command, options.args, global_env_options)
-        elif os.path.isfile(options.command) and options.command.endswith(".py"):
-            # Allow executing py scripts like `pdm run my_script.py`.
-            # In this case, the nearest `__pypackages__` will be loaded as
-            # the library source.
-            new_root = find_project_root(os.path.abspath(options.command))
-            project = Project(new_root) if new_root else project
-            self._run_command(
-                project,
-                ["python", options.command] + options.args,
-                **global_env_options,
-            )
         else:
             self._run_command(
                 project, [options.command] + options.args, **global_env_options

pdm/installers/_editable_install.py

@@ -3,14 +3,6 @@
 import sys
 import tokenize
 
-from setuptools.command import easy_install
-
-EXE_INITIALIZE = """
-import sys
-with open({0!r}) as fp:
-    exec(compile(fp.read(), __file__, "exec"))
-""".strip()
-
 
 def install(setup_py, prefix, lib_dir, bin_dir):
     __file__ = setup_py
@@ -30,16 +22,6 @@ def install(setup_py, prefix, lib_dir, bin_dir):
         "--site-dirs={0}".format(lib_dir),
     ]
     sys.path.append(lib_dir)
-    if os.getenv("INJECT_SITE", "").lower() in ("1", "true", "yes"):
-        # Patches the script writer to inject library path
-        easy_install.ScriptWriter.template = easy_install.ScriptWriter.template.replace(
-            "import sys",
-            EXE_INITIALIZE.format(
-                os.path.abspath(
-                    os.path.join(lib_dir, os.path.pardir, "site/sitecustomize.py")
-                )
-            ),
-        )
     exec(compile(code, __file__, "exec"))
 
 

pdm/installers/_pep582.py

@@ -0,0 +1,61 @@
+import os
+import site
+import sys
+from distutils.sysconfig import get_python_lib
+
+# Global state to avoid recursive execution
+_initialized = False
+
+
+def get_pypackages_path(maxdepth=5):
+    def find_pypackage(path):
+        packages_name = "__pypackages__/{}/lib".format(
+            ".".join(map(str, sys.version_info[:2]))
+        )
+        for _ in range(maxdepth):
+            if os.path.exists(os.path.join(path, packages_name)):
+                return os.path.join(path, packages_name)
+            if os.path.dirname(path) == path:
+                # Root path is reached
+                break
+            path = os.path.dirname(path)
+        return None
+
+    script_dir = os.path.dirname(os.path.abspath(sys.argv[0]))
+    return find_pypackage(script_dir) or find_pypackage(os.getcwd())
+
+
+def init():
+    global _initialized
+    if (
+        os.getenv("PYTHONPEP582", "").lower() not in ("true", "1", "yes")
+        or _initialized
+    ):
+        # Do nothing if pep 582 is not enabled explicitly
+        return
+    _initialized = True
+    # First, drop system-sites related paths.
+    libpath = get_pypackages_path()
+    if not libpath:
+        return
+    original_sys_path = sys.path[:]
+    known_paths = set()
+    system_sites = {
+        os.path.normcase(site)
+        for site in (
+            get_python_lib(plat_specific=False),
+            get_python_lib(plat_specific=True),
+        )
+    }
+    for path in system_sites:
+        site.addsitedir(path, known_paths=known_paths)
+    system_paths = set(
+        os.path.normcase(path) for path in sys.path[len(original_sys_path) :]
+    )
+    original_sys_path = [
+        path for path in original_sys_path if os.path.normcase(path) not in system_paths
+    ]
+    sys.path = original_sys_path
+
+    # Second, add lib directories, ensuring .pth file are processed.
+    site.addsitedir(libpath)

pdm/installers/installers.py

@@ -32,13 +32,6 @@ def format_dist(dist: Distribution) -> str:
     return formatter.format(version=stream.yellow(dist.version), path=path)
 
 
-EXE_INITIALIZE = """
-import sys
-with open({0!r}) as fp:
-    exec(compile(fp.read(), __file__, "exec"))
-""".strip()
-
-
 class Installer:  # pragma: no cover
     """The installer that performs the installation and uninstallation actions."""
 
@@ -59,14 +52,6 @@ def install_wheel(self, wheel: Wheel) -> None:
         paths = self.environment.get_paths()
         maker = distlib.scripts.ScriptMaker(None, None)
         maker.executable = self.environment.python_executable
-        if not self.environment.is_global:
-            site_custom_script = (
-                self.environment.packages_path / "site/sitecustomize.py"
-            ).as_posix()
-            maker.script_template = maker.script_template.replace(
-                "import sys",
-                EXE_INITIALIZE.format(site_custom_script),
-            )
         wheel.install(paths, maker)
 
     def install_editable(self, ireq: shims.InstallRequirement) -> None: