develop mode does not respect src structure

[ghost]

Originally reported by: marscher (Bitbucket: marscher, GitHub: marscher)

---

Assume you have project with a src folder:

#!python

root/
root/setup.py
root/src/my_pkg/__init__.py
....

in setup.py one have mapped the src folder, so it can be found during installation.

#!python

setup( ....
   package_dir = { 'my_pkg' : 'src' } 
)

If one tries to install an egg link to the pkg via setup.py develop, this mapping is happily ignored. I'am not sure, if this is an issue either with setuptools or distutils. Symlinking my_pkg to the root fixes the problem.

---

• Bitbucket: https://bitbucket.org/pypa/setuptools/issue/230

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

This is a long-standing issue, originally reported as Distribute 177. If you read the background there, you'll see that the problem doesn't have an obvious general solution.

Edit: Distribute 177 is here.

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Issue #276 was marked as a duplicate of this issue.

[ghost]

Original comment by mgedmin (Bitbucket: mgedmin, GitHub: mgedmin):

---

@marscher: your setup.py looks wrong. AFAIU it should say

package_dir = { 'my_pkg' : 'src/my_pkg' }

A simple workaround exists, as long as the package name matches the directory name: strip the last name from both sides of the mapping and use

package_dir = { '' : 'src' }

@jaraco: perhaps it'd be good to have 'python setup.py develop' fail with an error message instead of resulting in a broken .pth file? As long as it doesn't break the workaround, which is widely used in thousands of packages, of course.

I think it should be possible to solve this for all cases where the directory names in the source tree match package names, with a little bit of programming.

(EDIT: updated @-mentions to use people's bitbucket usernames rather than first names. Sorry unrelated people who were probably pinged by mail, I didn't know that bitbucket actually parsed @-mentions in comments!)

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Issue #463 was marked as a duplicate of this issue.

[jaraco]

@RonnyPfannschmidt has suggested a few options:

• symlink the packages during develop install
• create shim file
• create meta import hook

All of these run into challenges thinking about the total workflow (how are they registered/uninstalled).

If someone could put together a routine or package that demonstrates a technique for installing such a hook, then we can work to devise a workflow around it to use within the develop command and installers.

[RonnyPfannschmidt]

i documented some of it in https://mail.python.org/pipermail/distutils-sig/2017-September/031498.html

i made a prior implementation in https://github.com/RonnyPfannschmidt/gumby_elf/blob/03391bdf0cc66faaa2a5e034769e557c31974fd9/src/packing.py#L34-L43

i believe the best prospect is building/installing the dist info alongside a bootstrapped and a altered record file

[nthienan]

Hi there,

Any updates on this?

[jaraco]

No updates, although the packaging standard have developed and now there is some standards for supporting editable installs. Perhaps supporting that PEP could open up opportunities for this use case.

[abravalheri]

I believe v64.0.3+ solves the problem with package_dir and editable installs reported here.

[gnaggnoyil]

I'm sorry I tried setuptools 65.5.0 with python 3.10.9 with the case provided by OP and the problem still seems to exist. I ran python setup.py develop and then ran import my_pkg in a python interpreter. I still got the "ModuleNotFoundError: No module named 'my_pkg'" error here.

[abravalheri]

Hi @gnaggnoyil please note that it is not possible to fix this problem if you are using python setup.py develop. This is well known limitation of develop.

If you want to give it a try to editable install, please use pip install -e .

[gnaggnoyil]

@abravalheri Hello, I also tried pip install -e . and the result was the same. I also tried to inspect the files that pip installed in my venv and the files seems to be same as those installed through setup.py develop.

[abravalheri]

In this case, pip might be handling your project with the legacy procedure.

You can try adding pyproject.toml or run pip with the --use-pep517 flag -- this is going to be the default in pip 23.1. Also, make sure that you are running an up-to-date version of pip.

If that does not work. Please open a new issue with a minimal example that reproduces your error.

[DaanVanVugt]

Using the --use-pep517 flag fixed this problem for me.