F401 erroneously raised with "import as"

[pyflakes-bot]

Original report by mforbes-physics on Launchpad:

---

The following code raises an F401 error.

import scipy.optimize
import scipy as sp

print(sp.optimize)

I don't see any obvious workaround. I used to do:

import scipy.optimize
sp = scipy

import numpy

print(sp.optimize)

but now flake8 correctly flags the line "sp = scipy" as E402 module level import not at top of file, so this workaround is less useful. (One could put all the module assignments at the end, but this is not very good for reading the code.)

I seem to recall discussing this at some point but cannot find the old discussion.

[pyflakes-bot]

Original comment by jayvdb (@jayvdb) on Launchpad:

---

So the flake8 error F401 is pyflakes reporting the following for line 1: 'scipy.optimize' imported but unused.

Which is correct for the sample code you have given.

scipy.optimize is unused, as the object with name scipy is not accessed on line 2; instead the import on line 2 creates a new object sp which just happens to be a module called scipy in the import lookup system.

So I dont believe this is a bug, based on the sample code provided.

I am guessing that you dont really want to print scipy.optimize ? You are trying to hide the error, by doing a dummy usage?

The following will do the import, and hide the pyflakes error

import scipy.optimize as _
import scipy as sp

del _

def x():
    return sp.foo()

If you really need to print or access scipy.optimize, then this works

import scipy.optimize
import scipy as sp

print(scipy.optimize)

def x():
    return sp.foo()

But maybe your real code cant be solved those ways, in which case let me know the project and I'll take a look at the problem with real code.

[pyflakes-bot]

Original comment by mforbes-physics on Launchpad:

---

The print line is simply a way to "use" scipy.optimize. A "real" usage (MWE) would be

import scipy.optimize
import scipy as sp


res = sp.optimize.root(lambda x: x, 1)

This has the same problem. Your example with del _ is a indeed a workaround (though it is pretty ugly!)

It is extremely common to use scipy, numpy, and matplotlib via:

import numpy as np
import scipy as sp
from matplotlib import pyplot as plt

then to use np. sp. or plt. in ones code. However, especially with scipy, the top level import does not bring in the whole library, hence the need for

import scipy.optimize

Although this might be difficult to detect, I believe it is a real bug since scipy.optimize is indeed used (just through the standard alias sp.optimize)

[pyflakes-bot]

Original comment by jayvdb (@jayvdb?) on Launchpad:

---

Then a cleaner workaround would be something like

import scipy.optimize
import scipy as sp

assert sp.optimize == scipy.optimize

--

or even just assert scipy will do the trick, and will work for multiple scipy.x imports.

The difficulty is that from a symbol/name perspective, which is what pyflakes mostly uses, the name scipy is created in the module and never used.

But I am seeing your point .. we know that import scipy.optimize adds optimize to what will be later named sp, and it is used with that alias.

The related problem is that pyflakes doesnt yet do any sanity checking on submodule imports (import x.y). i.e. the following passes

import scipy.foo
import scipy.bar
scipy.baz

It should have errors about scipy.foo and scipy.bar being unused, and scipy on line 3 being an implicit import of scipy.

But maybe we can ignore the fact that submodule import usage tracking is not working, and focus on avoiding the error in your scenario of using import x as y. i.e. this passes

import scipy.optimize
import scipy
assert scipy

In the above, the 'optimize' part is ignored by pyflakes, so why shouldnt the following also pass:

import scipy.optimize
import scipy as sp
assert sp

I think we have enough information to allow the above the pass, without any regressions.

[pyflakes-bot]

Original comment by mforbes-physics on Launchpad:

---

Again, the reason I am now having an issue is that flake8 adds "E402 module level import not at top of file" which requires the assert statement to come after all the imports breaking locality.

It would be really nice if pyflakes could track the rename of scipy to py, but I think that the best workaround at this point is to del all the unused modules at the end of the import section:

import scipy.optimize
import scipy as sp

import numpy.linalg
import numpy as np

...

del scipy, numpy

...

Then the statement at least has some meaning - use sp., not scipy. etc. in the code, following conventions.

[pyflakes-bot]

Original comment by asmeurer (@asmeurer?) on Launchpad:

---

The problem is that pyflakes treats imports like variable assignments, but they aren't quite like that. If you write

a = func()

and then never use a, pyflakes is right to tell you that a is never used, because you could just as well have written

func()

without any assignment. But "import scipy.optimize" is the only way to load the scipy.optimize module (it isn't loaded with "import scipy" for performance purposes). So the purpose of the line is to load a module, but it also happens to load a name into the namespace.

Secondly, it's best practice to import scipy and numpy as sp and np, respectively.

and scipy on line 3 being an implicit import of scipy.

I don't think that's an error. Python explicitly runs the top-level __init__.py and puts the module name in the namespace. For instance, this code works just fine

import numpy.linalg
numpy.array

[pyflakes-bot]

Original comment by jayvdb (@jayvdb?) on Launchpad:

---

I agree pyflakes can special case submodule imports, as they cant be done another way.
Unless anyone objects, I will do the fix, which I suspect is a one-liner now.

and scipy on line 3 being an implicit import of scipy.

I don't think that's an error. Python explicitly runs the top-level __init__.py and puts the module name in the namespace. For instance, this code works just fine

import numpy.linalg
numpy.array

While this is possible, it can also be a problem in complex code. Pywikibot had one such gnarly problem (https://phabricator.wikimedia.org/T113161) which I should extract out into a simple example of how implicit imports can be problematic.

[asottile]

Duplicate of #159