We can/should partition more tools by config (black, flake8, etc...)

[mpcusack-color]

Is your feature request related to a problem? Please describe.
Originally discussed on slack: https://pantsbuild.slack.com/archives/C046T6T9U/p1670281834539229

A large (and knotted) portion of my monorepo is a django project. We would like start using pants to do mypy type checking on it and use mypy_django_plugin. The problem is mypy config plugin installation applies everywhere and installing this plugin requires that we add our entire django project as a mypy dependency, even when checking a file that is not part of the django code.

Having the django plugin is not absolutely required to run mypy on this code, but it makes the user experience significantly better and would likely speed up fixing all the current errors.

Mypy doesn’t have support for only enabling a plugin for a subset files/roots. There is only one configuration file ($ROOT/mypy.ini) and it doesn’t support hierarchical config like pytest’s conftest.py.

To enable the django plugin in mypy you need to make the project settings module and all transitive deps (the entire django app) a mypy tool resolve dependency. The plugin throws an error if the project can’t be django.setup().

Describe the solution you'd like
What I need is a way in pants to specify more than one mypy config file and then specify that certain targets should use the non default one. Apparently scalafmt already somewhat does this?

Related: #17328

[thejcannon]

I don't think this is unique to mypy or check. In a monorepo it's completely appropriate to have people expect different paths to use different configs (especially for formatting/linting as they might be owned by different teams). And I think it's rare for tools to support this themselves.

I'm going to change the title.

[mpcusack-color]

Just wanted to describe my problem to help make sure the proposed solution covers that use case. The one semi more specific part of this I can foresee is that the different configs will also need different extra_requirements (and thus different resolves?). Would be thrilled to see it solved in a more general way.

[thejcannon]

In the mypy-specific case, I think you'd be able to always have it installed in the mypy resolve, and enable/disable the loading of it in the config? Unclear, but I think that's a separate issue 😄

[mpcusack-color]

Happy to test things out, but that might be an issue. Our django project is quite large and brings in ~150 3rd party deps. Building the pex for it is painfully slow and is the primary reason a large chunk of our developers (the ones who work on the django code) aren't using pants. Making people who are not working on the django code build that pex every time they want to check an isolated file would not be acceptible.

[thejcannon]

So how we do mypy is interesting. mypy allows you to run in an isolated environment and point it to a different one.
So in this case, the mypy environment would have the plugin but none of your code. Building that PEX should be "fast".

[mpcusack-color]

The mypy_django_plugin requires the entire django project be included in the mypy venv along with the plugin itself.

It does a django.setup()

[thejcannon]

OH yeah, that's a huge issue (albeit separate 😂 )

[mpcusack-color]

Want me to be file another issue?

[thejcannon]

yes please!

[jonasrauber]

I don't think this is unique to mypy or check. In a monorepo it's completely appropriate to have people expect different paths to use different configs (especially for formatting/linting as they might be owned by different teams). And I think it's rare for tools to support this themselves.

Exactly this. Does anyone have a workaround?