Does not detect arguments supplied by decorator function

[bfelbo]

Pylint does not detect arguments supplied by a decorator function. This is important for us as we use a decorator function to simplify our interface with our Postgres DB, and make it easy for us to mock all DB interaction logic using test data. I've created a simplified example below that shows the error.

Steps to reproduce

Run pylint on this code.

import functools
from typing import Union as U, Optional as O, Dict, List, Set

def db_statement(queries: list):
    def outside_decorator(fn):

        # functools.wraps() ensures the function information presented to the outside
        # is the original function and not this wrapper madness.
        @functools.wraps(fn)
        def inside_decorator(*args, **kwargs):

            # Test data has been provided as 'db_data' kwarg and therefore no DB query
            if "db_data" in kwargs:
                db_data = kwargs["db_data"]
                del kwargs["db_data"]
                if db_data is None:
                    raise ValueError("Test DB data cannot be None!")
                return fn(*args, **kwargs, db_data=db_data)

            # Run the DB queries, allowing reuse of the same arguments for all queries
            if ("db_params") in kwargs:
                db_params = kwargs["db_params"]
                del kwargs["db_params"]
            else:
                db_params = None
            db_data = []
            for q in queries:
                # Mocked DB call to avoid any DB code in this example
                db_data.append([q, db_params])

            if db_data is None:
                raise ValueError("Data found in DB cannot be None!")

            return fn(*args, **kwargs, db_data=db_data)
        return inside_decorator
    return outside_decorator

@db_statement(
['''
SELECT user_id, created_at
FROM my_user_table
WHERE user_id IN (%(user_ids)s)
ORDER BY user_id;
'''])
def get_users_created_at(db_data: List[List], db_params: O[dict] = None) -> None:
    print(db_data)

if __name__ == "__main__":
    get_users_created_at(db_params={"user_ids": [1, 2, 3]})

Current behavior

For the code above, pylint gives the following error message: No value for argument 'db_data' in function call despite that value always being provided.

Expected behavior

No error message as the generator function will always ensure that the get_users_created_at function is called with a non-null db_data argument.

pylint --version output

pylint 2.1.1
astroid 2.0.4
Python 3.7.4 (default, Jul  9 2019, 18:13:23) 
[Clang 10.0.1 (clang-1001.0.46.4)]

[yuan1238y]

Me too.

Reproduce code:

import time
import asyncio
from datetime import timedelta
from tornado.ioloop import IOLoop
from tornado import gen
from asyncio.subprocess import PIPE

async def run_cmd(*args):
    p = await asyncio.create_subprocess_exec(*args, stdout=PIPE, stderr=PIPE)
    await p.communicate()

async def job(name, delay):
    print("Job", name, "start at", time.time())
    await run_cmd("sleep", str(delay))
    print("Job", name, "finishi at", time.time())

async def run_job(delay, *args):
    while True:
        await job(*args)
        await asyncio.sleep(delay)

def main():
    IOLoop.current().spawn_callback(run_job, 2, "1", 1)
    IOLoop.current().spawn_callback(run_job, 3, "5", 5)
    IOLoop.current().spawn_callback(run_job, 4, "10", 10)

    IOLoop.current().start()

if __name__ == "__main__":
    main()

Lint result:

{
"resource": "/home/test.py",
"owner": "python",
"code": "no-value-for-parameter",
"severity": 8,
"message": "No value for argument 'program' in function call",
"source": "pylint",
"startLineNumber": 9,
"startColumn": 15,
"endLineNumber": 9,
"endColumn": 15
}

{
"resource": "/home/test.py",
"owner": "python",
"code": "no-value-for-parameter",
"severity": 8,
"message": "No value for argument 'name' in function call",
"source": "pylint",
"startLineNumber": 19,
"startColumn": 15,
"endLineNumber": 19,
"endColumn": 15
}

{
"resource": "/home/test.py",
"owner": "python",
"code": "no-value-for-parameter",
"severity": 8,
"message": "No value for argument 'delay' in function call",
"source": "pylint",
"startLineNumber": 19,
"startColumn": 15,
"endLineNumber": 19,
"endColumn": 15
}

Version info:

pylint 2.3.1
astroid 2.2.5
Python 3.6.8 (default, Apr 25 2019, 21:02:35)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]

[PCManticore]

Right, this kind of analysis is not easily understood by pylint since it cannot parse the decorators figuring out where the extra parameter is passed.
For now there is no solution to this, but in 2.4 you will be able to use the new --signature-mutators option, which receives a list of decorators that mutate the parameter list, such as db_statement: http://pylint.pycqa.org/en/latest/whatsnew/2.4.html