apply_default keeps the function signature for mypy #9784
Conversation
boring-cyborg
bot
added
k8s
provider:amazon
mik-laj
force-pushed
the
apply-default-sign
branch
from
d27e5e1 to
2e302ba
Compare
|
|
||
| from airflow.exceptions import AirflowException | ||
|
|
||
| signature = inspect.signature | ||
|
|
||
| T = TypeVar('T', bound=Callable) # pylint: disable=invalid-name |
There was a problem hiding this comment.
This is the most interesting part.
There was a problem hiding this comment.
Not sure I understand it. I'm wondering if removing the @apply_defulats will solve the same issue? I think now we have BaseOperatorMeta we can consider removing this decorator.
There was a problem hiding this comment.
I will solve it because mypy is not smart enough to look into the metaclass, but we will still have remaining problems to solve. I consider it an intermediate solution until we cannot use the metaclass.
| return filecmp.cmp(qubole_result_1, qubole_result_2) | ||
|
|
||
| t1 = QuboleOperator( | ||
| hive_show_table = QuboleOperator( |
There was a problem hiding this comment.
Seems as unrelated change but lets keep it
There was a problem hiding this comment.
This is a related change. The variable t1 was used in this file, but once it had one operator and then another decorator. Mypy then reported that the variable type is invalid.
To solve this problem I had to enter more descriptive variable names.
https://github.com/apache/airflow/blob/2be7efa5c023d96f4266df67c7da2e90020e16f9%5E%5E%5E%5E/airflow/providers/qubole/example_dags/example_qubole.py#L255
https://github.com/apache/airflow/blob/2be7efa5c023d96f4266df67c7da2e90020e16f9%5E%5E%5E%5E/airflow/providers/qubole/example_dags/example_qubole.py#L72
mik-laj
force-pushed
the
apply-default-sign
branch
from
040c69d to
97e323c
Compare
mik-laj
force-pushed
the
apply-default-sign
branch
from
97e323c to
f9c4b68
Compare
mik-laj
force-pushed
the
apply-default-sign
branch
from
9d3ee7d to
b32d842
Compare
apply default cleared all information about constructor parameter types. I fixed it by using generic. This revealed some type inaccuracies that were previously hidden.
In addition, there was a problem with "multiple values for keyword argument"
python/mypy#6799
so I deleted the args parameter in several places to suit mypy. We probably should do it for all operators, but I would prefer to do it in a separate ticket.
Make sure to mark the boxes below before creating PR: [x]
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.