Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-3343] Update DockerOperator for Docker-py 3.0.0 API changes #4187

Merged
merged 1 commit into from
Nov 14, 2018

Conversation

ashb
Copy link
Member

@ashb ashb commented Nov 14, 2018

Make sure you have checked all steps below.

Jira

Description

  • When we updated the version of docker.py in [AIRFLOW-3203] Fix DockerOperator & some operator test #4049 we missed a breaking change. The API of wait() changed to return a dict, not just a number so this
    Operator wasn't actually working, but the tests were passing because the
    return was mocked in-correctly.

Since it appears that the python Docker module follows semver, with breaking changes only on major versions I have also changed the format specifier so we don't pull in 4.0.0 accidentally in the future.

I also removed shm_size from kwargs passed to BaseOperator to avoid
the deprecation warning about unknown args.

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.

Code Quality

  • Passes flake8

@ashb ashb requested a review from Fokko November 14, 2018 17:26
@ashb ashb force-pushed the dockerpy3-api-changes branch from 9314e01 to bf81bd5 Compare November 14, 2018 17:27
@ashb ashb changed the title [AIRFLOW-3339] Update DockerOperator for Docker-py 3.0.0 API changes [AIRFLOW-3343] Update DockerOperator for Docker-py 3.0.0 API changes Nov 14, 2018
@ashb
Copy link
Member Author

ashb commented Nov 14, 2018

@XD-DENG PTAL - this was the bug I pinged you about in slack.

@codecov-io
Copy link

codecov-io commented Nov 14, 2018

Codecov Report

❗ No coverage uploaded for pull request base (master@a5f33f3). Click here to learn what that means.
The diff coverage is 100%.

Impacted file tree graph

@@           Coverage Diff            @@
##             master   #4187   +/-   ##
========================================
  Coverage          ?   77.7%           
========================================
  Files             ?     199           
  Lines             ?   16303           
  Branches          ?       0           
========================================
  Hits              ?   12669           
  Misses            ?    3634           
  Partials          ?       0
Impacted Files Coverage Δ
airflow/operators/docker_operator.py 97.67% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a5f33f3...fff2a53. Read the comment docs.

@ashb ashb force-pushed the dockerpy3-api-changes branch from bf81bd5 to b2bd2c6 Compare November 14, 2018 21:00
Copy link
Member

@kaxil kaxil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor Changes, LGTM otherwise.

airflow/operators/docker_operator.py Show resolved Hide resolved
airflow/operators/docker_operator.py Show resolved Hide resolved
The API of `wait()` changed to return a dict, not just a number so this
Operator wasn't actually working, but the tests were passing because the
return was mocked in-correctly.

I also removed `shm_size` from kwargs passed to BaseOperator to avoid
the deprecation warning about unknown args.
@ashb ashb force-pushed the dockerpy3-api-changes branch from b2bd2c6 to fff2a53 Compare November 14, 2018 21:33
@ashb ashb merged commit 7d6e8cd into apache:master Nov 14, 2018
@ashb ashb deleted the dockerpy3-api-changes branch November 14, 2018 22:06
ashb added a commit that referenced this pull request Nov 14, 2018
…4187)

The API of `wait()` changed to return a dict, not just a number so this
Operator wasn't actually working, but the tests were passing because the
return was mocked in-correctly.

I also removed `shm_size` from kwargs passed to BaseOperator to avoid
the deprecation warning about unknown args.
tmiller-msft pushed a commit to cse-airflow/incubator-airflow that referenced this pull request Nov 27, 2018
…pache#4187)

The API of `wait()` changed to return a dict, not just a number so this
Operator wasn't actually working, but the tests were passing because the
return was mocked in-correctly.

I also removed `shm_size` from kwargs passed to BaseOperator to avoid
the deprecation warning about unknown args.
kaxil pushed a commit that referenced this pull request Nov 30, 2018
…4187)

The API of `wait()` changed to return a dict, not just a number so this
Operator wasn't actually working, but the tests were passing because the
return was mocked in-correctly.

I also removed `shm_size` from kwargs passed to BaseOperator to avoid
the deprecation warning about unknown args.
elizabethhalper pushed a commit to cse-airflow/incubator-airflow that referenced this pull request Dec 7, 2018
…pache#4187)

The API of `wait()` changed to return a dict, not just a number so this
Operator wasn't actually working, but the tests were passing because the
return was mocked in-correctly.

I also removed `shm_size` from kwargs passed to BaseOperator to avoid
the deprecation warning about unknown args.
aliceabe pushed a commit to aliceabe/incubator-airflow that referenced this pull request Jan 3, 2019
…pache#4187)

The API of `wait()` changed to return a dict, not just a number so this
Operator wasn't actually working, but the tests were passing because the
return was mocked in-correctly.

I also removed `shm_size` from kwargs passed to BaseOperator to avoid
the deprecation warning about unknown args.
kaxil pushed a commit that referenced this pull request Jan 9, 2019
…4187)

The API of `wait()` changed to return a dict, not just a number so this
Operator wasn't actually working, but the tests were passing because the
return was mocked in-correctly.

I also removed `shm_size` from kwargs passed to BaseOperator to avoid
the deprecation warning about unknown args.
cfei18 pushed a commit to cfei18/incubator-airflow that referenced this pull request Jan 23, 2019
The API of `wait()` changed to return a dict, not just a number so this
Operator wasn't actually working, but the tests were passing because the
return was mocked in-correctly.

I also removed `shm_size` from kwargs passed to BaseOperator to avoid
the deprecation warning about unknown args.
wmorris75 pushed a commit to modmed/incubator-airflow that referenced this pull request Jul 29, 2019
…pache#4187)

The API of `wait()` changed to return a dict, not just a number so this
Operator wasn't actually working, but the tests were passing because the
return was mocked in-correctly.

I also removed `shm_size` from kwargs passed to BaseOperator to avoid
the deprecation warning about unknown args.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants