🎉 Source Zuora: support of `Unlimited` option for `Data Query` #7093

bazarnov · 2021-10-16T15:25:08Z

What

#7053 - Source Zuora: needs support of Unlimited option for Data Query

How

edited spec.json by adding data_query enum
edited source.py to accept user's input of data_query option
fixed the CDK native logger importing issues

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions
Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here

…zarnov/sat-zoura-fix

bazarnov · 2021-10-20T10:45:45Z

/test connector=connectors/source-zuora

🕑 connectors/source-zuora https://github.com/airbytehq/airbyte/actions/runs/1363162723
✅ connectors/source-zuora https://github.com/airbytehq/airbyte/actions/runs/1363162723
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                     Stmts   Miss  Cover
	 ------------------------------------------------------------
	 source_zuora/__init__.py                     2      0   100%
	 source_zuora/source.py                     204     49    76%
	 source_zuora/zuora_auth.py                  19      0   100%
	 source_zuora/zuora_endpoint.py               4      0   100%
	 source_zuora/zuora_errors.py                24     10    58%
	 source_zuora/zuora_excluded_streams.py       2      0   100%
	 ------------------------------------------------------------
	 TOTAL                                      255     59    77%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                     Stmts   Miss  Cover
	 ------------------------------------------------------------
	 source_zuora/__init__.py                     2      2     0%
	 source_zuora/source.py                     204    204     0%
	 source_zuora/zuora_auth.py                  19     19     0%
	 source_zuora/zuora_endpoint.py               4      4     0%
	 source_zuora/zuora_auth.py                  19     19     0%/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-zuora/.venv/lib/python3.8/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
	 source_zuora/zuora_errors.py                24     24     0%
	 source_zuora/zuora_excluded_streams.py       2      2     0%
	   self._warn("No data was collected.", slug="no-data-collected")
	 ------------------------------------------------------------
	 TOTAL                                      255    255     0%

antixar

LGTM

airbyte-integrations/connectors/source-zuora/source_zuora/source.py

…zarnov/sat-zoura-fix

bazarnov · 2021-10-21T11:45:45Z

/test connector=connectors/source-zuora

🕑 connectors/source-zuora https://github.com/airbytehq/airbyte/actions/runs/1367749797
✅ connectors/source-zuora https://github.com/airbytehq/airbyte/actions/runs/1367749797
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                     Stmts   Miss  Cover
	 ------------------------------------------------------------
	 source_zuora/__init__.py                     2      0   100%
	 source_zuora/source.py                     202     47    77%
	 source_zuora/zuora_auth.py                  19      0   100%
	 source_zuora/zuora_endpoint.py               4      0   100%
	 source_zuora/zuora_errors.py                24     10    58%
	 source_zuora/zuora_excluded_streams.py       2      0   100%
	 ------------------------------------------------------------
	 TOTAL                                      253     57    77%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                     Stmts   Miss  Cover
	 ------------------------------------------------------------
	 source_zuora/__init__.py                     2      2     0%
	 source_zuora/source.py                     202    202     0%
	 source_zuora/zuora_auth.py                  19     19     0%
	 source_zuora/zuora_endpoint.py               4      4     0%
	 source_zuora/zuora_errors.py                24     24     0%
	 source_zuora/zuora_excluded_streams.py       2      2     0%
	 ------------------------------------------------------------
	 TOTAL                                      253    253     0%

bazarnov · 2021-10-21T12:42:37Z

/test connector=connectors/source-zuora

🕑 connectors/source-zuora https://github.com/airbytehq/airbyte/actions/runs/1367949682
✅ connectors/source-zuora https://github.com/airbytehq/airbyte/actions/runs/1367949682
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                     Stmts   Miss  Cover
	 ------------------------------------------------------------
	 source_zuora/__init__.py                     2      0   100%
	 source_zuora/source.py                     208     52    75%
	 source_zuora/zuora_auth.py                  19      0   100%
	 source_zuora/zuora_endpoint.py               4      0   100%
	 source_zuora/zuora_errors.py                30     13    57%
	 source_zuora/zuora_excluded_streams.py       2      0   100%
	 ------------------------------------------------------------
	 TOTAL                                      265     65    75%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                     Stmts   Miss  Cover
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-zuora/.venv/lib/python3.8/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
	 ------------------------------------------------------------
	 source_zuora/__init__.py                     2      2     0%
	 source_zuora/source.py                     208    208     0%
	 source_zuora/zuora_auth.py                  19     19     0%
	 source_zuora/zuora_endpoint.py               4      4     0%
	 source_zuora/zuora_errors.py                30     30     0%
	 source_zuora/zuora_excluded_streams.py       2      2     0%
	 ------------------------------------------------------------
	 TOTAL                                      265    265     0%

…zarnov/sat-zoura-fix

bazarnov · 2021-10-21T14:54:51Z

/test connector=connectors/source-zuora

🕑 connectors/source-zuora https://github.com/airbytehq/airbyte/actions/runs/1368468822
✅ connectors/source-zuora https://github.com/airbytehq/airbyte/actions/runs/1368468822
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                     Stmts   Miss  Cover
	 ------------------------------------------------------------
	 source_zuora/__init__.py                     2      0   100%
	 source_zuora/source.py                     208     52    75%
	 source_zuora/zuora_auth.py                  19      0   100%
	 source_zuora/zuora_endpoint.py               4      0   100%
	 source_zuora/zuora_errors.py                30     13    57%
	 source_zuora/zuora_excluded_streams.py       2      0   100%
	 ------------------------------------------------------------
	 TOTAL                                      265     65    75%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                     Stmts   Miss  Cover
	 ------------------------------------------------------------
	 source_zuora/__init__.py                     2      2     0%
	 Name                                     Stmts   Miss  Cover/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-zuora/.venv/lib/python3.8/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
	 source_zuora/source.py                     208    208     0%
	 source_zuora/zuora_auth.py                  19     19     0%
	 source_zuora/zuora_endpoint.py               4      4     0%
	 source_zuora/zuora_errors.py                30     30     0%
	 source_zuora/zuora_excluded_streams.py       2      2     0%
	 ------------------------------------------------------------
	   self._warn("No data was collected.", slug="no-data-collected")
	 TOTAL                                      265    265     0%

bazarnov · 2021-10-22T12:35:19Z

/publish connector=connectors/source-zuora

🕑 connectors/source-zuora https://github.com/airbytehq/airbyte/actions/runs/1372186386
✅ connectors/source-zuora https://github.com/airbytehq/airbyte/actions/runs/1372186386

…tehq#7093)

fixed SAT tests, added UNLIMITED option for Data Query, bumped version

499555a

github-actions bot added the area/connectors Connector related issues label Oct 16, 2021

bazarnov self-assigned this Oct 16, 2021

bazarnov linked an issue Oct 16, 2021 that may be closed by this pull request

🐛 🎉 Source Zuora: needs support of Unlimited option for Data Query #7053

Closed

changelog updated

74671b0

github-actions bot added the area/documentation Improvements or additions to documentation label Oct 16, 2021

bazarnov temporarily deployed to more-secrets October 16, 2021 15:33 Inactive

modified integration tests

11f73b8

bazarnov temporarily deployed to more-secrets October 16, 2021 18:27 Inactive

edited spec.json, zuora_auth.py, source.py, corrected configs

56c75e1

bazarnov temporarily deployed to more-secrets October 16, 2021 21:16 Inactive

bazarnov added 2 commits October 20, 2021 11:52

modified auth

0fd07e5

Merge remote-tracking branch 'origin/master' into bazarnov/sat-zoura-fix

7a17edb

bazarnov temporarily deployed to more-secrets October 20, 2021 08:55 Inactive

bazarnov added 2 commits October 20, 2021 13:38

small corrections

2df30b1

Merge branch 'master' of https://github.com/airbytehq/airbyte into ba…

7cf40b7

…zarnov/sat-zoura-fix

bazarnov temporarily deployed to more-secrets October 20, 2021 10:41 Inactive

jrhizor temporarily deployed to more-secrets October 20, 2021 10:44 Inactive

jrhizor temporarily deployed to more-secrets October 20, 2021 10:48 Inactive

bazarnov requested review from antixar and lazebnyi October 20, 2021 11:14

antixar reviewed Oct 21, 2021

View reviewed changes

airbyte-integrations/connectors/source-zuora/source_zuora/source.py Outdated Show resolved Hide resolved

airbyte-integrations/connectors/source-zuora/source_zuora/source.py Show resolved Hide resolved

bazarnov added 2 commits October 21, 2021 14:33

Merge branch 'master' of https://github.com/airbytehq/airbyte into ba…

fb022f0

…zarnov/sat-zoura-fix

updated after review

d4b77d3

bazarnov requested a review from antixar October 21, 2021 11:45

bazarnov temporarily deployed to more-secrets October 21, 2021 11:47 Inactive

jrhizor temporarily deployed to more-secrets October 21, 2021 11:48 Inactive

updated after review

6c7d68c

bazarnov temporarily deployed to more-secrets October 21, 2021 12:44 Inactive

jrhizor temporarily deployed to more-secrets October 21, 2021 12:45 Inactive

bazarnov added 2 commits October 21, 2021 17:53

added pattern to window_in_days to ensure the user's input is correct

04167fb

Merge branch 'master' of https://github.com/airbytehq/airbyte into ba…

523830c

…zarnov/sat-zoura-fix

bazarnov temporarily deployed to more-secrets October 21, 2021 14:56 Inactive

jrhizor temporarily deployed to more-secrets October 21, 2021 14:57 Inactive

antixar approved these changes Oct 21, 2021

View reviewed changes

jrhizor temporarily deployed to more-secrets October 22, 2021 12:37 Inactive

bazarnov temporarily deployed to more-secrets October 22, 2021 13:54 Inactive

bazarnov merged commit 9a435ec into master Oct 22, 2021

bazarnov deleted the bazarnov/sat-zoura-fix branch October 22, 2021 14:01

jrhizor mentioned this pull request Oct 25, 2021

Bump Airbyte version from 0.30.22-alpha to 0.30.23-alpha #7320

Merged

schlattk pushed a commit to schlattk/airbyte that referenced this pull request Jan 4, 2022

🎉 Source Zuora: support of Unlimited option for Data Query (airby…

c7a59e7

…tehq#7093)

ycherniaiev added connectors/sources-api connectors/source/zuora labels Jan 14, 2022

lazebnyi removed their request for review May 30, 2022 10:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 Source Zuora: support of `Unlimited` option for `Data Query` #7093

🎉 Source Zuora: support of `Unlimited` option for `Data Query` #7093

bazarnov commented Oct 16, 2021 •

edited

Loading

bazarnov commented Oct 20, 2021 •

edited by github-actions bot

Loading

antixar left a comment

bazarnov commented Oct 21, 2021 •

edited by github-actions bot

Loading

bazarnov commented Oct 21, 2021 •

edited by github-actions bot

Loading

bazarnov commented Oct 21, 2021 •

edited by github-actions bot

Loading

bazarnov commented Oct 22, 2021 •

edited by github-actions bot

Loading

🎉 Source Zuora: support of Unlimited option for Data Query #7093

🎉 Source Zuora: support of Unlimited option for Data Query #7093

Conversation

bazarnov commented Oct 16, 2021 • edited Loading

What

How

Pre-merge Checklist

Community member or Airbyter

Airbyter

bazarnov commented Oct 20, 2021 • edited by github-actions bot Loading

antixar left a comment

Choose a reason for hiding this comment

bazarnov commented Oct 21, 2021 • edited by github-actions bot Loading

bazarnov commented Oct 21, 2021 • edited by github-actions bot Loading

bazarnov commented Oct 21, 2021 • edited by github-actions bot Loading

bazarnov commented Oct 22, 2021 • edited by github-actions bot Loading

🎉 Source Zuora: support of `Unlimited` option for `Data Query` #7093

🎉 Source Zuora: support of `Unlimited` option for `Data Query` #7093

bazarnov commented Oct 16, 2021 •

edited

Loading

bazarnov commented Oct 20, 2021 •

edited by github-actions bot

Loading

bazarnov commented Oct 21, 2021 •

edited by github-actions bot

Loading

bazarnov commented Oct 21, 2021 •

edited by github-actions bot

Loading

bazarnov commented Oct 21, 2021 •

edited by github-actions bot

Loading

bazarnov commented Oct 22, 2021 •

edited by github-actions bot

Loading