🐛 Source Paypal Transaction: report full error message details, updated check method

[midavadim]

Enhanced check connection method

What

The customer receives error message during sync:
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api-m.sandbox.paypal.com/v1/reporting/balances?as_of_time=2021-01-01T00%3A00%3A00%2B00%3A00

Details messages says:
{'name': 'INVALID_REQUEST', 'message': 'Data for the given start date is not available.', 'debug_id': 'b501c1c6a2405', 'details': [], 'links': []}"}}

This is 'expected' Paypal behavior for sandbox account when we query API for a time period when the account was not created/used/activated.

Even though Paypal docs says that the valid start date can be 3 year old, in practice this date can't be evaluated before sync.

How

1. Added custom exception which allows seeing full error description, which allows understanding the problem.
2. Updated check method - which validates start_date (we can't know it beforehand). So now customer will see error message in UI saying that:
   "Data for the given start date (2021-01-01T00:00:00+00:00) is not available, please use more recent start date"

Recommended reading order

1. x.java
2. y.python

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

• Community member? Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Documentation updated
  • Connector's README.md
  • Connector's bootstrap.md. See description and examples
  • docs/SUMMARY.md
  • docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • docs/integrations/README.md
  • airbyte-integrations/builds.md
• PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• Credentials added to Github CI. Instructions.
• /test connector=connectors/<name> command is passing.
• New Connector version released on Dockerhub by running the /publish command described here
• After the connector is published, connector added to connector index as described here
• Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Updating a connector

Community member or Airbyter

• Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Documentation updated
  • Connector's README.md
  • Connector's bootstrap.md. See description and examples
  • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
• PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• Credentials added to Github CI. Instructions.
• /test connector=connectors/<name> command is passing.
• New Connector version released on Dockerhub by running the /publish command described here
• After the new connector version is published, connector version bumped in the seed directory as described here
• Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Connector Generator

• Issue acceptance criteria met
• PR name follows PR naming conventions
• If adding a new generator, add it to the list of scaffold modules being tested
• The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
• Documentation which references the generator is updated as needed.

[CLAassistant]

All committers have signed the CLA.

[midavadim]

/test connector=connectors/source-paypal-transaction

🕑 connectors/source-paypal-transaction https://github.com/airbytehq/airbyte/actions/runs/1549624665
❌ connectors/source-paypal-transaction https://github.com/airbytehq/airbyte/actions/runs/1549624665
🐛 https://gradle.com/s/xkvgm6lw5nxjc

[midavadim]

/test connector=connectors/source-paypal-transaction

🕑 connectors/source-paypal-transaction https://github.com/airbytehq/airbyte/actions/runs/1550212880
❌ connectors/source-paypal-transaction https://github.com/airbytehq/airbyte/actions/runs/1550212880
🐛 https://gradle.com/s/jypuhdsw3wv6w

[antixar]

Is there some bug report at an official PayPal forum/bug tracker about this sandbox behaviour? It would be nice if we can add more details to docs of the connector.

[antixar]

/test connector=connectors/source-paypal-transaction

🕑 connectors/source-paypal-transaction https://github.com/airbytehq/airbyte/actions/runs/1582000408
❌ connectors/source-paypal-transaction https://github.com/airbytehq/airbyte/actions/runs/1582000408
🐛

[bazarnov]

/test connector=connectors/source-paypal-transaction

🕑 connectors/source-paypal-transaction https://github.com/airbytehq/airbyte/actions/runs/1587772301
✅ connectors/source-paypal-transaction https://github.com/airbytehq/airbyte/actions/runs/1587772301
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        76      8    89%
	 source_acceptance_test/conftest.py                     109    109     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              235     95    60%
	 source_acceptance_test/tests/test_full_refresh.py       38     27    29%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  54     24    56%
	 source_acceptance_test/utils/compare.py                 62     25    60%
	 source_acceptance_test/utils/connector_runner.py       101     65    36%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  965    458    53%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                    Stmts   Miss  Cover
	 -----------------------------------------------------------
	 source_paypal_transaction/__init__.py       2      0   100%
	 source_paypal_transaction/source.py       193     94    51%
	 -----------------------------------------------------------
	 TOTAL                                     195     94    52%

[antixar]

LGTM
This is single possible solution. We can't fix it on our side.
Please publish this

[bazarnov]

/publish connector=connectors/source-paypal-transaction

🕑 connectors/source-paypal-transaction https://github.com/airbytehq/airbyte/actions/runs/1588079994
✅ connectors/source-paypal-transaction https://github.com/airbytehq/airbyte/actions/runs/1588079994