Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TDL 19241 add version timestamp in contacts #191

Merged
merged 4 commits into from
Jun 6, 2022

Conversation

prijendev
Copy link
Contributor

@prijendev prijendev commented May 31, 2022

Description of change

  • Added versionTimestamp in the schema of the contacts stream.
  • Added versionTimestamp in the record of the contacts stream.
  • Updated integration test cases.

Manual QA steps

  • Run the discover mode and verify that versionTimestamp field available in the schema of contacts stream in the catalog.
  • Run sync mode and verify that versionTimestamp  field is available in each record of the contacts stream.
  • Run sync mode with state and verify that the contacts stream respected the bookmark value.

Risks

Rollback steps

  • revert this branch

@prijendev prijendev changed the title Tdl 19241 add version timestamp in contacts TDL 19241 add version timestamp in contacts May 31, 2022
@@ -19,6 +19,12 @@ def get_matching_actual_record_by_pk(expected_primary_key_dict, actual_records):
can_save = True
return ret_records

FIELDS_ADDED_BY_TAP = {
# In 'contacts' streams 'versionTimeStamp' was not available in response to the second call.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# In 'contacts' streams 'versionTimeStamp' was not available in response to the second call.
# In 'contacts' streams 'versionTimeStamp' is not available in response of the second call.
Suggested change
# In 'contacts' streams 'versionTimeStamp' was not available in response to the second call.
# In 'contacts' streams 'versionTimeStamp' was not available in response to the second call.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explain first call and second call in ()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explain first call and second call in ()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explained first call and second call.

@@ -19,6 +19,12 @@ def get_matching_actual_record_by_pk(expected_primary_key_dict, actual_records):
can_save = True
return ret_records

FIELDS_ADDED_BY_TAP = {
# In 'contacts' streams 'versionTimeStamp' was not available in response to the second call.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# In 'contacts' streams 'versionTimeStamp' was not available in response to the second call.
# In 'contacts' streams 'versionTimeStamp' is not available in response of the second call.
Suggested change
# In 'contacts' streams 'versionTimeStamp' was not available in response to the second call.
# In 'contacts' streams 'versionTimeStamp' was not available in response to the second call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated comment.

@prijendev prijendev requested a review from dbshah1212 June 1, 2022 08:18
@prijendev prijendev requested review from kspeer825 and RushiT0122 June 1, 2022 12:49
@prijendev prijendev merged commit 752a177 into crest-master Jun 6, 2022
@prijendev prijendev mentioned this pull request Jun 6, 2022
KrisPersonal pushed a commit that referenced this pull request Jun 14, 2022
* TDL 19241 add version timestamp in contacts (#191)

* Added versionTimestamp in contacts stream.

* Updated bookmark test case.

* Updated comments.

* Updated replication method of contacts stream.

* TDL 16686 Implement request timeout (#177)

* Initial commit for request_timeout

* Updated unit test case error

* Removed path of unittest file

* Updated config file

* Updated readme file
@KrisPersonal KrisPersonal mentioned this pull request Jun 14, 2022
kethan1122 added a commit that referenced this pull request Sep 13, 2022
* TDL 19241 add version timestamp in contacts (#191)

* Added versionTimestamp in contacts stream.

* Updated bookmark test case.

* Updated comments.

* Updated replication method of contacts stream.

* TDL 16686 Implement request timeout (#177)

* Initial commit for request_timeout

* Updated unit test case error

* Removed path of unittest file

* Updated config file

* Updated readme file

* Initial commit.

* Updated interruptes sync offset test case

* Resolved comments.

* Replaced replication key of deals stream with property_hs_lastmodifieddate field.

* Updated discover test case.

* Updated ingterrupted_sync test case.

* Utilize bookmark of existing connection (#196)

* Clearing bookmark for existing connection.

* Updated bookmark logic to utilize older bookmark key.

* Updated logic to keep older bookmark key in the state.

* Added unittest cases.

* Updated unit test cases.

* Resolved review comments.

* Updated get_start to use old bookmark for companies stream  method.

* Tdl 15447 fix pylint issues (#204)

* fixed pylint issues TDL-15447

* changing methods to functions since there is no-self-use

* pylint issues

* more pylint issues

* added pylint disable comment for some scenarios

* remove duplicate-code from config.yaml

* pylint comment for test_bookmarks

* remove duplicate code

* removed wrong-import-order

Co-authored-by: kethan1122 <[email protected]>
Marcos314 added a commit to gupy-io/tap-hubspot that referenced this pull request May 11, 2023
* Qa/save test artifacts (singer-io#201)

* add logging and save test logs as artifacts

* name should be static method so it plays well with BaseCase

Co-authored-by: kspeer <[email protected]>

* skip fields missing from schema for contact_lists (singer-io#202)

Co-authored-by: kspeer <[email protected]>

* add limitExempt to all fields test workaround (singer-io#203)

Co-authored-by: kspeer <[email protected]>

* TDL-19319 Make replication key as automatic inclusion (singer-io#195)

* TDL 19241 add version timestamp in contacts (singer-io#191)

* Added versionTimestamp in contacts stream.

* Updated bookmark test case.

* Updated comments.

* Updated replication method of contacts stream.

* TDL 16686 Implement request timeout (singer-io#177)

* Initial commit for request_timeout

* Updated unit test case error

* Removed path of unittest file

* Updated config file

* Updated readme file

* Initial commit.

* Updated interruptes sync offset test case

* Resolved comments.

* Replaced replication key of deals stream with property_hs_lastmodifieddate field.

* Updated discover test case.

* Updated ingterrupted_sync test case.

* Utilize bookmark of existing connection (singer-io#196)

* Clearing bookmark for existing connection.

* Updated bookmark logic to utilize older bookmark key.

* Updated logic to keep older bookmark key in the state.

* Added unittest cases.

* Updated unit test cases.

* Resolved review comments.

* Updated get_start to use old bookmark for companies stream  method.

* Tdl 15447 fix pylint issues (singer-io#204)

* fixed pylint issues TDL-15447

* changing methods to functions since there is no-self-use

* pylint issues

* more pylint issues

* added pylint disable comment for some scenarios

* remove duplicate-code from config.yaml

* pylint comment for test_bookmarks

* remove duplicate code

* removed wrong-import-order

Co-authored-by: kethan1122 <[email protected]>

---------

Co-authored-by: Kyle Speer <[email protected]>
Co-authored-by: kspeer <[email protected]>
Co-authored-by: Prijen Khokhani <[email protected]>
Co-authored-by: kethan1122 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants