🐛 Source Jira: changed cursor field from created to updated #5426
Conversation
…s.started to fields.updated, update IssuesWorklogs slicing
|
/test connector=source-jira
|
| @@ -7455,10 +7455,10 @@ | |||
| }, | |||
| "supported_sync_modes": ["incremental"], | |||
| "source_defined_cursor": true, | |||
| "default_cursor_field": ["created"] | |||
| "default_cursor_field": ["fields", "updated"] | |||
There was a problem hiding this comment.
I dont sure about this change.
Previously issues.fields where a field containing additionalproperties. Yes, updated exists there.
It were stored as JSON column, and it will be possibly returned to additionalproperties without details (like it was before), to avoid data missing and db structure change. We'll discuss this tomorrow
|
@Phlair would you be able to review this one? |
| current_stream_state = current_stream_state.get(self.cursor_field) | ||
| if current_stream_state: | ||
| return {self.cursor_field: str(max(latest_record_date, pendulum.parse(current_stream_state)))} | ||
| issue_id = latest_record.get("issueId") |
There was a problem hiding this comment.
why is this changing? we shouldn't store separate state for each issue, this can get massive
There was a problem hiding this comment.
let's just make this stream full refresh for now
|
/test connector=source-jira
|
|
/test connector=source-jira
|
|
/test connector=source-jira
|
github-actions
bot
added
the
area/documentation
|
/publish connector=connectors/source-jira
|
vitaliizazmic
deleted the
vitalii/4505_jira_use_update_for_cursor_field
branch
What
Changed cursor field for Issues stream from created to updated JQL field. Updated cursor filed in schema for Issues stream. Improved slicing and cursor field for IssuesWorklogs stream.
Closes #4505
airbyte_secret./gradlew :airbyte-integrations:connectors:<name>:integrationTest.README.mddocs/SUMMARY.mddocs/integrations/<source or destination>/<name>.mdincluding changelog. See changelog exampledocs/integrations/README.mdairbyte-integrations/builds.md