Skip to content

Commit

Permalink
add more technical details
Browse files Browse the repository at this point in the history
  • Loading branch information
hsheth2 committed Oct 14, 2022
1 parent 6cd5d28 commit cf96736
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions docs/sync-status.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,21 @@ The DataHub UI will display the sync status in the top right corner of the page.

The last synchronized date is basically the last time an ingestion run saw an entity. It is computed as the most recent update to the entity, excluding changes done through the UI. If an ingestion run restates an entity but doesn't actually cause any changes, we still count that as an update for the purposes of sync status.

<details>
<summary>Technical details: computing the last synchronized timestamp</summary>

To compute the last synchronized timestamp, we look at the system metadata of all aspects associated with the entity.
We exclude any aspects where the system metadata `runId` value is unset or equal to `no-run-id-provided`, as this is what filters out changes made through the UI.
Finally, we take the most recent system metadata `lastObserved` timestamp across the aspects and use that as the last synchronized timestamp.

</details>

<p align="center">
<img width="70%" src="https://raw.githubusercontent.com/datahub-project/static-assets/master/imgs/sync-status-normal.png"/>
</p>

We'll automatically assign a color based on the sync status recency:

- Green: last synchronized in the past week
- Yellow: last synchronized in the past month
- Red: last synchronized more than a month ago
Expand Down

0 comments on commit cf96736

Please sign in to comment.