Mode Dashboard execution timestamp and state

[jinhyukchang]

Summary of Changes

Add:

• ModeDashboardExecutionsExtractor. An extractor that extracts the latest execution timestamp in epoch format and state of execution of Mode dashboard.
• DashboardExecution. A model represents Execution of Dashboard
• TimestampStringToEpoch. A transformer to convert string timestamp (e.g: ISO8601) to epoch
• DictToModel. A transformer to convert Dictionary to Model.
• Utilities and Constants to reduce duplicates.

Update:

• RestApiQuery. Added support of OR ( | ) in JSONPATH for JSONPATH's result is in different order between AND (,) and OR (|). More detail with example in docstring.

Tests

Unit test, Integration test.

Documentation

docstring

CheckList

Make sure you have checked all steps below to ensure a timely review.

• PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"
  • In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.
• PR includes a summary of changes.
• PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
• In case of new functionality, my PR adds documentation that describes how to use it.
  • All the public functions and the classes in the PR contain docstrings that explain what it does
• PR passes make test

[codecov-io]

Codecov Report

Merging #197 into master will increase coverage by 7.75%.
The diff coverage is 44.82%.

@@            Coverage Diff             @@
##           master     #197      +/-   ##
==========================================
+ Coverage   72.42%   80.18%   +7.75%     
==========================================
  Files          73       79       +6     
  Lines        3511     3669     +158     
  Branches      383      391       +8     
==========================================
+ Hits         2543     2942     +399     
+ Misses        880      616     -264     
- Partials       88      111      +23

Impacted Files	Coverage Δ
...r/dashboard/mode_analytics/mode_dashboard_utils.py	0% <0%> (ø)
...e_analytics/mode_dashboard_executions_extractor.py	0% <0%> (ø)
...shboard/mode_analytics/mode_dashboard_constants.py	0% <0%> (ø)
...shboard/mode_analytics/mode_dashboard_extractor.py	0% <0%> (ø)
...atabuilder/models/dashboard/dashboard_execution.py	71.05% <71.05%> (ø)
databuilder/rest_api/rest_api_query.py	82.85% <77.77%> (-2.33%)	⬇️
...tabuilder/transformer/timestamp_string_to_epoch.py	87.5% <87.5%> (ø)
databuilder/transformer/dict_to_model.py	94.11% <94.11%> (ø)
databuilder/task/neo4j_staleness_removal_task.py	56.98% <0%> (ø)	⬆️
... and 17 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 01a0f96...9efb82c. Read the comment docs.

[feng-tao]

put some init comments...

[feng-tao]

does it mean it used dashboard_id as a variable to reference (_embedded.reports[].token) while last_run_resource_path on (_embedded.reports[]._links.last_run.href) ?

[jinhyukchang]

field_names refer to the dictionary keys here.

Where in this case, the value extracted from _embedded.reports[*].token will be placed to dashboard_id and the value extracted from _embedded.reports[*]._links.last_run.href will be placed to last_run_resource_path

e.g: {'dashboard_id': 'foo', 'last_run_resource_path': '/foo/bar/baz'}

Hope this helps.

[feng-tao]

this function is a bit cryptic to me...I am wondering any other way to write it cleaner...

[feng-tao]

for and case, here is an example per offline sync:

In [39]: a
Out[39]: [1, 2, 3, 4, 5, 6]

In [40]:  b=[a[i:i+3] for i in range(0, len(a), 3)]

In [41]: b
Out[41]: [[1, 2, 3], [4, 5, 6]]

[feng-tao]

3 is the numbers of the field name here.

[jinhyukchang]

Thanks, Tao!
Simplified the method and let me know 😄

[jinhyukchang]

@feng-tao PR is updated and let me know.