Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add resource_reports field in Table API ( Atlas proxy) #149

Merged
merged 9 commits into from
Jul 21, 2020

Conversation

dechoma
Copy link
Contributor

@dechoma dechoma commented Jul 8, 2020

Summary of Changes

Add resource_reports field in Table API ( atlas proxy) to support integration with tools generating static HTML reports, such as popmon, pandas-profiling or great_expectations..

Tests

Updated existing unit tests

Documentation

N/A

CheckList

Make sure you have checked all steps below to ensure a timely review.

  • PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"
  • PR includes a summary of changes.
  • PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does
  • PR passes make test

Resolve amundsen-io/amundsen#525

@mgorsk1
Copy link
Contributor

mgorsk1 commented Jul 8, 2020

cc @feng-tao @verdan

Copy link
Member

@verdan verdan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small comment, the rest looks good to me.

@@ -377,6 +402,8 @@ def get_table(self, *, table_uri: str) -> Table:

columns = self._serialize_columns(entity=entity)

reports_guids = [report.get("guid") for report in attrs.get("reports") or list()]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dechoma is it possible to filter out the active reports here already ? I think you'll have the active status available here, so no need to make a query with the inactive queries.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, attributes contain only report typeName and guid{'owner': 'testuser', 'temporary': False, 'reports': [{'guid': '50160c85-cdd1-4bac-800a-9bb623b816d1', 'typeName': 'Report'}], ...() }

dechoma added 2 commits July 14, 2020 12:45
# Conflicts:
#	metadata_service/proxy/atlas_proxy.py
#	tests/unit/proxy/fixtures/atlas_test_data.py
#	tests/unit/proxy/test_atlas_proxy.py
@mgorsk1
Copy link
Contributor

mgorsk1 commented Jul 17, 2020

LGTM (just one nit)

wdyt @verdan ?

dechoma added 4 commits July 17, 2020 11:47
# Conflicts:
#	metadata_service/proxy/atlas_proxy.py
#	tests/unit/proxy/fixtures/atlas_test_data.py
#	tests/unit/proxy/test_atlas_proxy.py
@codecov-commenter
Copy link

Codecov Report

Merging #149 into master will increase coverage by 0.10%.
The diff coverage is 81.25%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #149      +/-   ##
==========================================
+ Coverage   73.25%   73.35%   +0.10%     
==========================================
  Files          26       26              
  Lines        1249     1265      +16     
  Branches      132      136       +4     
==========================================
+ Hits          915      928      +13     
- Misses        307      309       +2     
- Partials       27       28       +1     
Impacted Files Coverage Δ
metadata_service/proxy/atlas_proxy.py 84.24% <80.00%> (-0.25%) ⬇️
metadata_service/config.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 48dfa05...da7c8c2. Read the comment docs.

@feng-tao
Copy link
Member

cc @verdan
also could you bump the dependency?

Copy link
Member

@verdan verdan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@feng-tao
Copy link
Member

I will update the version later actually. Thanks for the pr and review :)

@feng-tao feng-tao merged commit 099a9ec into amundsen-io:master Jul 21, 2020
jerryzhu2007 pushed a commit to kylg/amundsenmetadatalibrary that referenced this pull request Aug 20, 2020
* commit '369685cc715e95af82dfa4dc14d0c58af8bb1ac9':
  chore: replace references to Lyft -> Amundsen (amundsen-io#174)
  feat: Data Owner Implementation of Atlas Proxy (amundsen-io#156)
  chore: fix docker push action (amundsen-io#172)
  chore: add docker publish action and remove travis (amundsen-io#171)
  chore: add pypi publish action (amundsen-io#170)
  fix: removing OidcConfig file and making statsd configurable through envrionment variable (amundsen-io#157)
  ci: add dependabot config (amundsen-io#169)
  Update repo name in travis file (amundsen-io#163)
  feat: Populate is_view property in AtlasProxy (amundsen-io#155)
  fix: Overlapping table name issue in Readers [AtlasProxy]
  feat: Add resource_reports field in Table API ( Atlas proxy) (amundsen-io#149)
  chore: apply license headers to all the source files (amundsen-io#153)
  feat: Add programmatic descriptions parser for [AtlasProxy] (amundsen-io#152)
  feat: Add Frequent Users feature in [AtlasProxy] (amundsen-io#147)
  feat: Implement configurable minimum number of readers for popular tables (amundsen-io#146)
  chore: update the email for the project (amundsen-io#148)

# Conflicts:
#	README.md
#	docs/configurations.md
#	docs/structure.md
#	metadata_service/config.py
#	metadata_service/oidc_config.py
#	metadata_service/proxy/neo4j_proxy.py
#	requirements.txt
#	setup.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants