Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ingest): use entry point for registering transformers #6628

Merged

Conversation

Masterchen09
Copy link
Contributor

This PR adds an entry point for registring transformers similar to the entry points for the sources and sinks. This makes it possible to refer to custom transformers installed by 3rd party packages using the shortened alias instead of the full path in recipes.

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions github-actions bot added the ingestion PR or Issue related to the ingestion of metadata label Dec 4, 2022
@github-actions
Copy link

github-actions bot commented Dec 4, 2022

Unit Test Results (metadata ingestion)

       8 files         8 suites   1h 1m 46s ⏱️
   752 tests    750 ✔️ 2 💤 0
1 506 runs  1 501 ✔️ 5 💤 0

Results for commit bf65236.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Dec 4, 2022

Unit Test Results (build & test)

633 tests   629 ✔️  15m 49s ⏱️
162 suites      4 💤
162 files        0

Results for commit bf65236.

♻️ This comment has been updated with latest results.

@Masterchen09 Masterchen09 force-pushed the feature-transformer-entrypoint branch from b41430e to 5f2e022 Compare December 4, 2022 11:24
@anshbansal anshbansal added the community-contribution PR or Issue raised by member(s) of DataHub Community label Dec 6, 2022
Copy link
Collaborator

@hsheth2 hsheth2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good - just requesting a small docs update

metadata-ingestion/docs/transformer/dataset_transformer.md Outdated Show resolved Hide resolved
@Masterchen09 Masterchen09 force-pushed the feature-transformer-entrypoint branch from 5f2e022 to bf65236 Compare December 7, 2022 22:22
Copy link
Collaborator

@hsheth2 hsheth2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@hsheth2 hsheth2 merged commit 05e18a0 into datahub-project:master Dec 8, 2022
@Masterchen09 Masterchen09 deleted the feature-transformer-entrypoint branch December 8, 2022 21:30
cccs-Dustin pushed a commit to CybercentreCanada/datahub that referenced this pull request Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community-contribution PR or Issue raised by member(s) of DataHub Community ingestion PR or Issue related to the ingestion of metadata
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants