Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(urn-validation): Add UrnValidation PDL annotation #12572

Conversation

david-leifker
Copy link
Collaborator

@david-leifker david-leifker commented Feb 7, 2025

  • Apply UrnValidation logic to StructuredPropertyDefinition valueType
  • Consolidate validation utils to metadata-utils
  • Does not apply to RESTATE to allow restoreIndices to function

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions github-actions bot added docs Issues and Improvements to docs product PR or Issue related to the DataHub UI/UX devops PR or Issue related to DataHub backend & deployment smoke_test Contains changes related to smoke tests labels Feb 7, 2025
Copy link

codecov bot commented Feb 7, 2025

Codecov Report

Attention: Patch coverage is 69.87448% with 72 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...com/linkedin/metadata/utils/UrnValidationUtil.java 50.51% 41 Missing and 7 partials ⚠️
...ory/plugins/SpringStandardPluginConfiguration.java 0.00% 10 Missing ⚠️
...data/aspect/validation/UrnAnnotationValidator.java 93.75% 1 Missing and 4 partials ⚠️
...in/metadata/models/annotation/AnnotationUtils.java 70.00% 1 Missing and 2 partials ⚠️
...ata/models/annotation/UrnValidationAnnotation.java 62.50% 2 Missing and 1 partial ⚠️
.../java/com/linkedin/metadata/models/AspectSpec.java 83.33% 1 Missing ⚠️
...om/linkedin/metadata/models/EntitySpecBuilder.java 88.88% 1 Missing ⚠️
...tadata/models/UrnValidationFieldSpecExtractor.java 94.73% 1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (57.86%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Files with missing lines Coverage Δ
...edin/metadata/aspect/utils/DefaultAspectsUtil.java 84.67% <ø> (ø)
...om/linkedin/metadata/entity/EntityServiceImpl.java 79.67% <ø> (ø)
...java/com/linkedin/metadata/entity/EntityUtils.java 77.69% <ø> (ø)
...roject/openapi/v2/controller/EntityController.java 0.00% <ø> (ø)
...roject/openapi/v3/controller/EntityController.java 52.95% <ø> (ø)
...va/com/linkedin/metadata/utils/EntityApiUtils.java 25.00% <ø> (ø)
...din/metadata/utils/EntityRegistryUrnValidator.java 60.00% <ø> (ø)
...nkedin/metadata/utils/RecordTemplateValidator.java 53.84% <ø> (ø)
.../java/com/linkedin/metadata/models/AspectSpec.java 84.37% <83.33%> (-0.11%) ⬇️
...om/linkedin/metadata/models/EntitySpecBuilder.java 68.83% <88.88%> (+0.87%) ⬆️
... and 6 more

... and 7 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7472c53...83b19cb. Read the comment docs.

@datahub-cyborg datahub-cyborg bot added the needs-review Label for PRs that need review from a maintainer. label Feb 7, 2025
@david-leifker david-leifker force-pushed the urn-existence-validation-annotation branch from 0f71052 to 2c53b38 Compare February 7, 2025 15:28
@datahub-cyborg datahub-cyborg bot added pending-submitter-merge and removed needs-review Label for PRs that need review from a maintainer. labels Feb 7, 2025
@@ -494,6 +495,27 @@ This annotation says that when we ingest an Entity with an Ownership Aspect, Dat
between that entity and the CorpUser or CorpGroup who owns it. This will be queryable using the Relationships resource
in both the forward and inverse directions.

#### @UrnValidation

This annotation can be applied to Urn fields inside an aspect. The annotation can optionally perform one or more of the following:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there plans for it be applied to be required for all urn fields? It would add reference consistency to the metadata graph.

Copy link
Collaborator

@esteban esteban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, @david-leifker!

* Apply UrnValidation logic to StructuredPropertyDefinition valueType
@david-leifker david-leifker force-pushed the urn-existence-validation-annotation branch from 2c53b38 to c7df219 Compare February 13, 2025 00:42
@david-leifker
Copy link
Collaborator Author

mutations/managing_secrets.js passes locally

@david-leifker david-leifker merged commit 7326bb9 into datahub-project:master Feb 13, 2025
196 of 200 checks passed
ttekampe pushed a commit to ttekampe/datahub that referenced this pull request Feb 14, 2025
ksrinath pushed a commit to ksrinath/datahub that referenced this pull request Feb 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops PR or Issue related to DataHub backend & deployment docs Issues and Improvements to docs pending-submitter-merge product PR or Issue related to the DataHub UI/UX smoke_test Contains changes related to smoke tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants