Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upstream #6

Merged
merged 26 commits into from
Mar 16, 2022
Merged

Upstream #6

merged 26 commits into from
Mar 16, 2022

Conversation

leifker
Copy link
Owner

@leifker leifker commented Mar 16, 2022

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable)

anshbansal and others added 26 commits March 14, 2022 08:50
…tity page (#4410)

* query for custom properties on containers

* updating queries

* hide soft deleted entities in lineage viz

* add banner for soft deleted entities

* fix tests
…#4358)

* Changes rollback behaviour to apply soft deletes by default

Summary:
Addresses feature request: Flag in delete command to only delete aspects touched by an ingestion run; add flag to nuke everything by modifying the default behaviour of a rollback operation which will not by default delete an entity if a keyAspect is being rolled-back.

Instead the key aspect is kept and a StatusAspect is upserted with removed=true, effectively making a soft delete.
Another PR will follow to perform garbage collection on these soft deleted entities.

To keep old behaviour, a new parameter to the cli ingest rollback endpoint: --hard-delete was added.

* Adds restli specs

* Fixes deleteAspect endpoint & adds support for nested transactions

* Enable regression test & fix docker-compose for local development

* Add generated quickstart

* Fix quickstart generation script

* Adds missing var env to docker-compose-without-neo4j

* Sets status removed=true when ingesting resources

* Adds soft deletes for ElasticSearch + soft delete flags across ingestion sub-commands

* Makes elastic search consistent

* Update tests with new behaviour

* apply review comments

* apply review comment

* Forces Elastic search to add documents with status removed false when ingesting

* Reset gradle properties to default

* Fix tests
…#4423)

* query for custom properties on containers

* retry login in cypress

* syntax fix
…4421)

* fix(ingest): extract redshift platform from sqlalchemy uri
@leifker leifker merged commit 0b58f4c into leifker:master Mar 16, 2022
@github-actions
Copy link

Unit Test Results (build & test)

  92 files  ±0    92 suites  ±0   19m 33s ⏱️ + 8m 19s
673 tests +1  614 ✔️ +2  59 💤 ±0  0  - 1 

Results for commit 9025bfb. ± Comparison against base commit 175cad1.

@github-actions
Copy link

Unit Test Results (metadata ingestion)

       5 files         5 suites   55m 5s ⏱️
   375 tests    358 ✔️   0 💤 4  13 🔥
1 650 runs  1 602 ✔️ 31 💤 4  13 🔥

For more details on these failures and errors, see this check.

Results for commit 9025bfb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.