Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add modal to import databases #11884

Merged
merged 5 commits into from
Dec 7, 2020
Merged

Conversation

betodealmeida
Copy link
Member

@betodealmeida betodealmeida commented Dec 2, 2020

SUMMARY

This PR adds a new button to import databases from the UI if the VERSIONED_EXPORT feature flag is enabled. It allows users to import databases in the new format proposed in #11167.

The biggest challenge in importing databases is that we export them without passwords, extra security information, nor certificates. When importing database we can inspect the sqlalchemy_uri attribute and see if it has a masked password (XXXXXXXX), and if true we can prompt the user for the password.

The second biggest challenge is that we don't know how many databases are present in the imported file, since users can export multiple databases at once in a single file. We also don't know how many of those require passwords, since not all databases use them.

To work around this when the user tries to import a file the frontend will upload it to the backend. The backend performs validation on the file, and if there are databases with masked passwords it returns a validation error (422). The frontend will parse the error message, and if the only errors are from missing passwords it will prompt the user for the passwords and resubmit the file, this time with the associated passwords.

This PR adds the backend validation, as well as the import modal. For the API calls I created a new hook similar to useSingleViewResource called useImportResource, which will be reused for other imports (dataset, chart, dashboard and saved queries).

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

output

TEST PLAN

Manually tested importing a database, and confirmed it prompts for the password and the database is configured correctly.

Also added tests.

ADDITIONAL INFORMATION

@betodealmeida
Copy link
Member Author

cc: @eschutho

@codecov-io
Copy link

codecov-io commented Dec 2, 2020

Codecov Report

Merging #11884 (5fb717c) into master (a7bba92) will decrease coverage by 3.93%.
The diff coverage is 51.81%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #11884      +/-   ##
==========================================
- Coverage   64.45%   60.52%   -3.94%     
==========================================
  Files         937      891      -46     
  Lines       45319    43517    -1802     
  Branches     4317     3841     -476     
==========================================
- Hits        29211    26339    -2872     
- Misses      15948    17178    +1230     
+ Partials      160        0     -160     
Flag Coverage Δ
cypress 54.99% <37.64%> (+27.11%) ⬆️
javascript ?
python 63.42% <100.00%> (-0.77%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset-frontend/src/views/CRUD/hooks.ts 60.40% <17.24%> (-5.49%) ⬇️
...tend/src/database/components/ImportModal/index.tsx 44.18% <44.18%> (ø)
...tend/src/views/CRUD/data/database/DatabaseList.tsx 67.46% <54.54%> (-11.85%) ⬇️
...end/src/views/CRUD/data/database/DatabaseModal.tsx 69.11% <100.00%> (+13.78%) ⬆️
superset/databases/api.py 87.44% <100.00%> (-1.54%) ⬇️
...uperset/databases/commands/importers/dispatcher.py 80.64% <100.00%> (+1.33%) ⬆️
...perset/databases/commands/importers/v1/__init__.py 97.14% <100.00%> (+0.17%) ⬆️
superset/databases/schemas.py 100.00% <100.00%> (ø)
superset/models/core.py 85.63% <100.00%> (-3.23%) ⬇️
superset/models/helpers.py 89.78% <100.00%> (+0.04%) ⬆️
... and 405 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a7bba92...5fb717c. Read the comment docs.

@hughhhh hughhhh self-requested a review December 7, 2020 17:50
Copy link
Member

@hughhhh hughhhh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

type="password"
value={passwords[fileName]}
onChange={event =>
setPasswords({ ...passwords, [fileName]: event.target.value })
Copy link
Member

@hughhhh hughhhh Dec 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

put this in a handleStyleContainerInputChange function at the top for consistency

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do that in #11936, so I only have to do it once.

@betodealmeida betodealmeida merged commit 2b9695c into apache:master Dec 7, 2020
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.0.0 labels Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/XL 🚢 1.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants