[hive-csv] specify hive namespace for tables #5268
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
john-bodley
reviewed
Jun 22, 2018
superset/config.py
Outdated
| @@ -314,6 +314,10 @@ class CeleryConfig(object): | |||
| # contain all the external tables | |||
| CSV_TO_HIVE_UPLOAD_DIRECTORY = 'EXTERNAL_HIVE_TABLES/' | |||
|
|
|||
| # The namespace within hive where the tables created from | |||
| # uploading CSVs will be stored | |||
| UPLOADED_CSV_HIVE_NAMESPACE = 'superset' | |||
There was a problem hiding this comment.
The new default changes the existing behvaior. You may want to considering setting this as None.
superset/db_engine_specs.py
Outdated
| if config.get('UPLOADED_CSV_HIVE_NAMESPACE'): | ||
| if '.' in table_name: | ||
| raise Exception( | ||
| """You can't specify a namespace. """ |
There was a problem hiding this comment.
If you’re going to use “”” which reprsents a multiline string then make the string multiline. It’s probably more desirable to use single quoted strings and wrap either via \ or encapsulate in ().
timifasubaa
force-pushed
the
specify_hive_namespace
branch
3 times, most recently
from
0985bc9 to
7786852
Compare
Codecov Report
@@ Coverage Diff @@
## master #5268 +/- ##
==========================================
- Coverage 63.29% 63.28% -0.02%
==========================================
Files 259 259
Lines 19693 19698 +5
Branches 1968 1968
==========================================
+ Hits 12465 12466 +1
- Misses 7219 7223 +4
Partials 9 9
Continue to review full report at Codecov.
|
john-bodley
reviewed
Jun 23, 2018
superset/db_engine_specs.py
Outdated
| if config.get('UPLOADED_CSV_HIVE_NAMESPACE'): | ||
| if '.' in table_name: | ||
| raise Exception( | ||
| "You can't specify a namespace." |
There was a problem hiding this comment.
Nit. We need a space after the period.
john-bodley
reviewed
Jun 23, 2018
superset/db_engine_specs.py
Outdated
| if '.' in table_name: | ||
| raise Exception( | ||
| "You can't specify a namespace." | ||
| 'All tables will be uploaded to the `{}` namespace'.format( |
There was a problem hiding this comment.
I would probably use ' rather than a backtick here.
timifasubaa
force-pushed
the
specify_hive_namespace
branch
from
7786852 to
bf8a246
Compare
john-bodley
approved these changes
Jun 23, 2018
paulbramsen
approved these changes
Jun 23, 2018
mistercrunch
pushed a commit
that referenced
this pull request
Jun 28, 2018
(cherry picked from commit bd24f85)
timifasubaa
added a commit
to airbnb/superset-fork
that referenced
this pull request
Jul 25, 2018
timifasubaa
added a commit
to airbnb/superset-fork
that referenced
this pull request
Sep 19, 2018
(cherry picked from commit bd24f85)
graceguo-supercat
pushed a commit
to airbnb/superset-fork
that referenced
this pull request
Sep 19, 2018
Cherrypick csv upload changes (apache#5268)
wenchma
pushed a commit
to wenchma/incubator-superset
that referenced
this pull request
Nov 16, 2018
mistercrunch
added
🍒 0.26.0
🍒 0.26.1
🍒 0.26.2
🍒 0.26.3
🍒 0.27.0
🏷️ bot
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR gives the administrators the option of forcing uploaded CSVs into a specific namespace in Hive.
@john-bodley @mistercrunch
cc @paulbramsen