-
Notifications
You must be signed in to change notification settings - Fork 28.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add insertInto and saveAsTable to Python API. #447
Conversation
lgtm |
Merged build triggered. |
Merged build started. |
@@ -305,6 +305,19 @@ def registerAsTable(self, name): | |||
""" | |||
self._jschema_rdd.registerAsTable(name) | |||
|
|||
def insertInto(self, tableName, overwrite = False): | |||
""" | |||
Inserts the contents of this SchemaRDD into the specified table, optionally overwriting any |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make it 72 chars wide max for docstring:
Note that this does not apply for docstrings: docstrings should be at most 72 characters wide to display properly in most Python consoles.
Merged build finished. All automated tests passed. |
All automated tests passed. |
Merged build triggered. |
Merged build started. |
Merged build finished. All automated tests passed. |
All automated tests passed. |
Merged, thanks. |
Author: Michael Armbrust <[email protected]> Closes #447 from marmbrus/pythonInsert and squashes the following commits: c7ab692 [Michael Armbrust] Keep docstrings < 72 chars. ff62870 [Michael Armbrust] Add insertInto and saveAsTable to Python API. (cherry picked from commit 10d0421) Signed-off-by: Matei Zaharia <[email protected]>
fix for SPARK-1027 fix for SPARK-1027 (https://spark-project.atlassian.net/browse/SPARK-1027) FIXES 1. change sparkhome from String to Option(String) in ApplicationDesc 2. remove sparkhome parameter in LaunchExecutor message 3. adjust involved files
Author: Michael Armbrust <[email protected]> Closes apache#447 from marmbrus/pythonInsert and squashes the following commits: c7ab692 [Michael Armbrust] Keep docstrings < 72 chars. ff62870 [Michael Armbrust] Add insertInto and saveAsTable to Python API.
#447) * Fail submission if submitter-local files are provided without resource staging server URI * Modified logic to validate only submitted jars; added orchestrator tests * Incorporated feedback * Fix failing test case
apache#447) * Fail submission if submitter-local files are provided without resource staging server URI * Modified logic to validate only submitted jars; added orchestrator tests * Incorporated feedback * Fix failing test case
Update the pipeline and jobs to add 1.13 K8s
* Spark Release * delete LocalFileAppender Co-authored-by: Enwei Jiao <[email protected]>
No description provided.