Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SQL] Add toString to DataFrame/Column #4436

Closed
wants to merge 2 commits into from
Closed

[SQL] Add toString to DataFrame/Column #4436

wants to merge 2 commits into from

Conversation

marmbrus
Copy link
Contributor

@marmbrus marmbrus commented Feb 6, 2015

No description provided.

@SparkQA
Copy link

SparkQA commented Feb 6, 2015

Test build #26944 has finished for PR 4436 at commit ba52968.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class UnresolvedGetField(child: Expression, fieldName: String) extends UnaryExpression
    • case class GetField(child: Expression, field: StructField, ordinal: Int) extends UnaryExpression
    • case class PrettyAttribute(name: String) extends Attribute with trees.LeafNode[Expression]
    • implicit class DebugSQLContext(sqlContext: SQLContext)

@scwf scwf mentioned this pull request Feb 7, 2015
Closed
@SparkQA
Copy link

SparkQA commented Feb 7, 2015

Test build #593 has finished for PR 4436 at commit ba52968.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Feb 9, 2015

Test build #27056 has finished for PR 4436 at commit b91825b.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class PrettyAttribute(name: String) extends Attribute with trees.LeafNode[Expression]
    • implicit class DebugSQLContext(sqlContext: SQLContext)

@marmbrus
Copy link
Contributor Author

marmbrus commented Feb 9, 2015

test this please

@SparkQA
Copy link

SparkQA commented Feb 9, 2015

Test build #27059 has finished for PR 4436 at commit 04be78e.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class PrettyAttribute(name: String) extends Attribute with trees.LeafNode[Expression]
    • implicit class DebugSQLContext(sqlContext: SQLContext)

@SparkQA
Copy link

SparkQA commented Feb 9, 2015

Test build #27063 has finished for PR 4436 at commit 04be78e.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class PrettyAttribute(name: String) extends Attribute with trees.LeafNode[Expression]
    • implicit class DebugSQLContext(sqlContext: SQLContext)

@marmbrus
Copy link
Contributor Author

marmbrus commented Feb 9, 2015

test this please

@SparkQA
Copy link

SparkQA commented Feb 9, 2015

Test build #27067 has finished for PR 4436 at commit 04be78e.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class PrettyAttribute(name: String) extends Attribute with trees.LeafNode[Expression]
    • implicit class DebugSQLContext(sqlContext: SQLContext)

@SparkQA
Copy link

SparkQA commented Feb 10, 2015

Test build #595 has finished for PR 4436 at commit b72a81b.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • "public class " + className + extendsText + " implements java.io.Serializable
    • case class MasterStateResponse(
    • class LocalSparkCluster(
    • * (4) the main class for the child
    • case class BoundPortsResponse(actorPort: Int, webUIPort: Int, restPort: Option[Int])
    • throw new SubmitRestMissingFieldException("Main class is missing.")
    • case class RegisterExecutor(
    • public class JavaDeveloperApiExample
    • // where index i corresponds to class i (i = 0, 1).
    • public class JavaLDAExample
    • * Here, we have a trait to be mixed in with the Estimator and Model (MyLogisticRegression
    • * class since the maxIter parameter is only used during training (not in the Model).
    • // where index i corresponds to class i (i = 0, 1).
    • class DoubleParam(parent: Params, name: String, doc: String, defaultValue: Option[Double])
    • class IntParam(parent: Params, name: String, doc: String, defaultValue: Option[Int])
    • class FloatParam(parent: Params, name: String, doc: String, defaultValue: Option[Float])
    • class LongParam(parent: Params, name: String, doc: String, defaultValue: Option[Long])
    • class BooleanParam(parent: Params, name: String, doc: String, defaultValue: Option[Boolean])
    • new Param(this, "probabilityCol", "column name for predicted class conditional probabilities",
    • case class UnresolvedGetField(child: Expression, fieldName: String) extends UnaryExpression
    • case class GetField(child: Expression, field: StructField, ordinal: Int) extends UnaryExpression
    • case class PrettyAttribute(name: String) extends Attribute with trees.LeafNode[Expression]
    • implicit class DebugSQLContext(sqlContext: SQLContext)

@marmbrus
Copy link
Contributor Author

test this please

@SparkQA
Copy link

SparkQA commented Feb 10, 2015

Test build #27145 has finished for PR 4436 at commit b72a81b.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class PrettyAttribute(name: String) extends Attribute with trees.LeafNode[Expression]
    • implicit class DebugSQLContext(sqlContext: SQLContext)

@marmbrus
Copy link
Contributor Author

sigh... test this please

@SparkQA
Copy link

SparkQA commented Feb 10, 2015

Test build #27162 has finished for PR 4436 at commit b72a81b.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class PrettyAttribute(name: String) extends Attribute with trees.LeafNode[Expression]
    • implicit class DebugSQLContext(sqlContext: SQLContext)

@SparkQA
Copy link

SparkQA commented Feb 10, 2015

Test build #27219 has finished for PR 4436 at commit 8a3c35f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class PrettyAttribute(name: String) extends Attribute with trees.LeafNode[Expression]
    • implicit class DebugSQLContext(sqlContext: SQLContext)

asfgit pushed a commit that referenced this pull request Feb 10, 2015
@marmbrus
[SQL] Add toString to DataFrame/Column
ef739d9 
Author: Michael Armbrust <[email protected]>

Closes #4436 from marmbrus/dfToString and squashes the following commits:

8a3c35f [Michael Armbrust] Merge remote-tracking branch 'origin/master' into dfToString
b72a81b [Michael Armbrust] add toString

(cherry picked from commit de80b1b)
Signed-off-by: Michael Armbrust <[email protected]>
@asfgit asfgit closed this in de80b1b Feb 10, 2015
@marmbrus marmbrus deleted the dfToString branch February 17, 2015 20:56
scwf
scwf reviewed Sep 24, 2015
View reviewed changes
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
* debugging information like the expression id.
*/
def prettyString: String = {
transform {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why use transform here? this will change the expression

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only replace some leaf nodes in the expression tree, does it cause problems?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, i am going through the code, and i think we'd better not use transform here. in which case we need change the expression?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@marmbrus @SparkQA @cloud-fan @scwf