Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-8074] Parquet should throw AnalysisException during setup for data type/name related failures. #6608

Closed
wants to merge 2 commits into from

Conversation

rxin
Copy link
Contributor

@rxin rxin commented Jun 3, 2015

No description provided.

@liancheng
Copy link
Contributor

We may also want to replace the sys.error in this line with the following:

              .getOrElse {
                throw new AnalysisException(
                  s"Failed to discover schema of Parquet file(s) in the following location(s):\n" +
                    paths.mkString("\n\t"))
              }

Otherwise LGTM.

@SparkQA
Copy link

SparkQA commented Jun 3, 2015

Test build #34079 timed out for PR 6608 at commit 5617cf6 after a configured wait of 150m.

@rxin
Copy link
Contributor Author

rxin commented Jun 3, 2015

Jenkins, retest this please.

@liancheng
Copy link
Contributor

I think we should also replace all asserts and requires along the way with AnalysisException.

@liancheng
Copy link
Contributor

Probably not all, but those related to analysis errors.

@rxin
Copy link
Contributor Author

rxin commented Jun 3, 2015

Let's do that later. We should audit all exceptions and decide what to do.

@SparkQA
Copy link

SparkQA commented Jun 3, 2015

Test build #34093 has finished for PR 6608 at commit 5617cf6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 3, 2015

Test build #34094 has finished for PR 6608 at commit b5dc8e2.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • trait TypeCheckResult
    • case class TypeCheckFailure(message: String) extends TypeCheckResult
    • abstract class UnaryArithmetic extends UnaryExpression
    • case class UnaryMinus(child: Expression) extends UnaryArithmetic
    • case class Sqrt(child: Expression) extends UnaryArithmetic
    • case class Abs(child: Expression) extends UnaryArithmetic
    • case class BitwiseNot(child: Expression) extends UnaryArithmetic
    • case class MaxOf(left: Expression, right: Expression) extends BinaryArithmetic
    • case class MinOf(left: Expression, right: Expression) extends BinaryArithmetic
    • case class Atan2(left: Expression, right: Expression)
    • case class Hypot(left: Expression, right: Expression)
    • case class EqualTo(left: Expression, right: Expression) extends BinaryComparison

asfgit pushed a commit that referenced this pull request Jun 3, 2015
…data type/name related failures.

Author: Reynold Xin <[email protected]>

Closes #6608 from rxin/parquet-analysis and squashes the following commits:

b5dc8e2 [Reynold Xin] Code review feedback.
5617cf6 [Reynold Xin] [SPARK-8074] Parquet should throw AnalysisException during setup for data type/name related failures.

(cherry picked from commit 939e4f3)
Signed-off-by: Reynold Xin <[email protected]>
@asfgit asfgit closed this in 939e4f3 Jun 3, 2015
@rxin rxin deleted the parquet-analysis branch June 4, 2015 01:15
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request Jun 12, 2015
…data type/name related failures.

Author: Reynold Xin <[email protected]>

Closes apache#6608 from rxin/parquet-analysis and squashes the following commits:

b5dc8e2 [Reynold Xin] Code review feedback.
5617cf6 [Reynold Xin] [SPARK-8074] Parquet should throw AnalysisException during setup for data type/name related failures.
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
…data type/name related failures.

Author: Reynold Xin <[email protected]>

Closes apache#6608 from rxin/parquet-analysis and squashes the following commits:

b5dc8e2 [Reynold Xin] Code review feedback.
5617cf6 [Reynold Xin] [SPARK-8074] Parquet should throw AnalysisException during setup for data type/name related failures.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants