Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-7269] [SQL] Incorrect analysis for aggregation (new implementation) #6110

Closed
wants to merge 1 commit into from
Closed

[SPARK-7269] [SQL] Incorrect analysis for aggregation (new implementation) #6110

wants to merge 1 commit into from

Conversation

chenghao-intel
Copy link
Contributor

This is another implementation other than #5798

@chenghao-intel chenghao-intel mentioned this pull request May 13, 2015
Closed
@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@SparkQA
Copy link

SparkQA commented May 13, 2015

Test build #32588 has started for PR 6110 at commit 5c2162e.

@SparkQA
Copy link

SparkQA commented May 13, 2015

Test build #32588 has finished for PR 6110 at commit 5c2162e.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Merged build finished. Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32588/
Test FAILed.

@chenghao-intel
Copy link
Contributor Author

retest this please

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@SparkQA
Copy link

SparkQA commented May 14, 2015

Test build #32701 has started for PR 6110 at commit 5c2162e.

@SparkQA
Copy link

SparkQA commented May 14, 2015

Test build #32701 has finished for PR 6110 at commit 5c2162e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32701/
Test PASSed.

cloud-fan
cloud-fan reviewed May 14, 2015
View reviewed changes
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
case x: Expression if nonSelectedGroupExprSet.contains(x) =>
case x: Expression
if nonSelectedGroupExprs.find(
ExpressionEquality(_) == ExpressionEquality(x)).isDefined =>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we add semanticEquals method to Expression and implement it with equals as default? Then we can override semanticEquals for AttributeReference and just write _.semanticEquals(x) here.
ExpressionEquality seems a little hack to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But you need to add semanticEquals for all of the non-leaf expression classes.

@cloud-fan cloud-fan mentioned this pull request May 15, 2015
Closed
asfgit pushed a commit that referenced this pull request May 18, 2015
@cloud-fan @marmbrus
[SPARK-7269] [SQL] Incorrect analysis for aggregation(use semanticEqu…
d6f5f37 
…als)

A modified version of #6110, use `semanticEquals` to make it more efficient.

Author: Wenchen Fan <[email protected]>

Closes #6173 from cloud-fan/7269 and squashes the following commits:

e4a3cc7 [Wenchen Fan] address comments
cc02045 [Wenchen Fan] consider elements length equal
d7ff8f4 [Wenchen Fan] fix 7269

(cherry picked from commit 103c863)
Signed-off-by: Michael Armbrust <[email protected]>
asfgit pushed a commit that referenced this pull request May 18, 2015
@cloud-fan @marmbrus
[SPARK-7269] [SQL] Incorrect analysis for aggregation(use semanticEqu…
103c863 
…als)

A modified version of #6110, use `semanticEquals` to make it more efficient.

Author: Wenchen Fan <[email protected]>

Closes #6173 from cloud-fan/7269 and squashes the following commits:

e4a3cc7 [Wenchen Fan] address comments
cc02045 [Wenchen Fan] consider elements length equal
d7ff8f4 [Wenchen Fan] fix 7269
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request May 28, 2015
@cloud-fan @jeanlyn
[SPARK-7269] [SQL] Incorrect analysis for aggregation(use semanticEqu…
dd61c2a 
…als)

A modified version of apache#6110, use `semanticEquals` to make it more efficient.

Author: Wenchen Fan <[email protected]>

Closes apache#6173 from cloud-fan/7269 and squashes the following commits:

e4a3cc7 [Wenchen Fan] address comments
cc02045 [Wenchen Fan] consider elements length equal
d7ff8f4 [Wenchen Fan] fix 7269
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request Jun 12, 2015
@cloud-fan @jeanlyn
[SPARK-7269] [SQL] Incorrect analysis for aggregation(use semanticEqu…
1a45aef 
…als)

A modified version of apache#6110, use `semanticEquals` to make it more efficient.

Author: Wenchen Fan <[email protected]>

Closes apache#6173 from cloud-fan/7269 and squashes the following commits:

e4a3cc7 [Wenchen Fan] address comments
cc02045 [Wenchen Fan] consider elements length equal
d7ff8f4 [Wenchen Fan] fix 7269
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
@cloud-fan @nemccarthy
[SPARK-7269] [SQL] Incorrect analysis for aggregation(use semanticEqu…
f314484 
…als)

A modified version of apache#6110, use `semanticEquals` to make it more efficient.

Author: Wenchen Fan <[email protected]>

Closes apache#6173 from cloud-fan/7269 and squashes the following commits:

e4a3cc7 [Wenchen Fan] address comments
cc02045 [Wenchen Fan] consider elements length equal
d7ff8f4 [Wenchen Fan] fix 7269
@chenghao-intel chenghao-intel deleted the analysis2 branch July 2, 2015 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@chenghao-intel @AmplabJenkins @SparkQA @cloud-fan