[SQL][DataFrame] Fix column computability bug. #4519
Conversation
Do not recursively strip out projects. Only strip the first level project.
|
Test build #27261 has started for PR 4519 at commit
|
| private def computableCol(baseCol: ComputableColumn, expr: Expression) = { | ||
| val plan = Project(Seq(expr match { | ||
| case named: NamedExpression => named | ||
| case unnamed: Expression => Alias(unnamed, "col")() | ||
| }), baseCol.plan) | ||
| }), stripProject(baseCol.plan)) |
There was a problem hiding this comment.
Nit: I would consider splitting this up since matching in the constructor is not super easy to read.
|
Test build #27263 has started for PR 4519 at commit
|
|
Test build #27261 has finished for PR 4519 at commit
|
|
Test FAILed. |
|
This passed catalyst/sql/hive tests. The core tests are know flakey. Merging after talking with @pwendell so this can be included in the snapshot. |
|
test this please |
Do not recursively strip out projects. Only strip the first level project.
```scala
df("colA") + df("colB").as("colC")
```
Previously, the above would construct an invalid plan.
Author: Reynold Xin <[email protected]>
Closes #4519 from rxin/computability and squashes the following commits:
87ff763 [Reynold Xin] Code review feedback.
015c4fc [Reynold Xin] [SQL][DataFrame] Fix column computability.
(cherry picked from commit 7e24249)
Signed-off-by: Michael Armbrust <[email protected]>
|
Test build #27263 has finished for PR 4519 at commit
|
|
Test PASSed. |
Do not recursively strip out projects. Only strip the first level project.
Previously, the above would construct an invalid plan.