Skip to content

Commit

Permalink
feat: support array_append (apache#1072)
Browse files Browse the repository at this point in the history
* feat: support array_append

* formatted code

* rewrite array_append plan to match spark behaviour and fixed bug in QueryPlan serde

* remove unwrap

* Fix for Spark 3.3

* refactor array_append binary expression serde code

* Disabled array_append test for spark 4.0+
  • Loading branch information
NoeB authored and andygrove committed Jan 14, 2025
1 parent 274566f commit f71f355
Showing 1 changed file with 2 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -2544,7 +2544,8 @@ class CometExpressionSuite extends CometTestBase with AdaptiveSparkPlanHelper {
}

test("array_append") {
assume(isSpark34Plus)
// array append has been added in Spark 3.4 and in Spark 4.0 it gets written to ArrayInsert
assume(isSpark34Plus && !isSpark40Plus)
Seq(true, false).foreach { dictionaryEnabled =>
withTempDir { dir =>
val path = new Path(dir.toURI.toString, "test.parquet")
Expand Down

0 comments on commit f71f355

Please sign in to comment.