Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SIGABRT on DecimalAvgAggregate<UnscaleLongDecimal, UnscaleShortDecimal> when spilling is engaged #236

Merged
merged 2 commits into from
May 5, 2023
Merged

SIGABRT on DecimalAvgAggregate<UnscaleLongDecimal, UnscaleShortDecimal> when spilling is engaged #236

merged 2 commits into from
May 5, 2023

Conversation

zhztheplayer
Copy link
Collaborator

Error

Stack: [0x00007f7b58327000,0x00007f7b58428000],  sp=0x00007f7b58424750,  free space=1013k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libvelox.so+0xa0a8b9f]  facebook::velox::FlatVector<facebook::velox::UnscaledShortDecimal>::resize(int, bool)+0x2b
C  [libvelox.so+0xa0a0592]  facebook::velox::functions::sparksql::aggregates::DecimalAverageAggregate<facebook::velox::UnscaledLongDecimal, facebook::velox::UnscaledShortDecimal>::extractAccumulators(char**, int, std::shared_ptr<facebook::velox::BaseVector>*)+0xc6
C  [libvelox.so+0xa9146d0]  facebook::velox::exec::Spiller::extractSpill(folly::Range<char**>, std::shared_ptr<facebook::velox::RowVector>&)+0x29c
C  [libvelox.so+0xa914937]  facebook::velox::exec::Spiller::extractSpillVector(std::vector<char*, facebook::velox::memory::StlAllocator<char*> >&, int, long, std::shared_ptr<facebook::velox::RowVector>&, unsigned long&)+0x21f
C  [libvelox.so+0xa915590]  facebook::velox::exec::Spiller::writeSpill(int)+0x6dc
C  [libvelox.so+0xa91576c]  facebook::velox::exec::Spiller::advanceSpill()::{lambda()#1}::operator()() const+0x3c
C  [libvelox.so+0xa91c70c]  std::_Function_handler<std::unique_ptr<facebook::velox::exec::Spiller::SpillStatus, std::default_delete<facebook::velox::exec::Spiller::SpillStatus> > (), facebook::velox::exec::Spiller::advanceSpill()::{lambda()#1}>::_M_invoke(std::_Any_data const&)+0x41
C  [libvelox.so+0xa920ac4]  std::function<std::unique_ptr<facebook::velox::exec::Spiller::SpillStatus, std::default_delete<facebook::velox::exec::Spiller::SpillStatus> > ()>::operator()() const+0x50
C  [libvelox.so+0xa91eb0f]  facebook::velox::AsyncSource<facebook::velox::exec::Spiller::SpillStatus>::move()+0x21f
C  [libvelox.so+0xa915d8d]  facebook::velox::exec::Spiller::advanceSpill()+0x4c3
C  [libvelox.so+0xa9163eb]  facebook::velox::exec::Spiller::spill(unsigned long, unsigned long)+0x1f1
C  [libvelox.so+0xaa414a4]  facebook::velox::exec::GroupingSet::spill(long, long)+0x47a
C  [libvelox.so+0xaa40ce7]  facebook::velox::exec::GroupingSet::ensureInputFits(std::shared_ptr<facebook::velox::RowVector> const&)+0x4d1
C  [libvelox.so+0xaa3ea86]  facebook::velox::exec::GroupingSet::addInputForActiveRows(std::shared_ptr<facebook::velox::RowVector> const&, bool)+0x104
C  [libvelox.so+0xaa3e8fb]  facebook::velox::exec::GroupingSet::addInput(std::shared_ptr<facebook::velox::RowVector> const&, bool)+0x163
C  [libvelox.so+0xa98f146]  facebook::velox::exec::HashAggregation::addInput(std::shared_ptr<facebook::velox::RowVector>)+0xc8
C  [libvelox.so+0xa821613]  facebook::velox::exec::Driver::runInternal(std::shared_ptr<facebook::velox::exec::Driver>&, std::shared_ptr<facebook::velox::exec::BlockingState>&, std::shared_ptr<facebook::velox::RowVector>&)+0x9f5
C  [libvelox.so+0xa82067e]  facebook::velox::exec::Driver::next(std::shared_ptr<facebook::velox::exec::BlockingState>&)+0x72
C  [libvelox.so+0xa79e882]  facebook::velox::exec::Task::next(folly::SemiFuture<folly::Unit>*)+0x67a
C  [libvelox.so+0x86f0106]  gluten::WholeStageResultIterator::Next()+0xaa
C  [libvelox.so+0x86d3fd7]  gluten::ResultIterator::Wrapper<gluten::WholeStageResultIteratorMiddleStage>::Next()+0x41
C  [libvelox.so+0x86d3b46]  arrow::Result<std::shared_ptr<gluten::ColumnarBatch> > arrow::Iterator<std::shared_ptr<gluten::ColumnarBatch> >::Next<gluten::ResultIterator::Wrapper<gluten::WholeStageResultIteratorMiddleStage> >(void*)+0x36
C  [libgluten.so+0x6ad2a1]  arrow::Iterator<std::shared_ptr<gluten::ColumnarBatch> >::Next()+0x47
C  [libgluten.so+0x6a8f43]  gluten::ResultIterator::GetNext()+0x61
C  [libgluten.so+0x6a8d06]  gluten::ResultIterator::HasNext()+0x28
C  [libgluten.so+0x69ecd3]  Java_io_glutenproject_vectorized_ArrowOutIterator_nativeHasNext+0xb1
J 19200  io.glutenproject.vectorized.ArrowOutIterator.nativeHasNext(J)Z (0 bytes) @ 0x00007f7cb01ff980 [0x00007f7cb01ff8c0+0xc0]
J 29521 C2 io.glutenproject.backendsapi.glutendata.GlutenIteratorApi$$anon$3.hasNext()Z (37 bytes) @ 0x00007f7cb2374b04 [0x00007f7cb2374ac0+0x44]
J 23613 C2 io.glutenproject.vectorized.CloseableColumnBatchIterator.hasNext()Z (50 bytes) @ 0x00007f7cb04be060 [0x00007f7cb04be000+0x60]
J 27165 C2 org.apache.spark.shuffle.GlutenColumnarShuffleWriter.internalWrite(Lscala/collection/Iterator;)V (768 bytes) @ 0x00007f7cb1a73ac4 [0x00007f7cb1a73a60+0x64]
J 24596 C1 org.apache.spark.shuffle.GlutenColumnarShuffleWriter.write(Lscala/collection/Iterator;)V (6 bytes) @ 0x00007f7cb14050cc [0x00007f7cb1405040+0x8c]
J 24582 C1 org.apache.spark.shuffle.ShuffleWriteProcessor.write(Lorg/apache/spark/rdd/RDD;Lorg/apache/spark/ShuffleDependency;JLorg/apache/spark/TaskContext;Lorg/apache/spark/Partition;)Lorg/apache/spark/scheduler/MapStatus; (245 bytes) @ 0x00007f7cb140232c [0x00007f7cb1401da0+0x58c]
J 23452 C1 org.apache.spark.scheduler.ShuffleMapTask.runTask(Lorg/apache/spark/TaskContext;)Lorg/apache/spark/scheduler/MapStatus; (193 bytes) @ 0x00007f7cb105c61c [0x00007f7cb1059be0+0x2a3c]
J 24576 C1 org.apache.spark.scheduler.ShuffleMapTask.runTask(Lorg/apache/spark/TaskContext;)Ljava/lang/Object; (6 bytes) @ 0x00007f7cb13ec48c [0x00007f7cb13ec400+0x8c]
J 24464 C1 org.apache.spark.scheduler.Task.run(JILorg/apache/spark/metrics/MetricsSystem;Lscala/collection/immutable/Map;Lscala/Option;)Ljava/lang/Object; (495 bytes) @ 0x00007f7cb13946c4 [0x00007f7cb1391980+0x2d44]
J 23282 C1 org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Lorg/apache/spark/executor/Executor$TaskRunner;Lscala/runtime/BooleanRef;)Ljava/lang/Object; (47 bytes) @ 0x00007f7cb0f95f44 [0x00007f7cb0f95ca0+0x2a4]
J 23293 C1 org.apache.spark.executor.Executor$TaskRunner$$Lambda$2220.apply()Ljava/lang/Object; (12 bytes) @ 0x00007f7cb0fa68cc [0x00007f7cb0fa6840+0x8c]
J 29654 C2 org.apache.spark.util.Utils$.tryWithSafeFinally(Lscala/Function0;Lscala/Function0;)Ljava/lang/Object; (207 bytes) @ 0x00007f7cb23e1148 [0x00007f7cb23e1100+0x48]
J 22217 C1 org.apache.spark.executor.Executor$TaskRunner.run()V (2942 bytes) @ 0x00007f7cb0c89074 [0x00007f7cb0c84960+0x4714]
J 16583 C1 java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V (225 bytes) @ 0x00007f7cafc10a6c [0x00007f7cafc0f980+0x10ec]
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub
V  [libjvm.so+0x6a00e5]
V  [libjvm.so+0x69d82f]
V  [libjvm.so+0x69de24]
V  [libjvm.so+0x743dc0]
V  [libjvm.so+0xad989f]
V  [libjvm.so+0xad9b93]
V  [libjvm.so+0x97b0b0]
C  [libpthread.so.0+0x8609]  start_thread+0xd9

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 19200  io.glutenproject.vectorized.ArrowOutIterator.nativeHasNext(J)Z (0 bytes) @ 0x00007f7cb01ff908 [0x00007f7cb01ff8c0+0x48]
J 29521 C2 io.glutenproject.backendsapi.glutendata.GlutenIteratorApi$$anon$3.hasNext()Z (37 bytes) @ 0x00007f7cb2374b04 [0x00007f7cb2374ac0+0x44]
J 23613 C2 io.glutenproject.vectorized.CloseableColumnBatchIterator.hasNext()Z (50 bytes) @ 0x00007f7cb04be060 [0x00007f7cb04be000+0x60]
J 27165 C2 org.apache.spark.shuffle.GlutenColumnarShuffleWriter.internalWrite(Lscala/collection/Iterator;)V (768 bytes) @ 0x00007f7cb1a73ac4 [0x00007f7cb1a73a60+0x64]
J 24596 C1 org.apache.spark.shuffle.GlutenColumnarShuffleWriter.write(Lscala/collection/Iterator;)V (6 bytes) @ 0x00007f7cb14050cc [0x00007f7cb1405040+0x8c]
J 24582 C1 org.apache.spark.shuffle.ShuffleWriteProcessor.write(Lorg/apache/spark/rdd/RDD;Lorg/apache/spark/ShuffleDependency;JLorg/apache/spark/TaskContext;Lorg/apache/spark/Partition;)Lorg/apache/spark/scheduler/MapStatus; (245 bytes) @ 0x00007f7cb140232c [0x00007f7cb1401da0+0x58c]
J 23452 C1 org.apache.spark.scheduler.ShuffleMapTask.runTask(Lorg/apache/spark/TaskContext;)Lorg/apache/spark/scheduler/MapStatus; (193 bytes) @ 0x00007f7cb105c61c [0x00007f7cb1059be0+0x2a3c]
J 24576 C1 org.apache.spark.scheduler.ShuffleMapTask.runTask(Lorg/apache/spark/TaskContext;)Ljava/lang/Object; (6 bytes) @ 0x00007f7cb13ec48c [0x00007f7cb13ec400+0x8c]
J 24464 C1 org.apache.spark.scheduler.Task.run(JILorg/apache/spark/metrics/MetricsSystem;Lscala/collection/immutable/Map;Lscala/Option;)Ljava/lang/Object; (495 bytes) @ 0x00007f7cb13946c4 [0x00007f7cb1391980+0x2d44]
J 23282 C1 org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Lorg/apache/spark/executor/Executor$TaskRunner;Lscala/runtime/BooleanRef;)Ljava/lang/Object; (47 bytes) @ 0x00007f7cb0f95f44 [0x00007f7cb0f95ca0+0x2a4]
J 23293 C1 org.apache.spark.executor.Executor$TaskRunner$$Lambda$2220.apply()Ljava/lang/Object; (12 bytes) @ 0x00007f7cb0fa68cc [0x00007f7cb0fa6840+0x8c]
J 29654 C2 org.apache.spark.util.Utils$.tryWithSafeFinally(Lscala/Function0;Lscala/Function0;)Ljava/lang/Object; (207 bytes) @ 0x00007f7cb23e1148 [0x00007f7cb23e1100+0x48]
J 22217 C1 org.apache.spark.executor.Executor$TaskRunner.run()V (2942 bytes) @ 0x00007f7cb0c89074 [0x00007f7cb0c84960+0x4714]
J 16583 C1 java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V (225 bytes) @ 0x00007f7cafc10a6c [0x00007f7cafc0f980+0x10ec]
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub

Plan:

-- Project[expressions: (n10_1:DECIMAL(22,2), "n9_0"["col_0"]), (n10_2:BOOLEAN, "n9_0"["col_1"])] -> n10_1:DECIMAL(22,2), n10_2:BOOLEAN
  -- Aggregation[PARTIAL n9_0 := decimal_sum(ROW["n8_5"])] -> n9_0:ROW<col_0:DECIMAL(22,2),col_1:BOOLEAN>
    -- Project[expressions: (n8_5:DECIMAL(12,2), ROW["n7_6"])] -> n8_5:DECIMAL(12,2)
      -- Project[expressions: (n7_5:DECIMAL(12,2), ROW["n0_0"]), (n7_6:DECIMAL(12,2), ROW["n0_1"]), (n7_7:BIGINT, ROW["n0_2"]), (n7_8:DECIMAL(18,7), ROW["n4_2"]), (n7_9:BIGINT, ROW["n4_3"])] -> n7_5:DECIMAL(12,2), n7_6:DECIMAL(12,2), n7_7:BIGINT, n7_8:DECIMAL(18,7), n7_9:BIGINT
        -- HashJoin[INNER n0_2=n4_3, filter: lessthan(try_cast ROW["n0_0"] as DECIMAL(18,7),ROW["n4_2"])] -> n0_0:DECIMAL(12,2), n0_1:DECIMAL(12,2), n0_2:BIGINT, n4_2:DECIMAL(18,7), n4_3:BIGINT
          -- ValueStream[] -> n0_0:DECIMAL(12,2), n0_1:DECIMAL(12,2), n0_2:BIGINT
          -- Filter[expression: isnotnull(ROW["n4_2"])] -> n4_2:DECIMAL(18,7), n4_3:BIGINT
            -- Project[expressions: (n4_2:DECIMAL(18,7), check_overflow(decimal_multiply(0.200000,ROW["n3_1"]),true,0)), (n4_3:BIGINT, ROW["n2_3"])] -> n4_2:DECIMAL(18,7), n4_3:BIGINT
              -- Aggregation[FINAL [n2_3] n3_1 := decimal_avg(ROW["n2_4"])] -> n2_3:BIGINT, n3_1:DECIMAL(16,6)
                -- Project[expressions: (n2_3:BIGINT, ROW["n1_0"]), (n2_4:ROW<col_0:DECIMAL(22,2),col_1:BIGINT>, row_constructor_with_null(ROW["n1_1"],ROW["n1_2"]))] -> n2_3:BIGINT, n2_4:ROW<col_0:DECIMAL(22,2),col_1:BIGINT>
                  -- ValueStream[] -> n1_0:BIGINT, n1_1:DECIMAL(22,2), n1_2:BIGINT

n2_4:ROW<col_0:DECIMAL(22,2)
n3_1 := decimal_avg(ROW["n2_4"])] -> ..., n3_1:DECIMAL(16,6)

@zhztheplayer
Copy link
Collaborator Author

zhztheplayer commented May 4, 2023 •
edited
Loading

Will open a PR to meta upstream soon

(After the base decimal operations are upstreamed)

@zhztheplayer zhztheplayer mentioned this pull request May 4, 2023
Merged
@zhztheplayer
Copy link
Collaborator Author

@liujiayi771

@zhztheplayer zhztheplayer marked this pull request as ready for review May 5, 2023 05:25
@zhztheplayer zhztheplayer force-pushed the wip-fix-avgdecimal-spill branch from 5dcf247 to 798aec9 Compare May 5, 2023 05:26
@zhztheplayer zhztheplayer merged commit 7a16526 into oap-project:main May 5, 2023
@liujiayi771
Copy link

I will try to submit Spark decimal agg func to meta upstream.

zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request May 31, 2023
@zhejiangxiaomai
Folder: exec
a179d4e 
relative pr:

Fix hashjoin runtime issue oap-project#106
INVALID_STATE on HashJoin when spill is turned on oap-project#154
SIGABRT on DecimalAvgAggregate<UnscaleLongDecimal, UnscaleShortDecimal> when spilling is engaged oap-project#236
Support kPreceeding & kFollowing for window range frame type oap-project#287
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request May 31, 2023
@zhejiangxiaomai
Folder: exec
65aee45 
relative pr:

Fix hashjoin runtime issue oap-project#106
INVALID_STATE on HashJoin when spill is turned on oap-project#154
SIGABRT on DecimalAvgAggregate<UnscaleLongDecimal, UnscaleShortDecimal> when spilling is engaged oap-project#236
Support kPreceeding & kFollowing for window range frame type oap-project#287
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 3, 2023
@zhejiangxiaomai
Folder: exec
de7528f 
relative pr:

Fix hashjoin runtime issue oap-project#106
INVALID_STATE on HashJoin when spill is turned on oap-project#154
SIGABRT on DecimalAvgAggregate<UnscaleLongDecimal, UnscaleShortDecimal> when spilling is engaged oap-project#236
Support kPreceeding & kFollowing for window range frame type oap-project#287
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 4, 2023
@zhejiangxiaomai
Folder: exec
a72a76b 
relative pr:

Fix hashjoin runtime issue oap-project#106
INVALID_STATE on HashJoin when spill is turned on oap-project#154
SIGABRT on DecimalAvgAggregate<UnscaleLongDecimal, UnscaleShortDecimal> when spilling is engaged oap-project#236
Support kPreceeding & kFollowing for window range frame type oap-project#287
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 11, 2023
@zhejiangxiaomai
Folder: exec
4f3ed22 
relative pr:

Fix hashjoin runtime issue oap-project#106
INVALID_STATE on HashJoin when spill is turned on oap-project#154
SIGABRT on DecimalAvgAggregate<UnscaleLongDecimal, UnscaleShortDecimal> when spilling is engaged oap-project#236
Support kPreceeding & kFollowing for window range frame type oap-project#287
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 12, 2023
@zhejiangxiaomai
Folder: exec
5ef6058 
relative pr:

Fix hashjoin runtime issue oap-project#106
INVALID_STATE on HashJoin when spill is turned on oap-project#154
SIGABRT on DecimalAvgAggregate<UnscaleLongDecimal, UnscaleShortDecimal> when spilling is engaged oap-project#236
Support kPreceeding & kFollowing for window range frame type oap-project#287
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 12, 2023
@zhejiangxiaomai
Folder: exec
01ce968 
relative pr:

Fix hashjoin runtime issue oap-project#106
INVALID_STATE on HashJoin when spill is turned on oap-project#154
SIGABRT on DecimalAvgAggregate<UnscaleLongDecimal, UnscaleShortDecimal> when spilling is engaged oap-project#236
Support kPreceeding & kFollowing for window range frame type oap-project#287
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 17, 2023
@zhejiangxiaomai
Folder: exec
1c61c3a 
relative pr:

Fix hashjoin runtime issue oap-project#106
INVALID_STATE on HashJoin when spill is turned on oap-project#154
SIGABRT on DecimalAvgAggregate<UnscaleLongDecimal, UnscaleShortDecimal> when spilling is engaged oap-project#236
Support kPreceeding & kFollowing for window range frame type oap-project#287
marin-ma pushed a commit to marin-ma/velox-oap that referenced this pull request Dec 15, 2023
@zhztheplayer
[OPPRO-184] Adopt single thread execution for Velox task (oap-project…
38e1b7a 
…#236)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@zhztheplayer @liujiayi771