Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Upgrade velox to 2023.3.29 #1231

Merged
merged 1 commit into from
Apr 1, 2023

Conversation

zhejiangxiaomai
Copy link
Contributor

@zhejiangxiaomai zhejiangxiaomai commented Mar 29, 2023

What changes were proposed in this pull request?

  1. velox change memory pool API. we need call addChild to create memory pool to allocate.
  2. planFragment constructor change a little bit.
  3. setMemoryUsageTracker was removed by velox.
    corresponding velox branch is https://github.com/zhejiangxiaomai/velox/tree/main_rebase_329
  4. add " cmake --install xsimd-build/ cmake --install gtest-build/" in build_velox.sh. because velox move xsimd/gtest into resolve_dependency module.

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

@github-actions
Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[Gluten-${ISSUES_ID}] ${detailed message}

See also:

@zhejiangxiaomai zhejiangxiaomai changed the title [VL] Upgrade velox to 3.27 [VL] Upgrade velox to 2023.3.27 Mar 29, 2023
@zhejiangxiaomai
Copy link
Contributor Author

zhejiangxiaomai commented Mar 29, 2023

get a core when run tpch.

spark.sql(" select s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address, s_phone, s_comment from part, supplier, partsupp, nation, region where p_partkey = ps_partkey and s_suppkey = ps_suppkey and p_size = 15 and p_type like '%BRASS' and s_nationkey = n_nationkey and n_regionkey = r_regionkey and r_name = 'EUROPE' and ps_supplycost = ( select min(ps_supplycost) from partsupp, supplier, nation, region where p_partkey = ps_partkey and s_suppkey = ps_suppkey and s_nationkey = n_nationkey and n_regionkey = r_regionkey and r_name = 'EUROPE') order by s_acctbal desc, n_name, s_name, p_partkey limit 1 ").show(1, false)

(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x00007f7750fde859 in __GI_abort () at abort.c:79
#2 0x00007f775036af2f in ?? () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#3 0x00007f7750caa683 in ?? () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#4 0x00007f7750afaf4e in JVM_handle_linux_signal () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#5 0x00007f7750aee0ac in ?? () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#6
#7 0x0000000000000000 in ?? ()
#8 0x00007f74782faf81 in facebook::velox::AlignedBuffer::freeToPool (this=0x7f7350025a30) at /root/gluten/cpp/../ep/build-velox/build/velox_ep/velox/buffer/Buffer.h:517
#9 0x00007f747824a48a in facebook::velox::Buffer::release (this=0x7f7350025a30) at /root/gluten/cpp/../ep/build-velox/build/velox_ep/velox/buffer/Buffer.h:75
#10 0x00007f747824a4e0 in facebook::velox::intrusive_ptr_release (buffer=0x7f7350025a30) at /root/gluten/cpp/../ep/build-velox/build/velox_ep/velox/buffer/Buffer.h:278
#11 0x00007f7478250fa7 in boost::intrusive_ptrfacebook::velox::Buffer::~intrusive_ptr (this=0x7f73500259f8, __in_chrg=) at /usr/include/boost/smart_ptr/intrusive_ptr.hpp:98
#12 0x00007f747c883923 in facebook::velox::(anonymous namespace)::VeloxToArrowBridgeHolder::~VeloxToArrowBridgeHolder (this=0x7f73500259d0, __in_chrg=) at ../../velox/vector/arrow/Bridge.cpp:37
#13 0x00007f747c883b54 in facebook::velox::(anonymous namespace)::releaseArrowArray (arrowArray=0x7f7350025970) at ../../velox/vector/arrow/Bridge.cpp:170
#14 0x00007f747c883ab2 in facebook::velox::(anonymous namespace)::releaseArrowArray (arrowArray=0x7f735004e440) at ../../velox/vector/arrow/Bridge.cpp:155
#15 0x00007f769c015d76 in Java_org_apache_arrow_c_jni_JniWrapper_releaseArray () from /tmp/jnilib-8155710934442135310.tmp
#16 0x00007f7739018427 in ?? ()
#17 0x00007f76764f09a0 in ?? ()
#18 0x00007f7675df0a08 in ?? ()
#19 0x0000000000000000 in ?? ()
@zhztheplayer could you please help me?

Copy link
Member

@zhztheplayer zhztheplayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhejiangxiaomai zhejiangxiaomai marked this pull request as ready for review March 29, 2023 13:56
@zhejiangxiaomai zhejiangxiaomai changed the title [VL] Upgrade velox to 2023.3.27 [VL] Upgrade velox to 2023.3.29 Mar 29, 2023
@kyligence-git
Copy link
Contributor

Can one of the admins verify this patch?

@zhejiangxiaomai zhejiangxiaomai force-pushed the main_rebase_327 branch 4 times, most recently from cb8c842 to 5654a54 Compare March 31, 2023 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants