Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update absl #21300

Merged
merged 10 commits into from
Jul 10, 2024
Merged

Update absl #21300

merged 10 commits into from
Jul 10, 2024

Conversation

snnn
Copy link
Member

@snnn snnn commented Jul 9, 2024

Description

Motivation and Context

Our macOS pipeline are failing because of a build error in absl. However, the bug fix we need is not available in the latest ABSL release.

Here is the issue: abseil/abseil-cpp#1536
And here is the fix: abseil/abseil-cpp@779a356

GTests uses ABSL. But this ABSL target also depends on GTest. So, it is a circular dependency. We should be able to avoid that by avoid building tests for ABSL. However, the version we are using has a problem with that: it has cmake target that still depends on GTest even when testing is disabled.

I don't know why we suddenly hit this problem.

@yuslepukhin
Copy link
Member

Do we know if the internals of the classes are changing? Question is related to natvis.

@snnn
Copy link
Member Author

snnn commented Jul 9, 2024

Most likely they are changing.

@snnn
Copy link
Member Author

snnn commented Jul 10, 2024

GCC 12 seems to have a bug with the latest ABSL code.

$ /opt/rh/gcc-toolset-12/root/usr/bin/c++ -DCPUINFO_SUPPORTED_PLATFORM=1 -DEIGEN_MPL2_ONLY -DEIGEN_USE_THREADS -DNSYNC_ATOMIC_CPP11 -DONLY_C_LOCALE=0 -DONNX_ML=1 -DONNX_NAMESPACE=onnx -DONNX_USE_LITE_PROTO=1 -DORT_ENABLE_STREAM -DORT_RUN_EXTERNAL_ONNX_TESTS -DPLATFORM_POSIX -D_GNU_SOURCE -D__ONNX_NO_DOC_STRINGS -I/tmp/build5/Release/_deps/utf8_range-src -I/onnxruntime_src/include/onnxruntime -I/onnxruntime_src/include/onnxruntime/core/session -I/tmp/build5/Release/_deps/pytorch_cpuinfo-src/include -I/tmp/build5/Release/_deps/google_nsync-src/public -I/tmp/build5/Release -I/onnxruntime_src/onnxruntime -I/tmp/build5/Release/_deps/abseil_cpp-src -I/tmp/build5/Release/_deps/safeint-src -I/tmp/build5/Release/_deps/gsl-src/include -I/tmp/build5/Release/_deps/date-src/include -I/tmp/build5/Release/_deps/onnx-src -I/tmp/build5/Release/_deps/onnx-build -I/tmp/build5/Release/_deps/protobuf-src/src -I/tmp/build5/Release/_deps/flatbuffers-src/include -I/tmp/build5/Release/_deps/mp11-src/include -I/tmp/build5/Release/_deps/eigen-src -I/tmp/build5/Release/_deps/re2-src -DNDEBUG -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -O3 -pipe -fstack-clash-protection -fcf-protection -ffunction-sections -fdata-sections -Wno-restrict  -DCPUINFO_SUPPORTED -O3 -DNDEBUG -fPIC -Wall -Wextra -Wno-deprecated-copy -Wno-nonnull-compare -Werror /onnxruntime_src/slice.cc  -c
In file included from /tmp/build5/Release/_deps/abseil_cpp-src/absl/container/flat_hash_set.h:45,
                 from /onnxruntime_src/include/onnxruntime/core/common/inlined_containers.h:21,
                 from /onnxruntime_src/include/onnxruntime/core/graph/graph.h:23,
                 from /onnxruntime_src/include/onnxruntime/core/graph/graph_viewer.h:8,
                 from /onnxruntime_src/include/onnxruntime/core/framework/op_node_proto_helper.h:9,
                 from /onnxruntime_src/include/onnxruntime/core/framework/op_kernel_info.h:9,
                 from /onnxruntime_src/include/onnxruntime/core/framework/op_kernel.h:20,
                 from /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/slice.h:7,
                 from /onnxruntime_src/slice.cc:4:
/tmp/build5/Release/_deps/abseil_cpp-src/absl/container/internal/raw_hash_set.h: In function ‘onnxruntime::common::Status onnxruntime::SliceOp::PrepareForComputeHelper(const gsl::span<const long int>&, const gsl::span<const long int>&, const gsl::span<const long int>&, const gsl::span<const long int>&, PrepareForComputeMetadata&)’:
/tmp/build5/Release/_deps/abseil_cpp-src/absl/container/internal/raw_hash_set.h:1292:54: error: ‘unique_axes.onnxruntime::InlinedHashSet<long int, std::allocator<long int> >::<unnamed>.absl::flat_hash_set<long int, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::<unnamed>.absl::container_internal::raw_hash_set<absl::container_internal::FlatHashSetPolicy<long int>, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::settings_.absl::container_internal::CompressedTuple<absl::container_internal::CommonFields, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::<unnamed>.absl::container_internal::internal_compressed_tuple::CompressedTupleImpl<absl::container_internal::CompressedTuple<absl::container_internal::CommonFields, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >, std::integer_sequence<long unsigned int, 0, 1, 2, 3>, true>::<unnamed>.absl::container_internal::internal_compressed_tuple::Storage<absl::container_internal::CommonFields, 0, false>::value.absl::container_internal::CommonFields::heap_or_soo_.absl::container_internal::HeapOrSoo::heap.absl::container_internal::HeapPtrs::control’ may be used uninitialized [-Werror=maybe-uninitialized]
 1292 |     ABSL_SWISSTABLE_IGNORE_UNINITIALIZED_RETURN(heap.control);
      |                                                      ^~~~~~~
In file included from /onnxruntime_src/slice.cc:13:
/onnxruntime_src/onnxruntime/core/providers/cpu/tensor/slice_helper.h:98:11: note: ‘unique_axes’ declared here
   98 |   AxesSet unique_axes;
      |           ^~~~~~~~~~~
/tmp/build5/Release/_deps/abseil_cpp-src/absl/container/internal/raw_hash_set.h: In function ‘onnxruntime::common::Status onnxruntime::SliceOp::PrepareForComputeHelper(const gsl::span<const long int>&, const gsl::span<const long int>&, const gsl::span<const long int>&, PrepareForComputeMetadata&)’:
/tmp/build5/Release/_deps/abseil_cpp-src/absl/container/internal/raw_hash_set.h:1292:54: error: ‘unique_axes.onnxruntime::InlinedHashSet<long int, std::allocator<long int> >::<unnamed>.absl::flat_hash_set<long int, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::<unnamed>.absl::container_internal::raw_hash_set<absl::container_internal::FlatHashSetPolicy<long int>, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::settings_.absl::container_internal::CompressedTuple<absl::container_internal::CommonFields, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >::<unnamed>.absl::container_internal::internal_compressed_tuple::CompressedTupleImpl<absl::container_internal::CompressedTuple<absl::container_internal::CommonFields, absl::hash_internal::Hash<long int>, std::equal_to<long int>, std::allocator<long int> >, std::integer_sequence<long unsigned int, 0, 1, 2, 3>, true>::<unnamed>.absl::container_internal::internal_compressed_tuple::Storage<absl::container_internal::CommonFields, 0, false>::value.absl::container_internal::CommonFields::heap_or_soo_.absl::container_internal::HeapOrSoo::heap.absl::container_internal::HeapPtrs::control’ may be used uninitialized [-Werror=maybe-uninitialized]
 1292 |     ABSL_SWISSTABLE_IGNORE_UNINITIALIZED_RETURN(heap.control);
      |                                                      ^~~~~~~
/onnxruntime_src/onnxruntime/core/providers/cpu/tensor/slice_helper.h:39:11: note: ‘unique_axes’ declared here
   39 |   AxesSet unique_axes;
      |           ^~~~~~~~~~~
cc1plus: all warnings being treated as errors

GCC 13 doesn't have this issue.

@snnn snnn marked this pull request as ready for review July 10, 2024 15:41
@snnn snnn requested a review from a team as a code owner July 10, 2024 15:41
@snnn snnn requested a review from a team as a code owner July 10, 2024 15:47
@snnn snnn merged commit 8749fa3 into main Jul 10, 2024
94 of 99 checks passed
@snnn snnn deleted the snnn/update_absl2 branch July 10, 2024 18:14
sumitsays pushed a commit that referenced this pull request Aug 8, 2024
Our macOS pipeline are failing because of a build error in absl.
However, the bug fix we need is not available in the latest ABSL
release.

Here  is the issue: abseil/abseil-cpp#1536
And here is the fix:
abseil/abseil-cpp@779a356

GTests uses ABSL. But this ABSL target also depends on GTest. So, it is
a circular dependency. We should be able to avoid that by avoid building
tests for ABSL. However, the version we are using has a problem with
that: it has cmake target that still depends on GTest even when testing
is disabled.

It's strange that we suddenly hit this problem and it only happens on macOS.
prathikr pushed a commit that referenced this pull request Aug 8, 2024
Our macOS pipeline are failing because of a build error in absl.
However, the bug fix we need is not available in the latest ABSL
release.

Here  is the issue: abseil/abseil-cpp#1536
And here is the fix:
abseil/abseil-cpp@779a356

GTests uses ABSL. But this ABSL target also depends on GTest. So, it is
a circular dependency. We should be able to avoid that by avoid building
tests for ABSL. However, the version we are using has a problem with
that: it has cmake target that still depends on GTest even when testing
is disabled.

It's strange that we suddenly hit this problem and it only happens on macOS.
sumitsays added a commit that referenced this pull request Aug 12, 2024
…1670)

### Description
This change cherry-picks 2 Pad fusion optimization:
#21640 and
#21556.

It also has to cherry-pick 2 extra changes to unblock pipeline and
dependency failure: #21300
and #21662 (didn't include
test which are part of 1.18.1 payload).

Also uploaded new version of
[onnxruntime_build_dependencies:10.177](https://dev.azure.com/onnxruntime/onnxruntime/_artifacts/feed/onnxruntime/UPack/onnxruntime_build_dependencies/overview/1.0.177)
and updated the same in `download-deps.yml`.

Additionally it also updates DML binary to 1.15.1.



### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

---------

Co-authored-by: Changming Sun <[email protected]>
Co-authored-by: Tianlei Wu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants