[ModuleInliner] Fix the heap maintenance

[kazutakahirata]

With expensive checks enabled but without this patch, std::pop_heap
triggers an assertion failure. This is because:

updateAndCheckDecreased(Heap.front())

updates the priority associated with Heap.front(), so Heap may no
longer be a valid heap. The libstdc++ version of std::pop_heap
requires that the entire range be a valid heap even though the core
task of std::pop_heap is just to swap the Heap.front() and
Heap.back().

This patch fixes the problem by:

• calling std::pop_heap to swap Heap.front() and Heap.back(),
• updating the priority of Heap.back(), and
• inserting Heap.back() back into the heap.

We could reduce the number of calls to updateAndCheckDecreased or the
number of elements being moved, but I think it's important to fix the
crash first.

Credit to Ivan Kosarev for identifying the problem and Liqiang Tao for
the analysis and initial version of the patch.

[llvmbot]

@llvm/pr-subscribers-llvm-analysis

Author: Kazu Hirata (kazutakahirata)

Changes

With expensive checks enabled but without this patch, std::pop_heap
triggers an assertion failure. This is because:

updateAndCheckDecreased(Heap.front())

updates the priority associated with Heap.front(), so Heap may no
longer be a valid heap. The libstdc++ version of std::pop_heap
requires that the entire range be a valid heap even though the core
task of std::pop_heap is just to swap the Heap.front() and
Heap.back().

This patch fixes the problem by:

• calling std::pop_heap to swap Heap.front() and Heap.back(),
• updating the priority of Heap.back(), and
• inserting Heap.back() back into the heap.

We could reduce the number of calls to updateAndCheckDecreased or the
number of elements being moved, but I think it's important to fix the
crash first.

Credit to Ivan Kosarev for identifying the problem and Liqiang Tao for
the analysis and initial version of the patch.

---

Full diff: https://github.com/llvm/llvm-project/pull/69251.diff

1 Files Affected:

• (modified) llvm/lib/Analysis/InlineOrder.cpp (+4-2)

diff --git a/llvm/lib/Analysis/InlineOrder.cpp b/llvm/lib/Analysis/InlineOrder.cpp
index 503880e3e8f0e93..b086ac15a207ed3 100644
--- a/llvm/lib/Analysis/InlineOrder.cpp
+++ b/llvm/lib/Analysis/InlineOrder.cpp
@@ -223,10 +223,12 @@ class PriorityInlineOrder : public InlineOrder<std::pair<CallBase *, int>> {
   // pushed right back into the heap. For simplicity, those cases where
   // the desirability of a call site increases are ignored here.
   void adjust() {
-    while (updateAndCheckDecreased(Heap.front())) {
-      std::pop_heap(Heap.begin(), Heap.end(), isLess);
+    std::pop_heap(Heap.begin(), Heap.end(), isLess);
+    while (updateAndCheckDecreased(Heap.back())) {
       std::push_heap(Heap.begin(), Heap.end(), isLess);
+      std::pop_heap(Heap.begin(), Heap.end(), isLess);
     }
+    std::push_heap(Heap.begin(), Heap.end(), isLess);
   }
 
 public:

[taoliq]

LGTM. Thanks for the fix.

[kosarev]

LGTM, thanks!