-
Notifications
You must be signed in to change notification settings - Fork 12.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[TTI][RISCV] Deduplicate type-based VP costing #115983
[TTI][RISCV] Deduplicate type-based VP costing #115983
Conversation
@llvm/pr-subscribers-llvm-transforms @llvm/pr-subscribers-backend-risc-v Author: Luke Lau (lukel97) ChangesWe have a lot of code in RISCVTTIImpl::getIntrinsicInstrCost for vp intrinsics, which just forward the cost to the underlying non-vp cost function. However I just also noticed that there is generic code in BasicTTIImpl's getIntrinsicInstrCost that does the same thing, added in #67178. The only difference is that BasicTTIImpl doesn't yet handle it for type-based costing. There doesn't seem to be any reason that it can't since it's just inspecting the argument types. This shuffles the VP costing up to handle both regular and type-based costing, and begins to deduplicate the VP specific costing in RISCVTTIImpl by moving them into BasicTTIImpl.h. It's not NFC since it picks up a couple of VP nodes that had slipped through the cracks. Future PRs can begin to move more of the code from RISCVTTIImpl to BasicTTIImpl. Patch is 34.21 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/115983.diff 3 Files Affected:
diff --git a/llvm/include/llvm/CodeGen/BasicTTIImpl.h b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
index b0316e67654dbc..8343498cea31ec 100644
--- a/llvm/include/llvm/CodeGen/BasicTTIImpl.h
+++ b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
@@ -1563,6 +1563,67 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
if (Intrinsic::isTargetIntrinsic(IID))
return TargetTransformInfo::TCC_Basic;
+ // VP Intrinsics should have the same cost as their non-vp counterpart.
+ // TODO: Adjust the cost to make the vp intrinsic cheaper than its non-vp
+ // counterpart when the vector length argument is smaller than the maximum
+ // vector length.
+ // TODO: Support other kinds of VPIntrinsics
+ if (VPIntrinsic::isVPIntrinsic(ICA.getID())) {
+ std::optional<unsigned> FOp =
+ VPIntrinsic::getFunctionalOpcodeForVP(ICA.getID());
+ if (FOp) {
+ if (ICA.getID() == Intrinsic::vp_load) {
+ Align Alignment;
+ if (auto *VPI = dyn_cast_or_null<VPIntrinsic>(ICA.getInst()))
+ Alignment = VPI->getPointerAlignment().valueOrOne();
+ unsigned AS = 0;
+ if (ICA.getArgTypes().size() > 1)
+ if (auto *PtrTy = dyn_cast<PointerType>(ICA.getArgTypes()[0]))
+ AS = PtrTy->getAddressSpace();
+ return thisT()->getMemoryOpCost(*FOp, ICA.getReturnType(), Alignment,
+ AS, CostKind);
+ }
+ if (ICA.getID() == Intrinsic::vp_store) {
+ Align Alignment;
+ if (auto *VPI = dyn_cast_or_null<VPIntrinsic>(ICA.getInst()))
+ Alignment = VPI->getPointerAlignment().valueOrOne();
+ unsigned AS = 0;
+ if (ICA.getArgTypes().size() >= 2)
+ if (auto *PtrTy = dyn_cast<PointerType>(ICA.getArgTypes()[1]))
+ AS = PtrTy->getAddressSpace();
+ return thisT()->getMemoryOpCost(*FOp, ICA.getArgTypes()[0], Alignment,
+ AS, CostKind);
+ }
+ if (VPBinOpIntrinsic::isVPBinOp(ICA.getID())) {
+ return thisT()->getArithmeticInstrCost(*FOp, ICA.getReturnType(),
+ CostKind);
+ }
+ }
+
+ std::optional<Intrinsic::ID> FID =
+ VPIntrinsic::getFunctionalIntrinsicIDForVP(ICA.getID());
+ if (FID) {
+ // Non-vp version will have same arg types except mask and vector
+ // length.
+ assert(ICA.getArgTypes().size() >= 2 &&
+ "Expected VPIntrinsic to have Mask and Vector Length args and "
+ "types");
+ ArrayRef<Type *> NewTys = ArrayRef(ICA.getArgTypes()).drop_back(2);
+
+ // VPReduction intrinsics have a start value argument that their non-vp
+ // counterparts do not have, except for the fadd and fmul non-vp
+ // counterpart.
+ if (VPReductionIntrinsic::isVPReduction(ICA.getID()) &&
+ *FID != Intrinsic::vector_reduce_fadd &&
+ *FID != Intrinsic::vector_reduce_fmul)
+ NewTys = NewTys.drop_front();
+
+ IntrinsicCostAttributes NewICA(*FID, ICA.getReturnType(), NewTys,
+ ICA.getFlags());
+ return thisT()->getIntrinsicInstrCost(NewICA, CostKind);
+ }
+ }
+
if (ICA.isTypeBasedOnly())
return getTypeBasedIntrinsicInstrCost(ICA, CostKind);
@@ -1823,68 +1884,6 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
}
}
- // VP Intrinsics should have the same cost as their non-vp counterpart.
- // TODO: Adjust the cost to make the vp intrinsic cheaper than its non-vp
- // counterpart when the vector length argument is smaller than the maximum
- // vector length.
- // TODO: Support other kinds of VPIntrinsics
- if (VPIntrinsic::isVPIntrinsic(ICA.getID())) {
- std::optional<unsigned> FOp =
- VPIntrinsic::getFunctionalOpcodeForVP(ICA.getID());
- if (FOp) {
- if (ICA.getID() == Intrinsic::vp_load) {
- Align Alignment;
- if (auto *VPI = dyn_cast_or_null<VPIntrinsic>(ICA.getInst()))
- Alignment = VPI->getPointerAlignment().valueOrOne();
- unsigned AS = 0;
- if (ICA.getArgs().size() > 1)
- if (auto *PtrTy =
- dyn_cast<PointerType>(ICA.getArgs()[0]->getType()))
- AS = PtrTy->getAddressSpace();
- return thisT()->getMemoryOpCost(*FOp, ICA.getReturnType(), Alignment,
- AS, CostKind);
- }
- if (ICA.getID() == Intrinsic::vp_store) {
- Align Alignment;
- if (auto *VPI = dyn_cast_or_null<VPIntrinsic>(ICA.getInst()))
- Alignment = VPI->getPointerAlignment().valueOrOne();
- unsigned AS = 0;
- if (ICA.getArgs().size() >= 2)
- if (auto *PtrTy =
- dyn_cast<PointerType>(ICA.getArgs()[1]->getType()))
- AS = PtrTy->getAddressSpace();
- return thisT()->getMemoryOpCost(*FOp, Args[0]->getType(), Alignment,
- AS, CostKind);
- }
- if (VPBinOpIntrinsic::isVPBinOp(ICA.getID())) {
- return thisT()->getArithmeticInstrCost(*FOp, ICA.getReturnType(),
- CostKind);
- }
- }
-
- std::optional<Intrinsic::ID> FID =
- VPIntrinsic::getFunctionalIntrinsicIDForVP(ICA.getID());
- if (FID) {
- // Non-vp version will have same Args/Tys except mask and vector length.
- assert(ICA.getArgs().size() >= 2 && ICA.getArgTypes().size() >= 2 &&
- "Expected VPIntrinsic to have Mask and Vector Length args and "
- "types");
- ArrayRef<Type *> NewTys = ArrayRef(ICA.getArgTypes()).drop_back(2);
-
- // VPReduction intrinsics have a start value argument that their non-vp
- // counterparts do not have, except for the fadd and fmul non-vp
- // counterpart.
- if (VPReductionIntrinsic::isVPReduction(ICA.getID()) &&
- *FID != Intrinsic::vector_reduce_fadd &&
- *FID != Intrinsic::vector_reduce_fmul)
- NewTys = NewTys.drop_front();
-
- IntrinsicCostAttributes NewICA(*FID, ICA.getReturnType(), NewTys,
- ICA.getFlags());
- return thisT()->getIntrinsicInstrCost(NewICA, CostKind);
- }
- }
-
// Assume that we need to scalarize this intrinsic.)
// Compute the scalarization overhead based on Args for a vector
// intrinsic.
diff --git a/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp b/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
index 6344bc4664d3b6..d98be8a331417c 100644
--- a/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
+++ b/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
@@ -1104,32 +1104,6 @@ RISCVTTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
return Cost * LT.first;
break;
}
- // vp integer arithmetic ops.
- case Intrinsic::vp_add:
- case Intrinsic::vp_and:
- case Intrinsic::vp_ashr:
- case Intrinsic::vp_lshr:
- case Intrinsic::vp_mul:
- case Intrinsic::vp_or:
- case Intrinsic::vp_sdiv:
- case Intrinsic::vp_shl:
- case Intrinsic::vp_srem:
- case Intrinsic::vp_sub:
- case Intrinsic::vp_udiv:
- case Intrinsic::vp_urem:
- case Intrinsic::vp_xor:
- // vp float arithmetic ops.
- case Intrinsic::vp_fadd:
- case Intrinsic::vp_fsub:
- case Intrinsic::vp_fmul:
- case Intrinsic::vp_fdiv:
- case Intrinsic::vp_frem: {
- std::optional<unsigned> FOp =
- VPIntrinsic::getFunctionalOpcodeForVP(ICA.getID());
- assert(FOp.has_value());
- return getArithmeticInstrCost(*FOp, ICA.getReturnType(), CostKind);
- break;
- }
// vp int cast ops.
case Intrinsic::vp_trunc:
case Intrinsic::vp_zext:
@@ -1163,23 +1137,6 @@ RISCVTTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
return getCmpSelInstrCost(*FOp, ICA.getArgTypes()[0], ICA.getReturnType(),
UI->getPredicate(), CostKind);
}
- // vp load/store
- case Intrinsic::vp_load:
- case Intrinsic::vp_store: {
- if (!ICA.getInst())
- break;
- Intrinsic::ID IID = ICA.getID();
- std::optional<unsigned> FOp = VPIntrinsic::getFunctionalOpcodeForVP(IID);
- assert(FOp.has_value());
- auto *UI = cast<VPIntrinsic>(ICA.getInst());
- if (ICA.getID() == Intrinsic::vp_load)
- return getMemoryOpCost(
- *FOp, ICA.getReturnType(), UI->getPointerAlignment(),
- UI->getOperand(0)->getType()->getPointerAddressSpace(), CostKind);
- return getMemoryOpCost(
- *FOp, ICA.getArgTypes()[0], UI->getPointerAlignment(),
- UI->getOperand(1)->getType()->getPointerAddressSpace(), CostKind);
- }
case Intrinsic::vp_select: {
Intrinsic::ID IID = ICA.getID();
std::optional<unsigned> FOp = VPIntrinsic::getFunctionalOpcodeForVP(IID);
diff --git a/llvm/test/Analysis/CostModel/RISCV/rvv-intrinsics.ll b/llvm/test/Analysis/CostModel/RISCV/rvv-intrinsics.ll
index bb98508f239c1b..ed8b928a5108b7 100644
--- a/llvm/test/Analysis/CostModel/RISCV/rvv-intrinsics.ll
+++ b/llvm/test/Analysis/CostModel/RISCV/rvv-intrinsics.ll
@@ -746,37 +746,37 @@ define void @abs() {
; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
;
; TYPEBASED-LABEL: 'abs'
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 14 for instruction: %1 = call <2 x i8> @llvm.vp.abs.v2i8(<2 x i8> undef, i1 false, <2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %1 = call <2 x i8> @llvm.vp.abs.v2i8(<2 x i8> undef, i1 false, <2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %2 = call <2 x i8> @llvm.abs.v2i8(<2 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 30 for instruction: %3 = call <4 x i8> @llvm.vp.abs.v4i8(<4 x i8> undef, i1 false, <4 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %3 = call <4 x i8> @llvm.vp.abs.v4i8(<4 x i8> undef, i1 false, <4 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %4 = call <4 x i8> @llvm.abs.v4i8(<4 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 62 for instruction: %5 = call <8 x i8> @llvm.vp.abs.v8i8(<8 x i8> undef, i1 false, <8 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %5 = call <8 x i8> @llvm.vp.abs.v8i8(<8 x i8> undef, i1 false, <8 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %6 = call <8 x i8> @llvm.abs.v8i8(<8 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 126 for instruction: %7 = call <16 x i8> @llvm.vp.abs.v16i8(<16 x i8> undef, i1 false, <16 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %7 = call <16 x i8> @llvm.vp.abs.v16i8(<16 x i8> undef, i1 false, <16 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %8 = call <16 x i8> @llvm.abs.v16i8(<16 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 14 for instruction: %9 = call <2 x i64> @llvm.vp.abs.v2i64(<2 x i64> undef, i1 false, <2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %9 = call <2 x i64> @llvm.vp.abs.v2i64(<2 x i64> undef, i1 false, <2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %10 = call <2 x i64> @llvm.abs.v2i64(<2 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 33 for instruction: %11 = call <4 x i64> @llvm.vp.abs.v4i64(<4 x i64> undef, i1 false, <4 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %11 = call <4 x i64> @llvm.vp.abs.v4i64(<4 x i64> undef, i1 false, <4 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %12 = call <4 x i64> @llvm.abs.v4i64(<4 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 69 for instruction: %13 = call <8 x i64> @llvm.vp.abs.v8i64(<8 x i64> undef, i1 false, <8 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = call <8 x i64> @llvm.vp.abs.v8i64(<8 x i64> undef, i1 false, <8 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %14 = call <8 x i64> @llvm.abs.v8i64(<8 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 141 for instruction: %15 = call <16 x i64> @llvm.vp.abs.v16i64(<16 x i64> undef, i1 false, <16 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %15 = call <16 x i64> @llvm.vp.abs.v16i64(<16 x i64> undef, i1 false, <16 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %16 = call <16 x i64> @llvm.abs.v16i64(<16 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %17 = call <vscale x 2 x i8> @llvm.vp.abs.nxv2i8(<vscale x 2 x i8> undef, i1 false, <vscale x 2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %17 = call <vscale x 2 x i8> @llvm.vp.abs.nxv2i8(<vscale x 2 x i8> undef, i1 false, <vscale x 2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %18 = call <vscale x 2 x i8> @llvm.abs.nxv2i8(<vscale x 2 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %19 = call <vscale x 4 x i8> @llvm.vp.abs.nxv4i8(<vscale x 4 x i8> undef, i1 false, <vscale x 4 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %19 = call <vscale x 4 x i8> @llvm.vp.abs.nxv4i8(<vscale x 4 x i8> undef, i1 false, <vscale x 4 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %20 = call <vscale x 4 x i8> @llvm.abs.nxv4i8(<vscale x 4 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %21 = call <vscale x 8 x i8> @llvm.vp.abs.nxv8i8(<vscale x 8 x i8> undef, i1 false, <vscale x 8 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %21 = call <vscale x 8 x i8> @llvm.vp.abs.nxv8i8(<vscale x 8 x i8> undef, i1 false, <vscale x 8 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %22 = call <vscale x 8 x i8> @llvm.abs.nxv8i8(<vscale x 8 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %23 = call <vscale x 16 x i8> @llvm.vp.abs.nxv16i8(<vscale x 16 x i8> undef, i1 false, <vscale x 16 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %23 = call <vscale x 16 x i8> @llvm.vp.abs.nxv16i8(<vscale x 16 x i8> undef, i1 false, <vscale x 16 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %24 = call <vscale x 16 x i8> @llvm.abs.nxv16i8(<vscale x 16 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %25 = call <vscale x 2 x i64> @llvm.vp.abs.nxv2i64(<vscale x 2 x i64> undef, i1 false, <vscale x 2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %25 = call <vscale x 2 x i64> @llvm.vp.abs.nxv2i64(<vscale x 2 x i64> undef, i1 false, <vscale x 2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = call <vscale x 2 x i64> @llvm.abs.nxv2i64(<vscale x 2 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %27 = call <vscale x 4 x i64> @llvm.vp.abs.nxv4i64(<vscale x 4 x i64> undef, i1 false, <vscale x 4 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %27 = call <vscale x 4 x i64> @llvm.vp.abs.nxv4i64(<vscale x 4 x i64> undef, i1 false, <vscale x 4 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %28 = call <vscale x 4 x i64> @llvm.abs.nxv4i64(<vscale x 4 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %29 = call <vscale x 8 x i64> @llvm.vp.abs.nxv8i64(<vscale x 8 x i64> undef, i1 false, <vscale x 8 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %29 = call <vscale x 8 x i64> @llvm.vp.abs.nxv8i64(<vscale x 8 x i64> undef, i1 false, <vscale x 8 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %30 = call <vscale x 8 x i64> @llvm.abs.nxv8i64(<vscale x 8 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %31 = call <vscale x 16 x i64> @llvm.vp.abs.nxv16i64(<vscale x 16 x i64> undef, i1 false, <vscale x 16 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %31 = call <vscale x 16 x i64> @llvm.vp.abs.nxv16i64(<vscale x 16 x i64> undef, i1 false, <vscale x 16 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = call <vscale x 16 x i64> @llvm.abs.nxv16i64(<vscale x 16 x i64> undef, i1 false)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
;
@@ -1218,37 +1218,37 @@ define void @reduce_add() {
; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
;
; TYPEBASED-LABEL: 'reduce_add'
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %1 = call i8 @llvm.vp.reduce.add.v2i8(i8 undef, <2 x i8> undef, <2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %1 = call i8 @llvm.vp.reduce.add.v2i8(i8 undef, <2 x i8> undef, <2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %2 = call i8 @llvm.vector.reduce.add.v2i8(<2 x i8> undef)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 26 for instruction: %3 = call i8 @llvm.vp.reduce.add.v4i8(i8 undef, <4 x i8> undef, <4 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %3 = call i8 @llvm.vp.reduce.add.v4i8(i8 undef, <4 x i8> undef, <4 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %4 = call i8 @llvm.vector.reduce.add.v4i8(<4 x i8> undef)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 54 for instruction: %5 = call i8 @llvm.vp.reduce.add.v8i8(i8 undef, <8 x i8> undef, <8 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %5 = call i8 @llvm.vp.reduce.add.v8i8(i8 undef, <8 x i8> undef, <8 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %6 = call i8 @llvm.vector.reduce.add.v8i8(<8 x i8> undef)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 110 for instruction: %7 = call i8 @llvm.vp.reduce.add.v16i8(i8 undef, <16 x i8> undef, <16 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %7 = call i8 @llvm.vp.reduce.add.v16i8(i8 undef, <16 x i8> undef, <16 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %8 = call i8 @llvm.vector.reduce.add.v16i8(<16 x i8> undef)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %9 = call i64 @llvm.vp.reduce.add.v2i64(i64 undef, <2 x i64> undef, <2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %9 = call i64 @llvm.vp.reduce.add.v2i64(i64 undef, <2 x i64> undef, <2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %10 = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> undef)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 26 for instruction: %11 = call i64 @llvm.vp.reduce.add.v4i64(i64 undef, <4 x i64> undef, <4 x i1> und...
[truncated]
|
@llvm/pr-subscribers-llvm-analysis Author: Luke Lau (lukel97) ChangesWe have a lot of code in RISCVTTIImpl::getIntrinsicInstrCost for vp intrinsics, which just forward the cost to the underlying non-vp cost function. However I just also noticed that there is generic code in BasicTTIImpl's getIntrinsicInstrCost that does the same thing, added in #67178. The only difference is that BasicTTIImpl doesn't yet handle it for type-based costing. There doesn't seem to be any reason that it can't since it's just inspecting the argument types. This shuffles the VP costing up to handle both regular and type-based costing, and begins to deduplicate the VP specific costing in RISCVTTIImpl by moving them into BasicTTIImpl.h. It's not NFC since it picks up a couple of VP nodes that had slipped through the cracks. Future PRs can begin to move more of the code from RISCVTTIImpl to BasicTTIImpl. Patch is 34.21 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/115983.diff 3 Files Affected:
diff --git a/llvm/include/llvm/CodeGen/BasicTTIImpl.h b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
index b0316e67654dbc..8343498cea31ec 100644
--- a/llvm/include/llvm/CodeGen/BasicTTIImpl.h
+++ b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
@@ -1563,6 +1563,67 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
if (Intrinsic::isTargetIntrinsic(IID))
return TargetTransformInfo::TCC_Basic;
+ // VP Intrinsics should have the same cost as their non-vp counterpart.
+ // TODO: Adjust the cost to make the vp intrinsic cheaper than its non-vp
+ // counterpart when the vector length argument is smaller than the maximum
+ // vector length.
+ // TODO: Support other kinds of VPIntrinsics
+ if (VPIntrinsic::isVPIntrinsic(ICA.getID())) {
+ std::optional<unsigned> FOp =
+ VPIntrinsic::getFunctionalOpcodeForVP(ICA.getID());
+ if (FOp) {
+ if (ICA.getID() == Intrinsic::vp_load) {
+ Align Alignment;
+ if (auto *VPI = dyn_cast_or_null<VPIntrinsic>(ICA.getInst()))
+ Alignment = VPI->getPointerAlignment().valueOrOne();
+ unsigned AS = 0;
+ if (ICA.getArgTypes().size() > 1)
+ if (auto *PtrTy = dyn_cast<PointerType>(ICA.getArgTypes()[0]))
+ AS = PtrTy->getAddressSpace();
+ return thisT()->getMemoryOpCost(*FOp, ICA.getReturnType(), Alignment,
+ AS, CostKind);
+ }
+ if (ICA.getID() == Intrinsic::vp_store) {
+ Align Alignment;
+ if (auto *VPI = dyn_cast_or_null<VPIntrinsic>(ICA.getInst()))
+ Alignment = VPI->getPointerAlignment().valueOrOne();
+ unsigned AS = 0;
+ if (ICA.getArgTypes().size() >= 2)
+ if (auto *PtrTy = dyn_cast<PointerType>(ICA.getArgTypes()[1]))
+ AS = PtrTy->getAddressSpace();
+ return thisT()->getMemoryOpCost(*FOp, ICA.getArgTypes()[0], Alignment,
+ AS, CostKind);
+ }
+ if (VPBinOpIntrinsic::isVPBinOp(ICA.getID())) {
+ return thisT()->getArithmeticInstrCost(*FOp, ICA.getReturnType(),
+ CostKind);
+ }
+ }
+
+ std::optional<Intrinsic::ID> FID =
+ VPIntrinsic::getFunctionalIntrinsicIDForVP(ICA.getID());
+ if (FID) {
+ // Non-vp version will have same arg types except mask and vector
+ // length.
+ assert(ICA.getArgTypes().size() >= 2 &&
+ "Expected VPIntrinsic to have Mask and Vector Length args and "
+ "types");
+ ArrayRef<Type *> NewTys = ArrayRef(ICA.getArgTypes()).drop_back(2);
+
+ // VPReduction intrinsics have a start value argument that their non-vp
+ // counterparts do not have, except for the fadd and fmul non-vp
+ // counterpart.
+ if (VPReductionIntrinsic::isVPReduction(ICA.getID()) &&
+ *FID != Intrinsic::vector_reduce_fadd &&
+ *FID != Intrinsic::vector_reduce_fmul)
+ NewTys = NewTys.drop_front();
+
+ IntrinsicCostAttributes NewICA(*FID, ICA.getReturnType(), NewTys,
+ ICA.getFlags());
+ return thisT()->getIntrinsicInstrCost(NewICA, CostKind);
+ }
+ }
+
if (ICA.isTypeBasedOnly())
return getTypeBasedIntrinsicInstrCost(ICA, CostKind);
@@ -1823,68 +1884,6 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
}
}
- // VP Intrinsics should have the same cost as their non-vp counterpart.
- // TODO: Adjust the cost to make the vp intrinsic cheaper than its non-vp
- // counterpart when the vector length argument is smaller than the maximum
- // vector length.
- // TODO: Support other kinds of VPIntrinsics
- if (VPIntrinsic::isVPIntrinsic(ICA.getID())) {
- std::optional<unsigned> FOp =
- VPIntrinsic::getFunctionalOpcodeForVP(ICA.getID());
- if (FOp) {
- if (ICA.getID() == Intrinsic::vp_load) {
- Align Alignment;
- if (auto *VPI = dyn_cast_or_null<VPIntrinsic>(ICA.getInst()))
- Alignment = VPI->getPointerAlignment().valueOrOne();
- unsigned AS = 0;
- if (ICA.getArgs().size() > 1)
- if (auto *PtrTy =
- dyn_cast<PointerType>(ICA.getArgs()[0]->getType()))
- AS = PtrTy->getAddressSpace();
- return thisT()->getMemoryOpCost(*FOp, ICA.getReturnType(), Alignment,
- AS, CostKind);
- }
- if (ICA.getID() == Intrinsic::vp_store) {
- Align Alignment;
- if (auto *VPI = dyn_cast_or_null<VPIntrinsic>(ICA.getInst()))
- Alignment = VPI->getPointerAlignment().valueOrOne();
- unsigned AS = 0;
- if (ICA.getArgs().size() >= 2)
- if (auto *PtrTy =
- dyn_cast<PointerType>(ICA.getArgs()[1]->getType()))
- AS = PtrTy->getAddressSpace();
- return thisT()->getMemoryOpCost(*FOp, Args[0]->getType(), Alignment,
- AS, CostKind);
- }
- if (VPBinOpIntrinsic::isVPBinOp(ICA.getID())) {
- return thisT()->getArithmeticInstrCost(*FOp, ICA.getReturnType(),
- CostKind);
- }
- }
-
- std::optional<Intrinsic::ID> FID =
- VPIntrinsic::getFunctionalIntrinsicIDForVP(ICA.getID());
- if (FID) {
- // Non-vp version will have same Args/Tys except mask and vector length.
- assert(ICA.getArgs().size() >= 2 && ICA.getArgTypes().size() >= 2 &&
- "Expected VPIntrinsic to have Mask and Vector Length args and "
- "types");
- ArrayRef<Type *> NewTys = ArrayRef(ICA.getArgTypes()).drop_back(2);
-
- // VPReduction intrinsics have a start value argument that their non-vp
- // counterparts do not have, except for the fadd and fmul non-vp
- // counterpart.
- if (VPReductionIntrinsic::isVPReduction(ICA.getID()) &&
- *FID != Intrinsic::vector_reduce_fadd &&
- *FID != Intrinsic::vector_reduce_fmul)
- NewTys = NewTys.drop_front();
-
- IntrinsicCostAttributes NewICA(*FID, ICA.getReturnType(), NewTys,
- ICA.getFlags());
- return thisT()->getIntrinsicInstrCost(NewICA, CostKind);
- }
- }
-
// Assume that we need to scalarize this intrinsic.)
// Compute the scalarization overhead based on Args for a vector
// intrinsic.
diff --git a/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp b/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
index 6344bc4664d3b6..d98be8a331417c 100644
--- a/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
+++ b/llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
@@ -1104,32 +1104,6 @@ RISCVTTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
return Cost * LT.first;
break;
}
- // vp integer arithmetic ops.
- case Intrinsic::vp_add:
- case Intrinsic::vp_and:
- case Intrinsic::vp_ashr:
- case Intrinsic::vp_lshr:
- case Intrinsic::vp_mul:
- case Intrinsic::vp_or:
- case Intrinsic::vp_sdiv:
- case Intrinsic::vp_shl:
- case Intrinsic::vp_srem:
- case Intrinsic::vp_sub:
- case Intrinsic::vp_udiv:
- case Intrinsic::vp_urem:
- case Intrinsic::vp_xor:
- // vp float arithmetic ops.
- case Intrinsic::vp_fadd:
- case Intrinsic::vp_fsub:
- case Intrinsic::vp_fmul:
- case Intrinsic::vp_fdiv:
- case Intrinsic::vp_frem: {
- std::optional<unsigned> FOp =
- VPIntrinsic::getFunctionalOpcodeForVP(ICA.getID());
- assert(FOp.has_value());
- return getArithmeticInstrCost(*FOp, ICA.getReturnType(), CostKind);
- break;
- }
// vp int cast ops.
case Intrinsic::vp_trunc:
case Intrinsic::vp_zext:
@@ -1163,23 +1137,6 @@ RISCVTTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
return getCmpSelInstrCost(*FOp, ICA.getArgTypes()[0], ICA.getReturnType(),
UI->getPredicate(), CostKind);
}
- // vp load/store
- case Intrinsic::vp_load:
- case Intrinsic::vp_store: {
- if (!ICA.getInst())
- break;
- Intrinsic::ID IID = ICA.getID();
- std::optional<unsigned> FOp = VPIntrinsic::getFunctionalOpcodeForVP(IID);
- assert(FOp.has_value());
- auto *UI = cast<VPIntrinsic>(ICA.getInst());
- if (ICA.getID() == Intrinsic::vp_load)
- return getMemoryOpCost(
- *FOp, ICA.getReturnType(), UI->getPointerAlignment(),
- UI->getOperand(0)->getType()->getPointerAddressSpace(), CostKind);
- return getMemoryOpCost(
- *FOp, ICA.getArgTypes()[0], UI->getPointerAlignment(),
- UI->getOperand(1)->getType()->getPointerAddressSpace(), CostKind);
- }
case Intrinsic::vp_select: {
Intrinsic::ID IID = ICA.getID();
std::optional<unsigned> FOp = VPIntrinsic::getFunctionalOpcodeForVP(IID);
diff --git a/llvm/test/Analysis/CostModel/RISCV/rvv-intrinsics.ll b/llvm/test/Analysis/CostModel/RISCV/rvv-intrinsics.ll
index bb98508f239c1b..ed8b928a5108b7 100644
--- a/llvm/test/Analysis/CostModel/RISCV/rvv-intrinsics.ll
+++ b/llvm/test/Analysis/CostModel/RISCV/rvv-intrinsics.ll
@@ -746,37 +746,37 @@ define void @abs() {
; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
;
; TYPEBASED-LABEL: 'abs'
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 14 for instruction: %1 = call <2 x i8> @llvm.vp.abs.v2i8(<2 x i8> undef, i1 false, <2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %1 = call <2 x i8> @llvm.vp.abs.v2i8(<2 x i8> undef, i1 false, <2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %2 = call <2 x i8> @llvm.abs.v2i8(<2 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 30 for instruction: %3 = call <4 x i8> @llvm.vp.abs.v4i8(<4 x i8> undef, i1 false, <4 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %3 = call <4 x i8> @llvm.vp.abs.v4i8(<4 x i8> undef, i1 false, <4 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %4 = call <4 x i8> @llvm.abs.v4i8(<4 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 62 for instruction: %5 = call <8 x i8> @llvm.vp.abs.v8i8(<8 x i8> undef, i1 false, <8 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %5 = call <8 x i8> @llvm.vp.abs.v8i8(<8 x i8> undef, i1 false, <8 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %6 = call <8 x i8> @llvm.abs.v8i8(<8 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 126 for instruction: %7 = call <16 x i8> @llvm.vp.abs.v16i8(<16 x i8> undef, i1 false, <16 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %7 = call <16 x i8> @llvm.vp.abs.v16i8(<16 x i8> undef, i1 false, <16 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %8 = call <16 x i8> @llvm.abs.v16i8(<16 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 14 for instruction: %9 = call <2 x i64> @llvm.vp.abs.v2i64(<2 x i64> undef, i1 false, <2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %9 = call <2 x i64> @llvm.vp.abs.v2i64(<2 x i64> undef, i1 false, <2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %10 = call <2 x i64> @llvm.abs.v2i64(<2 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 33 for instruction: %11 = call <4 x i64> @llvm.vp.abs.v4i64(<4 x i64> undef, i1 false, <4 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %11 = call <4 x i64> @llvm.vp.abs.v4i64(<4 x i64> undef, i1 false, <4 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %12 = call <4 x i64> @llvm.abs.v4i64(<4 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 69 for instruction: %13 = call <8 x i64> @llvm.vp.abs.v8i64(<8 x i64> undef, i1 false, <8 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %13 = call <8 x i64> @llvm.vp.abs.v8i64(<8 x i64> undef, i1 false, <8 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %14 = call <8 x i64> @llvm.abs.v8i64(<8 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 141 for instruction: %15 = call <16 x i64> @llvm.vp.abs.v16i64(<16 x i64> undef, i1 false, <16 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %15 = call <16 x i64> @llvm.vp.abs.v16i64(<16 x i64> undef, i1 false, <16 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %16 = call <16 x i64> @llvm.abs.v16i64(<16 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %17 = call <vscale x 2 x i8> @llvm.vp.abs.nxv2i8(<vscale x 2 x i8> undef, i1 false, <vscale x 2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %17 = call <vscale x 2 x i8> @llvm.vp.abs.nxv2i8(<vscale x 2 x i8> undef, i1 false, <vscale x 2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %18 = call <vscale x 2 x i8> @llvm.abs.nxv2i8(<vscale x 2 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %19 = call <vscale x 4 x i8> @llvm.vp.abs.nxv4i8(<vscale x 4 x i8> undef, i1 false, <vscale x 4 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %19 = call <vscale x 4 x i8> @llvm.vp.abs.nxv4i8(<vscale x 4 x i8> undef, i1 false, <vscale x 4 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %20 = call <vscale x 4 x i8> @llvm.abs.nxv4i8(<vscale x 4 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %21 = call <vscale x 8 x i8> @llvm.vp.abs.nxv8i8(<vscale x 8 x i8> undef, i1 false, <vscale x 8 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %21 = call <vscale x 8 x i8> @llvm.vp.abs.nxv8i8(<vscale x 8 x i8> undef, i1 false, <vscale x 8 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %22 = call <vscale x 8 x i8> @llvm.abs.nxv8i8(<vscale x 8 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %23 = call <vscale x 16 x i8> @llvm.vp.abs.nxv16i8(<vscale x 16 x i8> undef, i1 false, <vscale x 16 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %23 = call <vscale x 16 x i8> @llvm.vp.abs.nxv16i8(<vscale x 16 x i8> undef, i1 false, <vscale x 16 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %24 = call <vscale x 16 x i8> @llvm.abs.nxv16i8(<vscale x 16 x i8> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %25 = call <vscale x 2 x i64> @llvm.vp.abs.nxv2i64(<vscale x 2 x i64> undef, i1 false, <vscale x 2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %25 = call <vscale x 2 x i64> @llvm.vp.abs.nxv2i64(<vscale x 2 x i64> undef, i1 false, <vscale x 2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %26 = call <vscale x 2 x i64> @llvm.abs.nxv2i64(<vscale x 2 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %27 = call <vscale x 4 x i64> @llvm.vp.abs.nxv4i64(<vscale x 4 x i64> undef, i1 false, <vscale x 4 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %27 = call <vscale x 4 x i64> @llvm.vp.abs.nxv4i64(<vscale x 4 x i64> undef, i1 false, <vscale x 4 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %28 = call <vscale x 4 x i64> @llvm.abs.nxv4i64(<vscale x 4 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %29 = call <vscale x 8 x i64> @llvm.vp.abs.nxv8i64(<vscale x 8 x i64> undef, i1 false, <vscale x 8 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %29 = call <vscale x 8 x i64> @llvm.vp.abs.nxv8i64(<vscale x 8 x i64> undef, i1 false, <vscale x 8 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %30 = call <vscale x 8 x i64> @llvm.abs.nxv8i64(<vscale x 8 x i64> undef, i1 false)
-; TYPEBASED-NEXT: Cost Model: Invalid cost for instruction: %31 = call <vscale x 16 x i64> @llvm.vp.abs.nxv16i64(<vscale x 16 x i64> undef, i1 false, <vscale x 16 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %31 = call <vscale x 16 x i64> @llvm.vp.abs.nxv16i64(<vscale x 16 x i64> undef, i1 false, <vscale x 16 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %32 = call <vscale x 16 x i64> @llvm.abs.nxv16i64(<vscale x 16 x i64> undef, i1 false)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
;
@@ -1218,37 +1218,37 @@ define void @reduce_add() {
; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
;
; TYPEBASED-LABEL: 'reduce_add'
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %1 = call i8 @llvm.vp.reduce.add.v2i8(i8 undef, <2 x i8> undef, <2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %1 = call i8 @llvm.vp.reduce.add.v2i8(i8 undef, <2 x i8> undef, <2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %2 = call i8 @llvm.vector.reduce.add.v2i8(<2 x i8> undef)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 26 for instruction: %3 = call i8 @llvm.vp.reduce.add.v4i8(i8 undef, <4 x i8> undef, <4 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %3 = call i8 @llvm.vp.reduce.add.v4i8(i8 undef, <4 x i8> undef, <4 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %4 = call i8 @llvm.vector.reduce.add.v4i8(<4 x i8> undef)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 54 for instruction: %5 = call i8 @llvm.vp.reduce.add.v8i8(i8 undef, <8 x i8> undef, <8 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %5 = call i8 @llvm.vp.reduce.add.v8i8(i8 undef, <8 x i8> undef, <8 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 5 for instruction: %6 = call i8 @llvm.vector.reduce.add.v8i8(<8 x i8> undef)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 110 for instruction: %7 = call i8 @llvm.vp.reduce.add.v16i8(i8 undef, <16 x i8> undef, <16 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %7 = call i8 @llvm.vp.reduce.add.v16i8(i8 undef, <16 x i8> undef, <16 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %8 = call i8 @llvm.vector.reduce.add.v16i8(<16 x i8> undef)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 12 for instruction: %9 = call i64 @llvm.vp.reduce.add.v2i64(i64 undef, <2 x i64> undef, <2 x i1> undef, i32 undef)
+; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %9 = call i64 @llvm.vp.reduce.add.v2i64(i64 undef, <2 x i64> undef, <2 x i1> undef, i32 undef)
; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %10 = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> undef)
-; TYPEBASED-NEXT: Cost Model: Found an estimated cost of 26 for instruction: %11 = call i64 @llvm.vp.reduce.add.v4i64(i64 undef, <4 x i64> undef, <4 x i1> und...
[truncated]
|
LGTM |
if (auto *PtrTy = dyn_cast<PointerType>(ICA.getArgTypes()[1])) | ||
AS = PtrTy->getAddressSpace(); | ||
return thisT()->getMemoryOpCost(*FOp, ICA.getArgTypes()[0], Alignment, | ||
AS, CostKind); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Querying getMemoryOpCost()
without underlying instruction which may cause ARM TTI to return wrong costs.
But I think it's fine since they are not using vp intrinsics.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. This is from the existing code though so the non type-based cost would have been incorrect anyway
3953216
to
7633886
Compare
; ALL-NEXT: [[TMP1:%.*]] = insertelement <vscale x 1 x i64> poison, i64 [[Y:%.*]], i64 0 | ||
; ALL-NEXT: [[TMP2:%.*]] = shufflevector <vscale x 1 x i64> [[TMP1]], <vscale x 1 x i64> poison, <vscale x 1 x i32> zeroinitializer | ||
; ALL-NEXT: [[TMP3:%.*]] = call <vscale x 1 x i64> @llvm.vp.umax.nxv1i64(<vscale x 1 x i64> [[TMP2]], <vscale x 1 x i64> shufflevector (<vscale x 1 x i64> insertelement (<vscale x 1 x i64> poison, i64 42, i64 0), <vscale x 1 x i64> poison, <vscale x 1 x i32> zeroinitializer), <vscale x 1 x i1> [[MASK]], i32 [[EVL:%.*]]) | ||
; ALL-NEXT: [[TMP4:%.*]] = call <vscale x 1 x i64> @llvm.vp.mul.nxv1i64(<vscale x 1 x i64> [[X:%.*]], <vscale x 1 x i64> [[TMP3]], <vscale x 1 x i1> [[MASK]], i32 [[EVL]]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now that umax and friends are returning a proper cost, it looks like VectorCombine has decided not to scalarize this. Probably because it was previously going from an invalid vector cost to a valid scalar cost.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
We have a lot of code in RISCVTTIImpl::getIntrinsicInstrCost for vp intrinsics, which just forward the cost to the underlying non-vp cost function. However I just also noticed that there is generic code in BasicTTIImpl's getIntrinsicInstrCost that does the same thing, added in llvm#67178. The only difference is that BasicTTIImpl doesn't yet handle it for type-based costing. There doesn't seem to be any reason that it can't since it's just inspecting the argument types. This shuffles the VP costing up to handle both regular and type-based costing, and begins to deduplicate the VP specific costing in RISCVTTIImpl by moving them into BasicTTIImpl.h. It's not NFC since it picks up a couple of VP nodes that had slipped through the cracks. Future PRs can begin to move more of the code from RISCVTTIImpl to BasicTTIImpl.
7633886
to
98ed7ca
Compare
We have a lot of code in RISCVTTIImpl::getIntrinsicInstrCost for vp intrinsics, which just forward the cost to the underlying non-vp cost function. However I just also noticed that there is generic code in BasicTTIImpl's getIntrinsicInstrCost that does the same thing, added in llvm#67178. The only difference is that BasicTTIImpl doesn't yet handle it for type-based costing. There doesn't seem to be any reason that it can't since it's just inspecting the argument types. This shuffles the VP costing up to handle both regular and type-based costing, which allows us to deduplicate some of the VP specific costing in RISCVTTIImpl by delegating it to BasicTTIImpl.h. More of those nodes can be moved over to BasicTTIImpl.h later. It's not NFC since it picks up a couple of VP nodes that had slipped through the cracks. Future PRs can begin to move more of the code from RISCVTTIImpl to BasicTTIImpl.
Refered to: #115983
We have a lot of code in RISCVTTIImpl::getIntrinsicInstrCost for vp intrinsics, which just forward the cost to the underlying non-vp cost function.
However I just also noticed that there is generic code in BasicTTIImpl's getIntrinsicInstrCost that does the same thing, added in #67178. The only difference is that BasicTTIImpl doesn't yet handle it for type-based costing. There doesn't seem to be any reason that it can't since it's just inspecting the argument types.
This shuffles the VP costing up to handle both regular and type-based costing, which allows us to deduplicate some of the VP specific costing in RISCVTTIImpl by delegating it to BasicTTIImpl.h. More of those nodes can be moved over to BasicTTIImpl.h later.
It's not NFC since it picks up a couple of VP nodes that had slipped through the cracks. Future PRs can begin to move more of the code from RISCVTTIImpl to BasicTTIImpl.