[GPU] Match TileAndFuse Matmul heuristics to VectorDistribute

[nirvedhmeshram]

This provides comparable performance for CI tests.

[nirvedhmeshram]

It appears punet (which I assume has convs and hence the IGEMM path) prefers the old configurations. Going to experiment to see if having different heuristics for those two has better perf on CI tests and update this PR accordingly.

[qedawkins]

We could make the seeds a parameter to getMmaScheduleFromProblemAndTarget and pass different ones for conv and matmul

[nirvedhmeshram]

We could make the seeds a parameter to getMmaScheduleFromProblemAndTarget and pass different ones for conv and matmul

Yeah thats what I was thinking too, there is also the transposedLhs and transposedRhs bools that have an effect on the GEMM side need to see how sensitive the conv dispatches are to that..

[nirvedhmeshram]

Here is the clip performance with the old(exisiting) heuristics

Here it is with new heuristic (copied from vectordistribute)(from this PR)

I will share the configs in the problem dispatches in the next message

[nirvedhmeshram]

dispatch_120
is 116 us with old heuristic and 78us with new heuristic

Old(bad) config is

{lowering_config = #iree_gpu.lowering_config
<{mma_kind = #iree_gpu.mma_layout<MFMA_F32_16x16x16_F16>, 
promote_operands = [0, 1], reduction = [0, 0, 4], subgroup = [1, 4, 0], workgroup = [16, 256, 0]}>} 

New config is

 {lowering_config = #iree_gpu.lowering_config
<{mma_kind = #iree_gpu.mma_layout<MFMA_F32_16x16x16_F16>, 
promote_operands = [0, 1], reduction = [0, 0, 8], subgroup = [1, 2, 0], workgroup = [16, 128, 0]}>}

[kuhar]

I don't want to block it if improves performance on real-world programs, but I'd really hope we can explain our choice and derive it from the hw constants

[nirvedhmeshram]

I don't want to block it if improves performance on real-world programs, but I'd really hope we can explain our choice and derive it from the hw constants

That is fair, I was able to derive from the hw constants, just need a larger bestKElementCountPerSubgroup which can still be a function of the kCacheLineSizeBits. However, uncovered a bug in the process
#19675. Currently pushing with a workaround.

[nirvedhmeshram]

I believe the regression seen after heuristic change, could be something to do with what we found in #19671. I am going to wait for that to land and then rebase and see how the CI does,