[mlir] Use find_package() to locate MLIR static library

[zhanglx13]

The latest version of MLIR exports the static library that can be found by find_package(). The changes made on the MLIR side can be found here.

New names (static library related):

• MLIR project name: rocMLIR
• MLIR package name: rocMLIR
• MLIR static library name: rockCompiler

The corresponding changes on the MIOpen side are:

• Use find_package(rocMLIR 1.0.0) to find the rocMLIR package of version 1.0.0. This is the first version of MLIR that exports the static library.
• For backward compatibility, use find_library() to locate the library in the old way if find_package() does not find the package.
• Add rocMLIR::rockCompiler to the link library of target MIOpen.
• Update the commit id of MLIR in requirements.txt.

[jerryyin]

@zhanglx13 Could you update https://github.com/ROCmSoftwarePlatform/MIOpen/blob/develop/requirements.txt so that on MIOpen end it can pull in latest commit and run through MIOpen regressions?

[jerryyin]

@zhanglx13 FYI this is a ROCm 5.4 feature so can only merge after MIOpen branches.

Could you take a look at test failures and try to work through? Just so that when MIOpen finished branching we are good to go.

[krzysz00]

I thought MIOpen had branched already? In any case, this is still blocking deploying my changes because of the order the changes are going into MLIR.

[zhanglx13]

@junliume Do you know how to rebuild the docker image? For now, Jenkins let me set the parameter DOCKER_IMAGE_OVERRIDE but it does not rebuild the image if a new name is given.

The last successful CI build shows that the compatibility function is working well

[2022-08-18T21:39:49.140Z] -- Could NOT find Miir (missing: Miir_DIR)
[2022-08-18T21:39:49.140Z] -- Falling back to find library libMLIRMIOpen
[2022-08-18T21:39:49.140Z] -- Build with library libMLIRMIOpen: /opt/rocm/lib/libMLIRMIOpen.a

And this also shows that the image used contains the old version of MLIR static library. To test the new exported library, a image needs to be built with the updated version of MLIR in the requirements.txt.

[JehandadKhan]

@junliume Do you know how to rebuild the docker image? For now, Jenkins let me set the parameter DOCKER_IMAGE_OVERRIDE but it does not rebuild the image if a new name is given.

The last successful CI build shows that the compatibility function is working well

[2022-08-18T21:39:49.140Z] -- Could NOT find Miir (missing: Miir_DIR)
[2022-08-18T21:39:49.140Z] -- Falling back to find library libMLIRMIOpen
[2022-08-18T21:39:49.140Z] -- Build with library libMLIRMIOpen: /opt/rocm/lib/libMLIRMIOpen.a

And this also shows that the image used contains the old version of MLIR static library. To test the new exported library, a image needs to be built with the updated version of MLIR in the requirements.txt.

Hello @zhanglx13. You can rebuild the docker yourself with the docker build arguments that our CI echos, push it to either the internal or the external registry and then override the CI runs using the DOCKER_IMAGE_OVERRIDE parameter. Once your PR is merged we can update the image that the CI uses by default.

[zhanglx13]

The old MLIR is checked with build 32.
The new MLIR is checked with build 40.

[shurale-nkn]

LGTM

[zhanglx13]

@junliume I haven't seen a successful build of the docker image with the new dependencies. Still trying to figure out the cause.

[zhanglx13]

There is a blocker on the CK side.
After ROCm/composable_kernel#435 lands, we need to update the CK commit id in requirements and rebuild the docker image.

[zhanglx13]

Another issue caused by CK integration. See #1777 for details.

[zhanglx13]

Decision needs to be made by @junliume @JehandadKhan about CK integration. See #1777 for details.
If it is decided to bound with an old version of CK, then we need to revert fd08663 and update CK's commit id in requirements.txt to the chosen one.
If it is decided to keep up with the new CK, we can wait until all issues are resolved.

[zhanglx13]

Blocked by PR #1761.

[krzysz00]

I'd just like to reiterate that the changes in rocMLIR that require this PR to land are already in our 5.4 release branch, and so therefore this PR must land on MIOpen's 5.4 release branch - or at least that's what I can tell of the situation

[zhanglx13]

@junliume build#80 has all greens. However, rocm/miopen:ci does not use the rocm-5.4 version of MLIR. Therefore, I kicked off build#81, which builds a new docker image with the latest MLIR. I think we should be safe to merge this PR after we see all greens in build#81.