revert the MoE dependence

[zhyncs]

Motivation

follow-up: Add the CUDA 12.5 runtime Docker image and use the cuBLAS Grouped GEMM API.
https://developer.nvidia.com/blog/introducing-grouped-gemm-apis-in-cublas-and-more-performance-updates/

Modifications

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling.

[zhyncs]

cc @BBuf @ispobock