Run fp8 models on Ampere GPUs with Marlin Kernels

[RonanKMcGovern]

I see that awq is supported (likely via Marlin?) and am wondering whether fp8 can also be done on Ampere (via dequantisation)? It works on vLLM.

fp8 works natively on Lovelace and Hopper.

[RonanKMcGovern]

I just ran this and it's flawless. On an A40. 3X higher throughput than vllm at batch 64