add support for xpu/KunLunXin device

[fpzh2011]

No description provided.

[github-actions]

View latest API docs preview at: https://oneflow-staging.oss-cn-beijing.aliyuncs.com/docs/Oneflow-Inc/oneflow/pr/10540/

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce RTX 3080 Ti 

❌ OneFlow resnet50 time: 43.8ms (= 4383.3ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 57.3ms (= 5726.3ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.31 (= 57.3ms / 43.8ms)

OneFlow resnet50 time: 26.1ms (= 2613.2ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 37.5ms (= 3747.0ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.43 (= 37.5ms / 26.1ms)

OneFlow resnet50 time: 18.6ms (= 3728.6ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 36.0ms (= 7196.9ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.93 (= 36.0ms / 18.6ms)

OneFlow resnet50 time: 17.7ms (= 3547.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 31.5ms (= 6294.2ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.77 (= 31.5ms / 17.7ms)

OneFlow resnet50 time: 14.8ms (= 2953.4ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 29.7ms (= 5935.4ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 2.01 (= 29.7ms / 14.8ms)

OneFlow swin dataloader time: 0.201s (= 40.117s / 200, num_workers=1)
PyTorch swin dataloader time: 0.130s (= 25.918s / 200, num_workers=1)
Relative speed: 0.646 (= 0.130s / 0.201s)

OneFlow swin dataloader time: 0.059s (= 11.702s / 200, num_workers=4)
PyTorch swin dataloader time: 0.033s (= 6.609s / 200, num_workers=4)
Relative speed: 0.565 (= 0.033s / 0.059s)

OneFlow swin dataloader time: 0.034s (= 6.877s / 200, num_workers=8)
PyTorch swin dataloader time: 0.017s (= 3.305s / 200, num_workers=8)
Relative speed: 0.481 (= 0.017s / 0.034s)

❌ OneFlow resnet50 time: 49.1ms (= 4913.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 64.4ms (= 6435.7ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.31 (= 64.4ms / 49.1ms)

OneFlow resnet50 time: 36.3ms (= 3626.4ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 47.8ms (= 4783.6ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.32 (= 47.8ms / 36.3ms)

OneFlow resnet50 time: 27.6ms (= 5518.3ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 38.8ms (= 7752.3ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.40 (= 38.8ms / 27.6ms)

OneFlow resnet50 time: 25.2ms (= 5037.4ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 39.0ms (= 7800.5ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.55 (= 39.0ms / 25.2ms)

OneFlow resnet50 time: 24.4ms (= 4889.9ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 35.9ms (= 7189.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.47 (= 35.9ms / 24.4ms)

[github-actions]

CI failed when running job: Build llvm15. PR label automerge has been removed

[github-actions]

View latest API docs preview at: https://oneflow-staging.oss-cn-beijing.aliyuncs.com/docs/Oneflow-Inc/oneflow/pr/10540/

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce RTX 3080 Ti 

❌ OneFlow resnet50 time: 43.3ms (= 4326.0ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 57.0ms (= 5703.5ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.32 (= 57.0ms / 43.3ms)

OneFlow resnet50 time: 26.3ms (= 2631.3ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 37.4ms (= 3743.3ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.42 (= 37.4ms / 26.3ms)

OneFlow resnet50 time: 17.7ms (= 3545.2ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 36.9ms (= 7388.6ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 2.08 (= 36.9ms / 17.7ms)

OneFlow resnet50 time: 18.0ms (= 3596.6ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 32.5ms (= 6506.9ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.81 (= 32.5ms / 18.0ms)

OneFlow resnet50 time: 17.4ms (= 3487.1ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 29.3ms (= 5864.6ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.68 (= 29.3ms / 17.4ms)

OneFlow swin dataloader time: 0.201s (= 40.132s / 200, num_workers=1)
PyTorch swin dataloader time: 0.128s (= 25.604s / 200, num_workers=1)
Relative speed: 0.638 (= 0.128s / 0.201s)

OneFlow swin dataloader time: 0.059s (= 11.752s / 200, num_workers=4)
PyTorch swin dataloader time: 0.033s (= 6.537s / 200, num_workers=4)
Relative speed: 0.556 (= 0.033s / 0.059s)

OneFlow swin dataloader time: 0.030s (= 6.083s / 200, num_workers=8)
PyTorch swin dataloader time: 0.017s (= 3.307s / 200, num_workers=8)
Relative speed: 0.544 (= 0.017s / 0.030s)

❌ OneFlow resnet50 time: 49.5ms (= 4949.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 65.4ms (= 6535.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.32 (= 65.4ms / 49.5ms)

OneFlow resnet50 time: 37.5ms (= 3745.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 46.8ms (= 4681.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.25 (= 46.8ms / 37.5ms)

OneFlow resnet50 time: 27.8ms (= 5551.1ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 40.9ms (= 8175.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.47 (= 40.9ms / 27.8ms)

OneFlow resnet50 time: 25.3ms (= 5052.9ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 40.1ms (= 8025.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.59 (= 40.1ms / 25.3ms)

OneFlow resnet50 time: 24.9ms (= 4981.2ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 35.8ms (= 7159.0ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.44 (= 35.8ms / 24.9ms)