Change Azure global rate limit to per client

[feiskyer]

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:

/kind api-change
/kind bug
/kind cleanup
/kind deprecation
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

/kind feature
/area provider/azure
/priority important-soon
/sig cloud-provider

What this PR does / why we need it:

Change Azure global rate limit to per client, so that one client's rate limit issue won't block another client.

Example configure:

{
                 // default rate limit (enabled).
                 "cloudProviderRatelimit": true,
		"cloudProviderRateLimitBucket": 1,
		"cloudProviderRateLimitBucketWrite": 1,
		"cloudProviderRateLimitQPS": 1,
		"cloudProviderRateLimitQPSWrite": 1,
		"virtualMachineScaleSetRateLimit": {
			// VMSS specific (enabled).
			"cloudProviderRatelimit": true,
			"cloudProviderRateLimitBucket": 2,
			"CloudProviderRateLimitBucketWrite": 2,
			"cloudProviderRateLimitQPS": 0,
			"CloudProviderRateLimitQPSWrite": 0
		},
		"loadBalancerRateLimit": {
			// LB specific (disabled)
			"cloudProviderRatelimit": false,
		},
}

Which issue(s) this PR fixes:

Fixes #86019

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Azure global rate limit is switched to per-client. A set of new rate limit configure options are introduced, including routeRateLimit, SubnetsRateLimit, InterfaceRateLimit, RouteTableRateLimit, LoadBalancerRateLimit, PublicIPAddressRateLimit, SecurityGroupRateLimit, VirtualMachineRateLimit, StorageAccountRateLimit, DiskRateLimit, SnapshotRateLimit, VirtualMachineScaleSetRateLimit and VirtualMachineSizeRateLimit.

The original rate limit options would be default values for those new client's rate limiter.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: feiskyer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• staging/src/k8s.io/legacy-cloud-providers/azure/OWNERS [feiskyer]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[feiskyer]

/test pull-kubernetes-e2e-aks-engine-azure
/test pull-kubernetes-e2e-azure-disk
/test pull-kubernetes-e2e-azure-disk-vmss

[feiskyer]

/assign @andyzhangx

[andyzhangx]

code lgtm
is there upgrade test from old config to new config, make sure old config still works in new config?

[feiskyer]

is there upgrade test from old config to new config, make sure old config still works in new config?

@andyzhangx yep, old configured rate limits are still working.

[feiskyer]

/test pull-kubernetes-e2e-aks-engine-azure
/test pull-kubernetes-e2e-azure-disk
/test pull-kubernetes-e2e-azure-disk-vmss

[feiskyer]

/retest

[andyzhangx]

/lgtm

[feiskyer]

/test pull-kubernetes-e2e-gce