Improve error handling of GCE Attach Errors

[davidz627]

Depending on the type of error returned we should be returning:
"ABORTED" for operation pending errors
"RESOURCE_EXHAUSTED" for max volumes attached errors
"FAILED_PRECONDITION" for volume already attached to another node errors

right now we just return an INTERNAL ERROR for any gce attach error during controller publish volume

/cc @msau42

[hantaowang]

For attach at least, it seems the codes are not actually used in terms of the isFinalError logic, only logging/events.

From here, it seems that RESOURCE_EXHAUSTED should be used for the problem where too many devices are being attached at the same time, rather than the max devices per node reached issue.

[davidz627]

according to the spec:

Max volumes attached | 8 RESOURCE_EXHAUSTED | Indicates that the maximum supported number of volumes that can be attached to the specified node are already attached. Therefore, this operation will fail until at least one of the existing attached volumes is detached from the node. | Caller MUST ensure that the number of volumes already attached to the node is less then the maximum supported number of volumes before retrying with exponential backoff.

Looks like the external-attacher may be interpreting the error incorrectly. Maybe we should create an issue (or fix) there.

And no matter what the external-attacher says we need to conform to the spec (or change the spec if it is wrong) as the spec is the official contract we support and other entities could interact with our driver in unforseen or unknown ways that depend on Spec functionality

[davidz627]

in fact I would think that it isFinalError=true in the RESOURCE_EXHAUSTED case

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[davidz627]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[msau42]

/lifecycle frozen