Make ingester errors use mimirpb.ErrorCause

[duricanikolic]

What this PR does

This PR makes a further step towards the write path error handling improvement. Proto message mimirpb.ErrorCause (introduced in #6377) has been extended to cover, together with all possible causes of distributor errors, the causes of ingester errors as well. Its extended definition is:

enum ErrorCause {
  INVALID = 0;
  REPLICAS_DID_NOT_MATCH = 1;
  TOO_MANY_CLUSTERS = 2;
  BAD_DATA = 3;
  INGESTION_RATE_LIMITED = 4;
  REQUEST_RATE_LIMITED = 5;
  INSTANCE_LIMIT = 6;
  SERVICE_UNAVAILABLE = 7;
  TSDB_UNAVAILABLE = 8;
}

Values INSTANCE_LIMIT, SERVICE_UNAVAILABLE and TSDB_UNAVAILABLE have been added by this PR.

The definition of the ingesterError interface, implemented by all ingester-specific errors, has been changed from

type ingesterError interface {
  errorType() ingesterErrorType
}

to

type ingesterError interface {
  errorCause() mimirpb.ErrorCause
}

Its definition is now analogous to the definition of the the distributorError interface (introduced in #6377), which is implemented by all distributor-specific errors.

Moreover, the newErrorWithStatus() constructor has been updated in such a way that in a case of an ingesterError the resulting errorWithStatus contains also a status detail of type mimirpb.WriteErrorDetails corresponding to ingesterError.errorCause().

Which issue(s) this PR fixes or relates to

Part of #6008

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[replay]

I'm surprised the exceeding the user limit would result in bad data.
But since in this PR you don't want to change the behavior this is out of scope of the PR

[duricanikolic]

@replay We can always add an additional cause and change this case. Would it be more appropriate to mark it as TENANT_LIMIT?

[replay]

I think it would be better to not change any logic in this PR, but then maybe make that logic change in a separate PR, just to keep the changes distinct

[replay]

Looks good to me, thx!