fix(#710): when creation/update of a namespace fails,create new ones with new base-name

[MatousJobanek]

There is an issue when reset and update/create actions come after each other too quickly and OS is too slow, then the update/creation fails with conflict for jenkins PVC. The reason is that the original one is still not completely removed while a new one is already requested.

This fix detects such conflicts (or 403 when PVC is in being terminated). If there is such an error - let say for user [email protected] where the namespace is named mjobanek-jenkins - then it deletes all namespaces for the user (all that starts with mjobanek) and adds a suffix to the base-name - mjobanek2 if is available (not used by other user). Then it tries to create new namespaces with this base-name (mjobanek2, mjobanek2-jenkins, ...). If this attempt fails as well, then it returns an error

For more information please read: openshiftio/openshift.io#4598 (comment) and openshiftio/openshift.io#4598 (comment)

I know that the code looks terrible, but it is hard to do it nicely with the current codebase

fixes: #710

[alexeykazakov]

In other our services we actually pass the context as InternalServerError instead of interface{}. Which helps to fetch bugs when a controller doesn't declare 500 result in design on the compilation phase. See https://github.com/fabric8-services/fabric8-common/blob/41debd6a68ab0999841b3fdc53e3603f16fa76e2/goasupport/jsonapi_errors_helpers/jsonapi_errors_converter.go.txt#L149

Not critical for this PR though. Let's address it later (after merging the refactoring PR).

[codecov]

Codecov Report

Merging #714 into master will increase coverage by 2.15%.
The diff coverage is 38.95%.

@@            Coverage Diff             @@
##           master     #714      +/-   ##
==========================================
+ Coverage   59.28%   61.43%   +2.15%     
==========================================
  Files          34       34              
  Lines        2569     2570       +1     
==========================================
+ Hits         1523     1579      +56     
+ Misses        889      824      -65     
- Partials      157      167      +10

Impacted Files	Coverage Δ
controller/tenant.go	19.89% <0%> (+7.72%)	⬆️
openshift/apply.go	62.62% <100%> (+6.95%)	⬆️
tenant/repository.go	70% <100%> (-0.97%)	⬇️
update/update.go	80.16% <100%> (ø)	⬆️
jsonapi/jsonapi_utility.go	69.14% <25%> (-2.28%)	⬇️
openshift/clean_tenant.go	10% <30.43%> (+10%)	⬆️
openshift/init_tenant.go	39.71% <38.98%> (+6.97%)	⬆️
controller/tenants.go	73.01% <50%> (-1.64%)	⬇️
openshift/update.go	76.92% <60%> (-6.42%)	⬇️
... and 2 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update de06a79...155537c. Read the comment docs.