Remove mc mirror support from Operator (#133)

This PR removes the mc mirror support in favor of upcoming
server side copy in MinIO.

README.md

@@ -2,7 +2,7 @@
 
 MinIO is a high performance distributed object storage server, designed for large-scale private cloud infrastructure. MinIO is designed in a cloud-native manner to scale sustainably in multi-tenant environments. Orchestration platforms like Kubernetes provide perfect launchpad for MinIO to scale.
 
-MinIO-Operator brings native MinIO, [MCS](https://github.com/minio/mcs), [KES](https://github.com/minio/kes) and [mc Mirror](https://docs.minio.io/docs/minio-client-complete-guide.html#mirror) support to Kubernetes. MinIO-Operator currently supports following features:
+MinIO-Operator brings native MinIO, [MCS](https://github.com/minio/mcs), and [KES](https://github.com/minio/kes) support to Kubernetes. MinIO-Operator currently supports following features:
 
 | Feature                 | Reference Document |
 |-------------------------|--------------------|
@@ -11,7 +11,6 @@ MinIO-Operator brings native MinIO, [MCS](https://github.com/minio/mcs), [KES](h
 | Expand an existing MinIO cluster                               | [Expand a MinIO Cluster](https://github.com/minio/minio-operator/blob/master/docs/adding-zones.md). |
 | Deploy MCS with MinIO cluster  | [Deploy MinIO Instance with MCS](https://github.com/minio/minio-operator/blob/master/docs/mcs.md). |
 | Deploy KES with MinIO cluster  | [Deploy MinIO Instance with KES](https://github.com/minio/minio-operator/blob/master/docs/kes.md). |
-| Deploy mc mirror  | [Deploy Mirror Instance](https://github.com/minio/minio-operator/blob/master/docs/mirror.md). |
 
 ## Getting Started
 

docs/operator-fields.md

@@ -11,10 +11,6 @@ MinIO Operator creates native Kubernetes resources within the cluster. If the Mi
 - Secret: `minioinstance-tls` (If `requestAutoCert` is enabled)
 - CertificateSigningRequest: `minioinstance-csr` (If `requestAutoCert` is enabled)
 
-If the MirrorInstance is named as `mirrorinstance`, resources and their names as created by MinIO Operator are:
-
-- Job: `mirrorinstance`
-
 ## MinIOInstance Fields
 
 | Field                 | Description |
@@ -52,19 +48,5 @@ If the MirrorInstance is named as `mirrorinstance`, resources and their names as
 | spec.kes | Defines the KES configuration. Refer [this](https://github.com/minio/kes) |
 | spec.kes.replicas | Number of KES pods to be created. |
 | spec.kes.image | Defines the KES image. |
-| spec.kes.kesSecret | Secret to specify KES Configuration. This is a mandatory field. |
-| spec.kes.metadata | This allows a way to map metadata to the KES pods. Internally `metadata` is a struct type as [explained here](https://godoc.org/k8s.io/apimachinery/pkg/apis/meta/v1#ObjectMeta). |
-
-## MirrorInstance Fields
-
-| Field                 | Description |
-|-----------------------|-------------|
-| kind                  | This defines the resource type to be created. MinIO Operator CRD defines the `kind` for Mirror Operation as `MirrorInstance`.|
-| metadata              | This field allows a way to assign metadata to a MirrorInstance. Internally `metadata` is a struct type as [explained here](https://godoc.org/k8s.io/apimachinery/pkg/apis/meta/v1#ObjectMeta).|
-| spec.metadata         | Define the object metadata to be passed to all the members pods of this MirrorInstance. This allows adding annotations and labels.|
-| spec.image            | Set the container registry and image tag for MinIO Client to be used in the MirrorInstance.|
-| spec.env              | Add Mirror specific environment variables. There are two mandatory fields required for Mirror to work. `MC_HOST_source` is the environment variable to specify the source MinIO cluster for mirror operation. `MC_HOST_target` is the environment variable to specify the target MinIO cluster for mirror operation. The value of these environment variables in the format `https://<access_key>:<secret_key>@<minio_server_url>`. Refer [the document](https://github.com/minio/mc/blob/master/docs/minio-client-complete-guide.md#specify-host-configuration-through-environment-variable) for further details. |
-| spec.args.source      | Specify the source location for mirror operation. This can be a top level alias (e.g `source`), a bucket (e.g `source/bucket`), or a prefix (e.g `source/bucket/prefix`.) |
-| spec.args.target      | Specify the target location for mirror operation. This can be a top level alias (e.g `target`), a bucket (e.g `target/bucket`), or a prefix (e.g `target/bucket/prefix`.)  |
-| spec.args.flags       | Specify the flags to fine tune the mirror operation. Refer the [mc mirror documentation](https://github.com/minio/mc/blob/master/docs/minio-client-complete-guide.md#mirror) for possible values for flags. |
-| spec.resources | Specify CPU and Memory resources for each MirrorInstance Job container. Refer [this document](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-types) for details. |
+| spec.kes.configSecret | Secret to specify KES Configuration. This is a mandatory field. |
+| spec.kes.metadata | This allows a way to map metadata to the KES pods. Internally `metadata` is a struct type as [explained here](https://godoc.org/k8s.io/apimachinery/pkg/apis/meta/v1#ObjectMeta). [Note: Should match the labels in `spec.kes.selector`] |

main.go

@@ -34,7 +34,6 @@ import (
 	clientset "github.com/minio/minio-operator/pkg/client/clientset/versioned"
 	informers "github.com/minio/minio-operator/pkg/client/informers/externalversions"
 	"github.com/minio/minio-operator/pkg/controller/cluster"
-	"github.com/minio/minio-operator/pkg/controller/mirror"
 	kubeinformers "k8s.io/client-go/informers"
 	"k8s.io/client-go/kubernetes"
 	certapi "k8s.io/client-go/kubernetes/typed/certificates/v1beta1"
@@ -101,15 +100,12 @@ func main() {
 
 	var kubeInformerFactory kubeinformers.SharedInformerFactory
 	var minioInformerFactory informers.SharedInformerFactory
-	var mirrorInformerFactory informers.SharedInformerFactory
 	if isNamespaced {
 		kubeInformerFactory = kubeinformers.NewSharedInformerFactoryWithOptions(kubeClient, time.Second*30, kubeinformers.WithNamespace(namespace))
 		minioInformerFactory = informers.NewSharedInformerFactoryWithOptions(controllerClient, time.Second*30, informers.WithNamespace(namespace))
-		mirrorInformerFactory = informers.NewSharedInformerFactoryWithOptions(controllerClient, time.Second*30, informers.WithNamespace(namespace))
 	} else {
 		kubeInformerFactory = kubeinformers.NewSharedInformerFactory(kubeClient, time.Second*30)
 		minioInformerFactory = informers.NewSharedInformerFactory(controllerClient, time.Second*30)
-		mirrorInformerFactory = informers.NewSharedInformerFactory(controllerClient, time.Second*30)
 	}
 
 	mainController := cluster.NewController(kubeClient, controllerClient, *certClient,
@@ -119,26 +115,16 @@ func main() {
 		minioInformerFactory.Operator().V1().MinIOInstances(),
 		kubeInformerFactory.Core().V1().Services())
 
-	mirrorController := mirror.NewController(kubeClient, controllerClient,
-		kubeInformerFactory.Batch().V1().Jobs(),
-		mirrorInformerFactory.Operator().V1().MirrorInstances())
-
 	go kubeInformerFactory.Start(stopCh)
 	go minioInformerFactory.Start(stopCh)
-	go mirrorInformerFactory.Start(stopCh)
 
 	if err = mainController.Start(2, stopCh); err != nil {
 		klog.Fatalf("Error running mainController: %s", err.Error())
 	}
 
-	if err = mirrorController.Start(2, stopCh); err != nil {
-		klog.Fatalf("Error running mirrorController: %s", err.Error())
-	}
-
 	<-stopCh
 	klog.Info("Shutting down the MinIO Operator")
 	mainController.Stop()
-	mirrorController.Stop()
 }
 
 // setupSignalHandler registered for SIGTERM and SIGINT. A stop channel is returned

minio-operator.yaml

@@ -90,29 +90,6 @@ spec:
           type: string
           jsonPath: ".status.currentState"
 ---
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  name: mirrorinstances.operator.min.io
-spec:
-  group: operator.min.io
-  scope: Namespaced
-  names:
-    kind: MirrorInstance
-    singular: mirrorinstance
-    plural: mirrorinstances
-  versions:
-    - name: v1
-      served: true
-      storage: true
-      schema:
-        openAPIV3Schema:
-          type: object
-          properties:
-            spec:
-              type: object
-              x-kubernetes-preserve-unknown-fields: true
----
 apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: ClusterRole
 metadata:

pkg/apis/operator.min.io/v1/constants.go

@@ -105,20 +105,6 @@ const LivenessInitialDelay = 120
 // LivenessPeriod specifies the interval in calling the liveness endpoint
 const LivenessPeriod = 60
 
-// MC Mirror Related Constants
-
-// DefaultMCImage specifies the default mc Docker hub image
-const DefaultMCImage = "minio/mc:RELEASE.2020-05-06T18-00-07Z"
-
-// MirrorJobRestartPolicy specifies the restart policy for the job created for mirroring
-const MirrorJobRestartPolicy = corev1.RestartPolicyOnFailure
-
-// DefaultMirrorFlags specifies the restart policy for the job created for mirroring
-var DefaultMirrorFlags = []string{"--no-color", "--json"}
-
-// MirrorCRDResourceKind is the Kind of a Cluster.
-const MirrorCRDResourceKind = "MirrorInstance"
-
 // MCS Related Constants
 
 // DefaultMCSImage specifies the latest MCS Docker hub image
@@ -165,7 +151,7 @@ const KESServicePortName = "http-kes"
 // KESMinIOKey is the name of key that KES creates on the KMS backend
 const KESMinIOKey = "my-minio-key"
 
-// KESJobRestartPolicy specifies the restart policy for the job created for mirroring
+// KESJobRestartPolicy specifies the restart policy for the job created for key creation
 const KESJobRestartPolicy = corev1.RestartPolicyOnFailure
 
 // KESHLSvcNameSuffix specifies the suffix added to MinIOInstance name to create a headless service for KES

pkg/apis/operator.min.io/v1/helper.go

@@ -404,24 +404,3 @@ func (mi *MinIOInstance) OwnerRef() []metav1.OwnerReference {
 		}),
 	}
 }
-
-// EnsureDefaults will ensure that if a user omits and fields in the
-// spec that are required, we set some sensible defaults.
-func (mi *MirrorInstance) EnsureDefaults() *MirrorInstance {
-	if mi.Spec.Image == "" {
-		mi.Spec.Image = DefaultMCImage
-	}
-	return mi
-}
-
-// HasMetadata returns true if the user has provided a pod metadata
-// for a MinIOInstance else false
-func (mi *MirrorInstance) HasMetadata() bool {
-	return mi.Spec.Metadata != nil
-}
-
-// HasSelector returns true if the user has provided a pod selector
-// field (ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-selector)
-func (mi *MirrorInstance) HasSelector() bool {
-	return mi.Spec.Selector != nil
-}

pkg/apis/operator.min.io/v1/names.go

@@ -24,9 +24,6 @@ import (
 // MinIOServerName specifies the default container name for MinIOInstance
 const MinIOServerName = "minio"
 
-// MirrorContainerName specifies the default container name for MirrorInstance
-const MirrorContainerName = "mirror"
-
 // KESContainerName specifies the default container name for KES
 const KESContainerName = "kes"
 

pkg/apis/operator.min.io/v1/register.go

@@ -55,8 +55,6 @@ func addKnownTypes(scheme *runtime.Scheme) error {
 	scheme.AddKnownTypes(SchemeGroupVersion,
 		&MinIOInstance{},
 		&MinIOInstanceList{},
-		&MirrorInstance{},
-		&MirrorInstanceList{},
 	)
 	metav1.AddToGroupVersion(scheme, SchemeGroupVersion)
 	return nil

pkg/apis/operator.min.io/v1/types.go

@@ -203,59 +203,3 @@ type MinIOInstanceList struct {
 
 	Items []MinIOInstance `json:"items"`
 }
-
-// +genclient
-// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
-// +k8s:defaulter-gen=true
-// +kubebuilder:subresource:status
-// +kubebuilder:resource:scope=Namespaced,shortName=mirrorinstance,singular=mirrorinstance
-
-// MirrorInstance is an instance of mc mirror
-// Refer: https://docs.minio.io/docs/minio-client-complete-guide.html#mirror
-type MirrorInstance struct {
-	metav1.TypeMeta   `json:",inline"`
-	metav1.ObjectMeta `json:"metadata"`
-
-	Spec MirrorInstanceSpec `json:"spec"`
-}
-
-// MirrorInstanceSpec defines the specification for a MinIOInstance backup. This includes the source and
-// target where the backup should be stored. Note that both source and target are expected to be
-// AWS S3 API compliant services.
-type MirrorInstanceSpec struct {
-	// Image defines the mc Docker image.
-	// +optional
-	Image string `json:"image,omitempty"`
-	// ImagePullSecret defines the secret to be used for pull image from a private Docker image.
-	// +optional
-	ImagePullSecret corev1.LocalObjectReference `json:"imagePullSecret,omitempty"`
-	// Env is used to add alias (sourceAlias and targetAlias MinIO servers) to mc.
-	Env []corev1.EnvVar `json:"env"`
-	// Args allows configuring fields
-	Args Args `json:"args"`
-	// If provided, use these requests and limit for cpu/memory resource allocation
-	// +optional
-	Resources corev1.ResourceRequirements `json:"resources,omitempty"`
-	// Metadata defines the object metadata passed to each pod that is a part of this MinIOInstance
-	Metadata *metav1.ObjectMeta `json:"metadata,omitempty"`
-	// Selector is a label query over pods that should match the replica count.
-	// +optional
-	Selector *metav1.LabelSelector `json:"selector,omitempty"`
-}
-
-// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
-
-// MirrorInstanceList is a list of MirrorInstance resources
-type MirrorInstanceList struct {
-	metav1.TypeMeta `json:",inline"`
-	metav1.ListMeta `json:"metadata"`
-
-	Items []MirrorInstance `json:"items"`
-}
-
-// Args specifies configuration for mirror jobs
-type Args struct {
-	Source string   `json:"source"`
-	Target string   `json:"target"`
-	Flags  []string `json:"flags"`
-}