Document how to use the registry addon - covering usage on none & Windows

[robertdumitrescu]

The operating system version: Microsoft Windows 10 PRO - Version 10.0.17134 Build 17134

Can we add to the documentation a tutorial of some sort on how to "Use local images by re-using the Docker daemon" or how to have a minikube registry up and running on Windows?

I am really struggling with this one and in the end, after spending 6 hours trying to set up with no success I ended up using Amazon ECR for pushing images and then pulling down to Minikube with skaffold.

Some sort of step by step tutorial for local development with local registries or other mechanisms on Windows with Minikube would be highly appreciated.

Thank you! Let me know your thoughts guys.

[AkarshES]

This might be useful: https://blog.hasura.io/sharing-a-local-registry-for-minikube-37c7240d0615/
Essentially the same thing is supposed to be usable by enabling the registry addon in minikube but there is an issue in the current release which has a fix now in this commit

[kumarom]

The https://blog.hasura.io/sharing-a-local-registry-for-minikube-37c7240d0615/ point to https://gist.github.com/coco98/b750b3debc6d517308596c248daf3bb1 which also deploys a daemon sets. This daemon set is missing from minikube addon registry yaml files. Is this something that needs to be fixed..?

I confirm that once this daemon set is created, I can access registry on localhost:5000 when using minikube v1.0.1 with none driver. This is without needing a kubectl port-forward. I was able to push a container image to registry using

docker push localhost:5000/test-img

And then ran it in minikube using

kubectl run -i -t test-img --image=localhost:5000/test-img --restart=Never

The final state of kubernetes was something like below:

ubuntu@ubuntu:~$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                       READY     STATUS      RESTARTS   AGE
default       test-img                                   0/1       Completed   0          4s
kube-system   coredns-78fcdf6894-25hgh                   1/1       Running     0          45m
kube-system   coredns-78fcdf6894-z9rxg                   1/1       Running     0          45m
kube-system   default-http-backend-ddc5ff66b-xnhb4       1/1       Running     0          45m
kube-system   etcd-minikube                              1/1       Running     0          45m
kube-system   kube-addon-manager-minikube                1/1       Running     0          45m
kube-system   kube-apiserver-minikube                    1/1       Running     0          45m
kube-system   kube-controller-manager-minikube           1/1       Running     0          45m
kube-system   kube-proxy-zqrqt                           1/1       Running     0          45m
kube-system   kube-registry-proxy-qrpsd                  1/1       Running     0          9m
kube-system   kube-registry-v0-4kbmn                     1/1       Running     0          9m
kube-system   kube-scheduler-minikube                    1/1       Running     0          45m
kube-system   nginx-ingress-controller-cc65f44b6-c5qvs   1/1       Running     0          45m
kube-system   storage-provisioner                        1/1       Running     0          45m
kube-system   tiller-deploy-7c69c66d57-t2kqm             1/1       Running     0          44m

I also believe that this can be made to work with other minikube drivers. All that is needed is to ensure that Insecure Registries includes minikube IP address. And the docker push needs to be adjusted appropriately.

[kumarom]

@tstromberg Here is the contents of /etc/kubernetes/addons/registry-proxy.yaml that I used:

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  labels:
    kubernetes.io/minikube-addons: registry
    addonmanager.kubernetes.io/mode: Reconcile
  name: registry-proxy
  namespace: kube-system
spec:
  template:
    metadata:
      labels:
        kubernetes.io/minikube-addons: registry
        addonmanager.kubernetes.io/mode: Reconcile
    spec:
      containers:
      - image: gcr.io/google_containers/kube-registry-proxy:0.4
        imagePullPolicy: IfNotPresent
        name: registry-proxy
        ports:
        - name: registry
          containerPort: 80
          hostPort: 5000
        env:
        - name: REGISTRY_HOST
          value: registry.kube-system.svc.cluster.local
        - name: REGISTRY_PORT
          value: "80"
        resources:
          limits:
            cpu: 100m
            memory: 50Mi

Can we include this as part of registry addon please..?

[tstromberg]

@kumarom - Yes, feel free to send me a PR to include this. Here are the files that make up the registry addon:

https://github.com/kubernetes/minikube/tree/b936caf71730c3e91f85543eca3ce5e4cbb2b9bd/deploy/addons/registry

Thank you for sharing your discovery!

[kumarom]

@tstromberg Created a PR #4341 as per your recommendation.

[kumarom]

@robertdumitrescu Does this help you (works with minikube v1.2.0 onwards) ? If yes, can this issue be closed now..

[robertdumitrescu]

Yes, it can be closed. Thank you for all your support guys!

[alirizakeles]

Hi,

Unfortunately proxy does not work or I misunderstood how it works. My minikube version is v1.1.1

$ minikube start
...

$ minikube addons enable registry
registry was successfully enabled

I can see registry and proxy are active running.

$ kubectl get pod --all-namespaces 
NAMESPACE     NAME                               READY   STATUS    RESTARTS   AGE
...
kube-system   kube-proxy-ctr6k                   1/1     Running   0          7m29s
kube-system   kube-scheduler-minikube            1/1     Running   0          6m33s
kube-system   registry-vjf9m                     1/1     Running   0          2m29s
kube-system   storage-provisioner                1/1     Running   0          7m27s

But when I try to reach port 5000 it does not work:

$ minikube ssh
$ curl localhost:5000
curl: (7) Failed to connect to localhost port 5000: Connection refused

Btw, instructions and specs in the blog post mentioned above works well.

[kumarom]

@alirizakeles this should help you (works with minikube v1.2.0 onwards). The change for registry-proxy merged in #4529 which is included in minikube 1.2.0

[alirizakeles]

@kumarom thanks. i did not check the document you mentioned, another blog post helped me and it worked. i am happy with my setup now. after upgrading my minikube installation i'll try the addon again.

[CrossBound]

I'm confused... the link you keep sending people to only has instructions for: "Method 1: Without minikube registry addon".

How does this help me with the registry addon?
I'm in the same boat trying to figure out how to make the minikube registry addon work.

[CrossBound]

Oh, and the instructions are apparently only for linux/mac users... I'm on Windows.

[CrossBound]

It looks like the correct link is this one.

Also, for windows there is some minor modifications that are required. You can see the details here.

I was able to get this to work with Docker for Windows with these modifications.

[CrossBound]

FYI, the instructions here have been updated with instructions for Windows.

[talonx]

The instructions at https://minikube.sigs.k8s.io/docs/handbook/registry/ do not work for Linux anymore with minikube v1.11.0. They used to work with v.1.10. More specifically, the proxy does not listen on 5000 on the host (minikube VM) network anymore.

[medyagh]

@talonx which driver are you using ?

[talonx]

@medyagh I did not specify anything on the CLI - I guess it falls back on virtualbox.