Skip to content
This repository has been archived by the owner on Apr 21, 2019. It is now read-only.

Federation: Support deploying federation control plane on non-cloud environment #101

Closed
5 of 6 tasks
irfanurrehman opened this issue Oct 31, 2017 · 30 comments
Closed
5 of 6 tasks

Federation: Support deploying federation control plane on non-cloud environment #101

irfanurrehman opened this issue Oct 31, 2017 · 30 comments
Assignees
@irfanurrehman
Labels
area/federation lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/multicluster Categorizes an issue or PR as relevant to sig-multicluster.

Comments

@irfanurrehman
Copy link
Contributor

Issue by shashidharatd
Thursday Jan 26, 2017 at 18:42 GMT
Originally opened as kubernetes/kubernetes#40536

There may be scenarios where there is a need to federate k8s clusters on private-cloud (on-premise) environments.
Listed below are some of the tasks to achieve the scenario above

  • Federation control plane deployment itself should be able to be deployed in non-cloud environments. #40516
  • Should be possible to disable persistent storage for etcd. #40862
  • Ability to configure desired DNS provider. Public DNS servers (AWS Route53/ Google CloudDNS) or Private DNS servers (e.g. CoreDNS) #40528
  • Document Federation CoreDNS support. [Federation] Setting up CoreDNS as DNS provider for Cluster Federation kubernetes/website#2810
  • coredns federation provider should support etcd with TLS #47049
  • Document using hosted DNS providers (AWS Route53/ Google CloudDNS) in non-cloud environments.
@irfanurrehman irfanurrehman added area/federation priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/multicluster Categorizes an issue or PR as relevant to sig-multicluster. labels Oct 31, 2017
@irfanurrehman
Copy link
Contributor Author

Comment by shashidharatd
Thursday Jan 26, 2017 at 18:43 GMT

@marun, Please feel free to correct or add any additional points to this issue.

@irfanurrehman
Copy link
Contributor Author

Comment by shashidharatd
Thursday Jan 26, 2017 at 18:54 GMT

This issue will partly address the deployment part of federation on non-google clouds as described in #39870

@irfanurrehman
Copy link
Contributor Author

Comment by marun
Thursday Jan 26, 2017 at 23:36 GMT

I think this issue can serve as an umbrella to track issues targeting improved support for non-cloud deployments with kubefed. Rather than conflating the work involved, work items can be tracked and discussed separately:

@irfanurrehman
Copy link
Contributor Author

Comment by marun
Thursday Jan 26, 2017 at 23:46 GMT

cc: @madhusudancs @kubernetes/sig-federation-misc

@irfanurrehman
Copy link
Contributor Author

Comment by mattymo
Friday Mar 17, 2017 at 14:17 GMT

kubernetes/kubernetes#42995 should be considered for the list

@irfanurrehman
Copy link
Contributor Author

Comment by shashidharatd
Wednesday May 03, 2017 at 13:35 GMT

/assign

@irfanurrehman
Copy link
Contributor Author

Comment by liqlin2015
Monday Aug 07, 2017 at 07:43 GMT

Is there any way we can use a local service load balancer for federation service?

And is there any plan to support services only with external IP? Currently in the dns.go:wantsDNSRecords() function, we hard coded to only support v1.ServiceTypeLoadBalancer type of services. Can we use external IP as another option for on-prem clusters?

@irfanurrehman
Copy link
Contributor Author

Comment by shashidharatd
Monday Aug 07, 2017 at 10:56 GMT

what is the meaning of local service load balancer.

Currently only loadbalancer type services are supported for federated services. Currently there is no other option available as we know, which works.

@irfanurrehman
Copy link
Contributor Author

Comment by luckyfengyong
Monday Aug 07, 2017 at 17:52 GMT

@shashidharatd In the doc of federation service discovery, it is mentioned as follows

image

After checking the codes we found only when a service is created with the type of Loadbalancer, the DNS record of the service is created in federation dns service. Just wondering what if the service is created with exnternalip. For that case, can we also ask federation dns service to create the DNS record?

Basically our use case is just like what this ticket is described to run cluster federation within on-prem environment. However we find there is no out-of-box external loadbalancer service available.

Therefore we will need either ask dns server of cluster federation supporting the service with externalip, or have a out-of-box loadbalancer service (just like coredns as out-of-box dns server of cluster federation)

Any suggestion?

@irfanurrehman
Copy link
Contributor Author

Comment by quinton-hoole
Friday Sep 08, 2017 at 02:19 GMT

Required for GA of federation.

@irfanurrehman
Copy link
Contributor Author

Comment by luxas
Friday Sep 08, 2017 at 11:29 GMT

@quinton-hoole Any relevant design doc/proposal for this issue/feature request?

@irfanurrehman
Copy link
Contributor Author

Comment by quinton-hoole
Thursday Sep 14, 2017 at 17:20 GMT

@luxas Nothing beyond what it linked in this issue, that I am aware of.

@irfanurrehman
Copy link
Contributor Author

Comment by shashidharatd
Wednesday Sep 27, 2017 at 14:13 GMT

All the tasks we envisioned are complete and hence closing this issue. If anyone feels this issue needs to be kept open for additional tasks, please feel free to do so.

@irfanurrehman
Copy link
Contributor Author

Comment by luxas
Wednesday Sep 27, 2017 at 15:31 GMT

@shashidharatd Where is the documentation for this feature?

@irfanurrehman
Copy link
Contributor Author

Comment by shashidharatd
Wednesday Sep 27, 2017 at 15:52 GMT

@luxas, here is the documentation for running federation with CoreDNS as dns provider. https://kubernetes.io/docs/tasks/federation/set-up-coredns-provider-federation.
In non-cloud environments, we could also use hosted DNS providers such as AWS Route53 or Google Cloud-DNS by specifying the dns-provider configuration as secret to kubefed init.

I agree there is no detailed document for user explaining how to use the hosted DNS providers (AWS Route53/Google Cloud-DNS). We shall add this document in the upcoming release.

@irfanurrehman
Copy link
Contributor Author

Comment by shashidharatd
Wednesday Sep 27, 2017 at 15:57 GMT

Added a new item to document explaining how to use the hosted DNS providers in non-cloud environments and so reopened this issue.

@irfanurrehman
Copy link
Contributor Author

Comment by luxas
Wednesday Sep 27, 2017 at 16:09 GMT

Thanks!

@irfanurrehman
Copy link
Contributor Author

Comment by pilsy
Sunday Oct 15, 2017 at 14:48 GMT

Any answer to kubernetes/kubernetes#40536 (comment) ?

@irfanurrehman
Copy link
Contributor Author

cc @shashidharatd

@irfanurrehman irfanurrehman mentioned this issue Oct 31, 2017
Closed
6 tasks
@moonek
Copy link

moonek commented Dec 12, 2017

I want to know the current situation.
I am interested in the on-premise environment.
The only dns-provider in on-premise understood as CoreDNS.
The service type for cross-cluster service discovery is documented as Loadbalancer and External IP.
image
However, the ServiceType in the CoreDNS Setup Guide is written as LoadBalancer or NodePort.
image

It is mentioned that only LoadBalancer is possible in this issue.
Which is right? I am very confused.

@shashidharatd
Copy link
Contributor

If you have a user service that need to be federated across clusters (multi-cluster scenario). The service type has to be a LoadBalancer type in-order to be able to do federated service discovery across clusters.

Now coming to the documents you are referring. The mention is about CoreDNS service. CoreDNS server/service itself runs within a kubernetes cluster and the guide is talking about setting up CoreDNS service.

In some of the on-prem environments like openstack the LoadBalancer type service is possible and on the on-prem environments which lack LoadBalancer type service federated service discovery is not possible through federation currently

@moonek
Copy link

moonek commented Dec 13, 2017

Thank you for confirming.

@mengesb mengesb mentioned this issue Feb 16, 2018
Closed
@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 13, 2018
@dholt
Copy link

dholt commented Mar 16, 2018

Any updates on the last item (Document using hosted DNS providers (AWS Route53/ Google CloudDNS) in non-cloud environments.)?

@shashidharatd
Copy link
Contributor

@dholt, Thanks for asking. We need some volunteers to do that. I would create a separate issue for that item. Would you be interested in taking up some portion?

@shashidharatd
Copy link
Contributor

Created a separate issue to track the above incomplete work mentioned in #101 (comment)

@shashidharatd
Copy link
Contributor

There is some work done by @marun as commented in this kubernetes/kubernetes#40620 (comment) related to configuraing Google Cloud DNS in non-google environment.

@dholt
Copy link

dholt commented Mar 16, 2018

I would be happy to try and help document. I don't think I'm qualified to help implement however, and it looked to me from browsing the code like this feature may be implemented for Google Cloud DNS but not AWS Route53?

@fejta-bot
Copy link

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Apr 15, 2018
@fejta-bot
Copy link

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@irfanurrehman irfanurrehman

Labels
area/federation lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/multicluster Categorizes an issue or PR as relevant to sig-multicluster.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@dholt @moonek @shashidharatd @irfanurrehman @k8s-ci-robot @fejta-bot