Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Kubernetes cluster support #98

Closed
Bluebugs opened this issue Apr 1, 2022 · 3 comments
Closed

Add Kubernetes cluster support #98

Bluebugs opened this issue Apr 1, 2022 · 3 comments
Labels
enhancement New feature or request meta An issue that describe a high level problem that might require multiple issues to be resolved first

Comments

@Bluebugs
Copy link
Contributor

Bluebugs commented Apr 1, 2022

Is your feature request related to a problem? Please describe:

It is possible to setup a cluster of machine that will be able to run multiple containers at the same time or sequentially using Kubernetes based infrastructure. Technically, it should be possible to reuse most of the existing logic in fyne-cross to drive a cluster to do the build work.

Is it possible to construct a solution with the existing API?

Specifying the cluster is done implicitly with the kubernetes config file. What could be necessary and trigger the switch to use kubernetes could be the specification of a namespace to use with the '-n' parameter usually used by kubectl command.

Describe the solution you'd like to see:

This is still a reflection and not something where I am sure on how things need to be done.

A little bit of background, kubernetes has an API which orchestrate things in the cluster. This has to use it, but this API is low bandwidth. We shouldn't for example transfer file through it in large quantity. It is ok for debug purpose, but not for production. The container in a kubernetes cluster are named pods. They have ressource attached to them, like filesystem and config. The pods are what fyne-cross should control.

Each machine that form a cluster is named a node and with certain configuration, they can scale on demand too. That's something fyne-cross shouldn't try to control.

You can experiment locally with tool like minikube.

The main difference between podman/docker backend and kubernetes backend is that the container is not going to run on the same machine. Which means all the necessary data must be send over to the container and not rely on the availability of the local file system.

My current idea for that purpose is that all the mounting point are packaged with tar zstd and uploaded to a S3 bucket by fyne-cross. This is usually available has a public front service and can be mimicked with open source solution like min.io. Then the pod during initialization is pointed to that S3 bucket and locally unpack the content. This allow for uploading things once and reusing it potentially for multiple build in a row. Once a build is done, the resulting artifact would also be uploaded to S3. fyne-cross could then download them back on the local machine.

All the configuration for this cloud services are usually stored in environment variable or yaml file. We might start by just supporting environment variable. I will try to build a prototype of this shortly to see how usable it is.

@Bluebugs Bluebugs mentioned this issue Apr 6, 2022
3 tasks
@metal3d
Copy link
Contributor

metal3d commented Apr 11, 2022

Actually, the best practice is to use CI/CD tools to make this. And Kubernetes can use "Jobs" to make what you expect too but that's not a good practice. I already have made this with Gitlab runners and Drone.io that builds my apps on Kubernetes.
If you really want to use fyne-cross in Kubernetes with one command line, this can be done with a simple sript that launches the needed image in "wait mode" (sleep 3600), then get the corresponding pod, then launching the build command + kubernetes cp afterward to get back the result. We made this at work to build training supports while our CI/CI system was not ready.
But, if the need is really important, after decided how to write / get the results, maybe I can take a look (Kubernetes is one of my specialities)

@Bluebugs
Copy link
Contributor Author

The main downside of kubernetes cp is that the transfer goes through the control plane and consume bandwidth in the wrong place for every architecture targeted. As who ever is using fyne-cross in that configuration might want to build for multiple target at the same time, avoiding unnecessary transfer is high on the goal which is why I went with the S3 API which seems to be a standard in cloud storage. I should have a prototype PR in the coming days, I will be interested to get your review on it.

@Bluebugs Bluebugs added meta An issue that describe a high level problem that might require multiple issues to be resolved first enhancement New feature or request labels Apr 11, 2022
@Bluebugs Bluebugs mentioned this issue Apr 13, 2022
5 tasks
@Bluebugs Bluebugs mentioned this issue Jun 28, 2022
6 tasks
@Bluebugs
Copy link
Contributor Author

This is currently working in develop, if there is bug we should open follow up issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request meta An issue that describe a high level problem that might require multiple issues to be resolved first
Projects
None yet
Development

No branches or pull requests

2 participants