CLI support for rancher desktop

[wiruzman]

It could be nice to control rancher desktop through cli for administrators and automation.

Actions you can do through the desktop app also reflects the cli commands:

• Restart
• Change kubernetes context
• Reset kubernetes
• Change kubernetes version(reset implies)
• ...more, you get the idea :)

This could be done by rd alias or something else.

[ericpromislow]

Overview of the first solution:

• Create a backend CommandServer

  • Simple API: Input: a JSON-encoded array representing the command-line
  • Simple API: Output: a JSON-encoded structure containing status, message, and message metadata (i.e. a type)

• The three layers of the backend:

  • CommandHandler: JSON-parse the input and verify it's a valid command, and converts command-output to a JSON payload to return to the client. This includes usage error-messages and help info.
  • Dispatch-table function: A simple function that calls the appropriate CommandWorker method to then handle business logic.
  • CommandWorker - implemented in background.ts, uses existing background.ts code to carry out requested actions.

• Front end will be written in Go, with two layers:

  • a command layer to process the command-line
  • an API layer that talks to the server

• The protocol: the server is just a text-based TCP server, reading and writing JSON, as specified above

• The server starts listening on a randomly selected port on localhost and writes it to APPHOME/.rdCliPort. The client consults that file to find the port to connect to. The permissions on the file are 0o600 so only the user running Rancher Desktop can open that file and get the port, but we might have to change that to allow different users if the app is started automatically by a different user. Listening on localhost is to avoid external threats using port-sniffers.

Installing the front-end client

This is built in Go so we can compile it to a standalone executable ,rd/rd.exe, and bundle it with the rest of the app, in resources/PLATFORM/bin. Users can point to that file using their own devices, but we can also add a command to the menubar, Install CLI, that uses the File-Save dialog to copy the file to a directory of their choosing. The code can even warn them if the targeted directory isn't in the current path.

[ericpromislow]

Currently supported commands

pref - pref NAME: returns status of setting X
set-pref - set-pref NAME=VALUE: change setting NAME to VALUE: boolean|string|number
prefs - prefs: returns a JSON hash of all the settings
version - version: get the current k8s version
versions - versions: aLl supported k8s versions
set-version - set-version VERSION: change to the specified k8s version
reset - reset: Restart Kubernetes
reset-all - reset-all: Restart Kubernetes/VM, clear state
shutdown - shutdown: Shut down the UI
help - help|-h|--help: Show this text
-j - output json
-d - debug/verbose mode

[ericpromislow]

Operation

Rancher Desktop obviously can't respond to commands until the CommandServer has been started. Since it currently interacts mostly with the preferences system, we can actually start it up early in operation. If/when we add more functions, say to interact with the list of port-mappings, we can return a not-available status for premature requests.

Some modal dialogs prevent the CommandServer from functioning, in particular the modal dialog warning the user the app is about to ask for their password. More testing is needed to find out how a reset request will be handled while a modal dialog is present.

What we know so far:

• the password-prompting heads-up dialog blocks the CLI -- messages are processed after the dialog is closed.
• you can run commands during the actual sudo dialog. But on macos if you shutdown the app, the sudo dialog is left standing, as it's implemented with a standalone applet.
• the error-message box doesn't block the CLI

[ericpromislow]

A note on UI<->settings synchronization

Currently when a pref is changed in the UI, we either make a note of that in a notification bar, or tell the user that a restart is needed immediately if they want to change the pref (say by changing container engines). This CLI currently circumvents that. It can disable Kubernetes, and the checkbox is unchecked, but the app is still running k3s, and needs to be restarted. Without a notification bar, the user can't know that.

[ericpromislow]

Proposed Commands

The list in #816 (comment) was just for a working prototype, but we're throwing that away.

New command-list:

set NAME=VALUE ...
-- Updates the specified names, and runs the appropriate kind of restart (fast or slow) at the end if necessary

get (--all | NAME...)
-- Returns the value of the specified settings.
`get --all` returns all settings. It's an error to specify both `--all` and specific names.

SETTING NAMES:

• kubernetes-version -- /v?\d+\.\d+\.\d+/ - 'v' is optional, require all three parts.

• kubernetes-enabled | k8s-enabled -- value is 'true' or 'false' (ignoring case).

• memory |  -- Amount in GB. Suffixes matching /[MG]i?B?/ are supported and treated the usual way.
The default suffix is GiB (1024 ** 3).  Must be numeric. Valid decimal values are supported ('4.5' but not '4.5.6').
If the supplied number is too large an error message is given with the maximum supported size.
Not supported on Windows. See https://github.com/cloudfoundry/bytefmt for a similar implementation.

• cpu -- number of CPUs to assign the VM. Must be numeric. 
If the supplied number is too large an error message is given with the maximum supported count.
Not supported on Windows.

• kubernetes-api-port -- self-explanatory. Must be numeric.

• container-engine - supported values are 'docker' and 'containerd'. 'moby' is silently accepted as an alias for 'dockerd'.

• kubernetes-traefik-enabled -- value is 'true' or 'false' (ignoring case).

ssh [-d DISTRO=rancher-desktop]* [command]
-- On Mac and Linux, this ssh's into the VM (just a wrapper around the `LIMA_HOME... limactl shell 0` dance)
and runs either the specified command (a list of string values that are passed to `/bin/bash`),
or `/bin/bash` on the target VM.
-- On Windows, this does the same thing with 'wsl -d DISTRO COMMAND' where DISTRO by default is rancher-desktop.

* - Windows only.

restart-kubernetes [--clear]
The `--clear` option deletes any Kubernetes workloads and container images.

restart-vm [--clear]

shutdown
-- Shuts down the VM and UI

factory-reset
-- Shuts down the UI and clears everything

GLOBAL OPTIONS:
• --debug|-d
• --verbose|-v
• --json

By default, output is human-readable. With --json, if there are no errors, the only output is the final JSON payload.
Any errors appear in the JSON output payload in an errors: [ERROR-MESSAGE1, ERROR-MESSAGE2, ...] section.
Verbose output is always written immediately to stderr, regardless of the output mode.

Other notes:
• Error, warning, and debug messages are written to stderr. All other output goes to stdout.
• Exit status of 0 on success, 1 on usage errors, 2 on other errors.

For later release:

• list-services [--exclude-kubernetes | -k] [--forwarded-only | -f]
- Lists all the services that are currently running, similar to the `Port Forwarding` page

• enable-port-forward SERVICE ...
-- Starts port-forwarding on each provided named service
-- Returns the status of each service, with either the port it's forwarded on, or a failure note with a message

• disable-port-forward SERVICE ...
-- cancels forwarding to the named services

For Windows only:

• list-integrations
-- gives a list of all the WSL integrations, with their status

• enable-integration NAME
• disable-integration NAME

[jandubois]

set NAME=VALUE ...
-- Updates the specified names, and runs the appropriate kind of restart (fast or slow) at the end if necessary

get NAME...
-- Returns the value of the specified settings

list-settings
-- Returns a list of all the settings with their current values

Shouldn't this be a variant of get, either without arguments, or get --all?

(After going through the whole set of commands, I'm no longer sure; maybe having a list-settings command is fine too, to go along with the other list-* commands).

SETTING NAMES:

• kubernetes-version | k8s-version -- /v?\d+.\d+.\d+/ - 'v' is optional, require all three parts.

Not really a fan of adding aliases at this point. We should have one canonical name, and stick with it. It should align closely with the name used in the UI, so users can map them without referring to documentation.

• kubernetes-enabled | k8s-enabled -- value is 'true' or 'false' (ignoring case).

• VM-memory | memory | -- Amount in GB. Suffixes of "GB" and "GiB" are supported and treated the usual way. The default suffix is GiB (1024 ** 3). Must be numeric. Not supported on Windows. If the supplied number is too large an error message is given with the maximum supported size.

Should also support "MB" and "MiB" units to specify e.g. 2½GB. I assume "numeric" means "integer", so don't expect 2.5GB to work.

• number-CPUs -- number of CPUs to assign the VM. Must be numeric. Not supported on Windows. If the supplied number is too large an error message is given with the maximum supported count.

I prefer cpu and memory as the names.

I think all names should be lower-case and separate words with dashes, like kubernetes-version.

• port -- port to run the Kubernetes API on. Must be numeric.

Maybe kubernetes-api-port to be more specific?

• container-engine - supported values are 'moby' and 'containerd'.

I would say containerd and dockerd, but moby silently accepted as an alias for dockerd (because we use moby in the UI).

• traefik -- value is 'true' or 'false' (ignoring case).

Should be kubernetes-traefik-enabled. We should group all kubernetes options under a kubernetes- prefix. And we should consistently use the -enabled suffix, so this option matches kubernetes-enabled.

I can't think through implications for services and ports right now. I think we can postpone this; I don't expect those commands to be supported in the first release:

list-services [--exclude-kubernetes | -k] [--forwarded-only | -f]

• Lists all the services that are currently running, similar to the Port Forwarding page

enable-port-forward SERVICE ...
-- Starts port-forwarding on each provided named service
-- Returns the status of each service, with either the port it's forwarded on, or a failure note with a message

disable-port-forward SERVICE ...
-- cancels forwarding to the named services

For Windows only:

• list-integrations
-- gives a list of all the WSL integrations, with their status

• enable-integration NAME
• disable-integration NAME

ssh [command]
-- Mac and Linux only: ssh's into the VM (just a wrapper around the LIMA_HOME... limactl shell 0 dance)

Why not for Windows (just a wrapper for wsl -d rancher-desktop)?

Might be useful for creating cross-platform scripts that run commands inside the VM.

restart-kubernetes [--clear-images]

I would find --delete-images clearer, but that still doesn't make it obvious that it will also delete workloads.

restart-vm [--clear-images]

shutdown
-- Shuts down the UI

I assume it also shuts down the VM and everything, not just the window.

factory-reset
-- Shuts down the UI and clears everything

GLOBAL OPTIONS:
• --debug|-d
• --verbose|-v
• --json-output
-- By default, output is human-readable. With --json-output, if there are no errors, the only output is the final JSON payload. The --json-output and --verbose options are mutually incompatible.

I think with --json-output (maybe just --json), the output should still be JSON (and go to stdout), even when an error ocurrs: {"error": "error message"} or something like that (together with a non-zero exit code).

Other notes:
• Error, warning, and debug messages are written to stderr. All other output goes to stdout.
• Exit status of 0 on success, 1 on usage errors, 2 on other errors.

I assume you will create a corresponding JSON schema to map these commands to a REST API.

[ericpromislow]

[github isn't great for discussion threads. I'll elide any uncontentious parts, and update the changes in the original Proposed Commands section to avoid having to piece together multiple comments into a while.

list-settings
-- Returns a list of all the settings with their current values

Shouldn't this be a variant of get, either without arguments, or get --all?

(After going through the whole set of commands, I'm no longer sure; maybe having a list-settings command is fine
too, to go along with the other list-* commands).

Actually I'm fine with get --all.

• kubernetes-enabled | k8s-enabled -- value is 'true' or 'false' (ignoring case).
• VM-memory | memory | -- Amount in GB. Suffixes of "GB" and "GiB" are supported and treated the usual way. The default suffix is GiB (1024 ** 3). Must be numeric. Can have a decimal point with values to the right. If the supplied number is too large an error message is given with the maximum supported size. Not supported on Windows.

Should also support "MB" and "MiB" units to specify e.g. 2½GB. I assume "numeric" means "integer", so don't expect 2.5GB to work.

Actually, why not?

[... other name-based suggestions pulled into the main spec above]

ssh [command]
-- Mac and Linux only: ssh's into the VM (just a wrapper around the LIMA_HOME... limactl shell 0 dance)

Why not for Windows (just a wrapper for wsl -d rancher-desktop)?

Might be useful for creating cross-platform scripts that run commands inside the VM.

Yes, exactly. The default distro would be rancher-desktop for lack of a better default, but I imagine most of the time users will target their own distro.

restart-kubernetes [--clear-images]

I would find --delete-images clearer, but that still doesn't make it obvious that it will also delete workloads.

What about just --clear and users can read the docs (including --help) to find out exactly what's getting cleared?

restart-vm [--clear-images]
shutdown
-- Shuts down the UI

I assume it also shuts down the VM and everything, not just the window.

Yes.

[...]

I assume you will create a corresponding JSON schema to map these commands to a REST API.

Yes, that's next.

[jandubois]

ssh [command]
-- Mac and Linux only: ssh's into the VM (just a wrapper around the LIMA_HOME... limactl shell 0 dance)

Why not for Windows (just a wrapper for wsl -d rancher-desktop)?
Might be useful for creating cross-platform scripts that run commands inside the VM.

Yes, exactly. The default distro would be rancher-desktop for lack of a better default, but I imagine most of the time users will target their own distro.

I don't understand. rd ssh would always open a shell inside the rancher-desktop VM. Why would we ever support any other name? You can just use wsl directly.

I just want rd ssh uname -a do the same thing on Linux, macOS, and Windows.

[ericpromislow]

Right, that makes sense. I was thinking of my own usage, where I typically use our default VM on macos, and ignore the rancher-desktop VM on Windows (most of the time).

[jandubois]

This could be done by rd alias or something else.

I just realized that Windows has an rd command to delete (empty) directories. I think we should call it rdctl. The user can still create an rd alias, if they feel comfortable with it.

[jandubois]

Further commands I want to see:

• rdctl start --cpu 4

  This will use the platform-native command to start the app with the required parameters, e.g. (macOS):

  open -a "Rancher Desktop" --args --cpu 4
  

  If the app is already running, it would just change the settings, i.e. be equivalent to

  rdctl set --cpu 4
  

  It should be possible to start RD this way without displaying the "first-run" dialog if all the required configuration settings are provided by the start command.

• rdctl status

  Returns the current status of the app and system-services. Details TBD; might replace [UI] Improve App background statuses feedback #1217.

• rdctl wait

  Can be used to wait for a specific state to be reached (essentially just rdctl status in a loop with a timeout).

So you can do something like this (syntax made up):

rdctl start --container-runtime dockerd --kubernetes-version 1.23.2
rdctl wait apiserver
kubectl ...
rdctl shutdown

[jandubois]

Should also support "MB" and "MiB" units to specify e.g. 2½GB. I assume "numeric" means "integer", so don't expect 2.5GB to work.

Actually, why not?

Yeah, maybe it is fine and just a personal dislike on my part. As long as we offer "MB" and "MiB" units for people who feel the same as me. 😄

Intuitively it feels wrong to use floating point for something that has indivisable base units (you cannot have fractional bytes). Just like using floating point for monetary units invariably ends with tears.

[ericpromislow]

Work on this story is being done through #1748. This is a good place to keep discussing the specs, and report large-grained progress.

[mook-as]

(Making feedback on the version as of May 4, 3:23 PM; sorry, I thought I committed the comment…)

set NAME=VALUE ...
-- Updates the specified names, and runs the appropriate kind of restart (fast or slow) at the end if necessary

Presumably, if the machine isn't running, we don't restart?

It might be better to never automatically restart, and add a status needs-restart or something to check if it's needed; otherwise modifying multiple settings that individually need to restart can get pretty painful. Alternatively, have a --no-restart or something the user can supply to suppress this behaviour.

ssh [-d DISTRO=rancher-desktop]* [command]

(This comment duplicates Jan's; removed.)

restart-kubernetes [--clear]

Does this return an error if Kubernetes is disabled? Or does it start it (toggling the pref)?

Presumably, --clear deletes all images? In the containerd case, does this delete images outside the k8s.io namespace?

[jandubois]

More commands I would like to see:

• rdctl api (see https://cli.github.com/manual/gh_api for inspiration)

• rdctl debug-report (create a tarball of system info, log and config files)

[ericpromislow]

Proposal: Support channel names for rdctl set --version NAME , including stable, latest, and any other recognized channel names (like v1.21 for the selected v1.21.PATCH version.

[jandubois]

I'm going to close this issue now, as it mostly serves as a collection of things we eventually want to expose via the CLI.

The 1.2 release includes the work from #1748, and the next release will include the issues tracked by #1896. The will be additional work coming after that, but there is no concrete action item associated with this issue any more.