Docker Desktop's kubectl interferes with Hokusai's

[artsyjian]

There has been cases where a Mac user's PATH points to the kubectl [installed by Docker Desktop](https://docs.docker.com/desktop/kubernetes/), and not to /usr/local/bin/kubectl (installed by Hokusai). So far we've seen this for M1 users.

In one case, Docker Desktop's kubectl lives in ~/.docker/... and this dir is listed at the very beginning of user's PATH. The user's Shell init script (in this case, for zsh) has been modified by Docker Desktop to source a file which presumably did that to the PATH. The work-around for this specific case is to comment out the source or add a line in the Shell init script toward the bottom to undo that PATH change, but this is cumbersome and might not survive the next Docker Desktop install/update.

I am not sure what Hokusai can do to totally solve the problem, but some ideas for mitigation:

• Currently hokusai check simply checks that which kubectl returns no error. We can have it read the returned path and warn if it's not /usr/local/bin, which can help user notice a path problem.

• Currently hokusai configure copies the downloaded kubectl to /usr/local/bin/kubectl. Perhaps it's feasible to name the executable differently in /usr/local/bin, for example, hokusai_kubectl, then there should no longer be any conflict with Docker Desktop. The problem is that if user ever has to run kubectl directly, the user must know about hokusai_kubectl.

[artsyjian]

More ideas:

• hokusai check can check the version of the kubectl binary, wherever that binary is stored. If the version matches that of the Kubernetes cluster, then there's no problem, otherwise, fail the check. So we do not care about whether the kubectl binary was installed by Hokusai or Docker Desktop, we only care about the version.

• Change hokusai configure's default directory for installing kubectl, to ~/.local/bin. It seems a standard practice to house user installed executables there. When Hokusai invokes kubectl, it specifies the full path: ~/.local/bin/kubectl.

[artsyjian]

Possibly another occurrence. In this case it's also an M1 user, but there's no ~/.docker/bin/kubectl. Hokusai commands resulted in:

error: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1alpha1"

I can confirm on my local Mac (Intel) that latest Hokusai version doesn't produce that error, but ~/.docker/bin/kubectl does. So it would seem that in this case the user's Docker Desktop overwrote /usr/local/bin/kubectl. When the user ran hokusai configure ..., the problem resolved, likely because Hokusai's installing kubectl to /usr/local/bin overwrote Docker Desktop's. So this would be an example of kubectl conflict not involving PATH.

[artsyjian]

Here's one case where a user's /usr/bin/local/kubectl is aliased to Docker Desktop's, revealed by a hokusai configure...

...
Downloading and installing kubectl...

Traceback (most recent call last):
  File "shutil.py", line 815, in move
PermissionError: [Errno 13] Permission denied: '/var/folders/pd/n6bf48ls3q1203_l6ybl7myr0000gq/T/tmp8yan3zwe/kubectl' -> '/usr/local/bin/kubectl'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "hokusai/lib/command.py", line 17, in wrapper
  File "hokusai/commands/configure.py", line 41, in configure
  File "shutil.py", line 835, in move
  File "shutil.py", line 444, in copy2
  File "shutil.py", line 266, in copyfile
PermissionError: [Errno 1] Operation not permitted: '/usr/local/bin/kubectl'

13:48:15
jon@sabretooth:~/code/gravity(main)% ll /usr/local/bin/kubectl                                                                                                                                                                                                                                                            {1}
lrwxr-xr-x@ 1 root  wheel  55 Feb 16 13:23 /usr/local/bin/kubectl@ -> /Applications/Docker.app/Contents/Resources/bin/kubectl

[artsyjian]

Given that we have seen Docker Desktop causing kubectl conflict in both ways:

• Changing PATH to give preference to ~/.docker/bin/kubectl
• Overwriting /usr/local/bin/kubectl

We might have to have Hokusai invoke kubectl using full path (thus safe from PATH problem) and that path be different from /usr/local/bin/kubectl (e.g. ~/.local/bin/kubectl).

How is that path determined?

hokusai configure takes an --install-to option:

$ hokusai configure --help
Usage: hokusai configure [OPTIONS]

  Install and configure kubectl

Options:
  --kubectl-version TEXT     The version of kubectl to install
  --s3-bucket TEXT           The S3 bucket name containing your org's kubectl config file
  --s3-key TEXT              The S3 key of your org's kubectl config file
  --config_file TEXT         Your org's kubectl config file
  --platform [darwin|linux]  The platform OS (default: darwin)
  --install-to TEXT          Install kubectl to (default: /usr/local/bin)
  --install-config-to TEXT   Install kubectl config to (default: ~/.kube)
  -v, --verbose              Verbose output
  -h, --help                 Show this message and exit.

So at kubectl-invoke time, Hokusai can use the same path that was used for that option. This is currently not possible because Hokusai does not keep track of that option's value. A way is required for Hokusai to "remember" that value.

We can accomplish that by introducing a user Hokusai config file (e.g. ~/.hokusai.conf) which specifies install-to:

# hokusai.conf

install-to=~/.local/bin

Then hokusai configure and kubectl-invoking code can just refer to that file, and hokusai configure's --install-to option should no longer be required.

[artsyjian]

More about the potential ~/.hokusai.conf. Do we require it to exist when user runs hokusai configure? It seems we should not, as the user would have to manually create it, which is tedious. Two cases at hokusai configure time:

• The conf file does not exist, meaning this is the first time hokusai configure is run, or the conf file did exist at one point but was deleted. Hokusai should create the file and populate it with the options (default or otherwise) that is being used by hokusai configure now (e.g. --install-to, --platform, ...).
• The conf file exists, meaning hokusai configure was run in the past and user is re-running it. The order of precedence then will be: 1) options explicitly specified by user's hokusai configure now, 2) conf file, 3) hokusai configure's default values for the options.