Go Report	Unit tests	License

One kubectl to rule them all,
one kubectl to find them,
One kubectl to bring them all
and in the darkness bind them.

Managing different kubernetes clusters often requires to keep multiple versions of the kubectl available on the system, plus it poses the challenge to ensure the right binary is used when talking with a cluster.

kubernetes defines a clear version skew policy for all its components. This is what is stated about kubectl:

kubectl is supported within one minor version (older or newer) of kube-apiserver.

Example:

kube-apiserver is at 1.18
kubectl is supported at 1.19, 1.18, and 1.17

kuberlr (kube-ruler) is a simple wrapper for kubectl. Its main purpose is to make it easy to manage clusters running different versions of kubernetes.

This is how kuberlr looks like in action:

kuberlr can run on Linux, macOS and Windows.

Installation

You can find pre-built binaries of kuberlr under the GitHub release tab.

Put the kuberlr binary somewhere in your PATH and create a symlink named kubectl pointing to it.

For example, assuming ~/bin has a high priority inside of your PATH:

$ cp kuberlr /bin/
$ ln -s ~/bin/kuberlr ~/bin/kubectl

Note: it's also possible to skip the creation of the symlink and use kuberlr kubectl instead.

Usage

Use the kubectl "fake binary" as you usually do. Behind the scene kuberlr will ensure a compatible version of kubectl is used.

You can invoke the kuberlr binary in a direct fashion to access its sub-commands. For example, the kuberlr bins will print all the kubectl binaries that are available to the user.

The kuberlr get sub-command downloads a specific kubectl version. When only a major.minor version is given (e.g. kuberlr get 1.36), kuberlr asks the upstream mirror for the most recent patch release of that release line (e.g. 1.36.4) and downloads it. A full major.minor.patch version (e.g. kuberlr get 1.36.0) downloads exactly that release instead. If the most recent patch release can't be determined (e.g. the mirror is unreachable, or the release line is unknown), kuberlr falls back to downloading patch release 0 of that release line.

The kuberlr update sub-command updates local kubectl binaries to the latest patch release.

The kuberlr rm sub-command removes local kubectl binaries. Give one version to remove one release, or a major.minor version to remove every release of that series. Use --prune to remove every release except the newest patch of each series. Use --all to remove every local release. Add --dry-run to see what a command would remove, without removing anything.

kuberlr rm only removes binaries that kuberlr downloaded. It never removes system-wide binaries, for example the ones in /usr/bin.

How it works

kuberlr connects to the API server of your kubernetes cluster and figures out its version.

kuberlr obtains the url of the kubernetes cluster either by looking at the ~/.kube/config file or by reading the contents of the file referenced by the KUBECONFIG environment variable.

Once the version of the remote server is know, kuberlr looks for a compatible kubectl binary under the ~/.kuberlr/<GOOS>-<GOARCH>/ directory and /usr/bin.

kuberlr reuses an already existing binary if it respects the kubectl version skew policy, otherwise it downloads the right one from the upstream mirror into the local user cache (~/.kuberlr/<GOOS>-<GOARCH>/).

kuberlr names the kubectl binaries it downloads using the following naming scheme: kubectl<major version>.<minor version>.<patch level>.

Finally kuberlr performs an execve(2) syscall and leaves the control to the kubectl binary. (٭)

Note well: by default kuberlr will download the missing kubectl binaries from the upstream mirror. This behaviour can be disabled via kuberlr's configuration file.

The execve syscall is not available on Windows. On this platform another approach is used, but the end result doesn't change. (٭)

Some kubectl commands never talk to the API server: config, completion, kuberc, plugin, kustomize, options, help, version --client and the hidden commands used by shell completion. kuberlr recognizes these commands by looking at the first argument and skips the remote version lookup for them. It runs the newest kubectl binary already available on the system instead. If a global flag comes before the command (for example kubectl --context foo config view), kuberlr uses the normal lookup.

Reusing system-wide kubectl binaries

As pointed above kuberlr looks for a compatible kubectl binary both at user level (~/.kuberlr/<GOOS>-<GOARCH>/) and at system level (/usr/bin).

The kubectl binaries installed at system level must respect one of these naming schemes in order to be used:

• kubectl<major version>.<minor version>.<patch level> (e.g.: kubectl1.18.3)
• kubectl<major version>.<minor version>: this would be handled as kubectl version <major version>.<minor version>.0

Output

kuberlr stays silent when everything it needs is already in place. When it has something to say, it writes to stderr one line at a time, prefixed with kuberlr:, so that its messages can be told apart from the ones of kubectl:

$ kubectl get pods
kuberlr: downloading kubectl1.31.4 from https://dl.k8s.io/release/v1.31.4/bin/linux/amd64/kubectl
kubectl1.31.4 100% |████████████████████████████████████████| (56/56 MB, 12 MB/s) done.
NAME                     READY   STATUS    RESTARTS   AGE
...

Warnings and errors are tagged as warning: and error:. When kuberlr can't find or download a suitable kubectl, it prints an error and exits with status 1.

Three settings control the output. They can be set in the configuration file or through environment variables, see Configuration:

• Verbosity (KUBERLR_VERBOSITY): 0 (default) shows informational messages, warnings and errors. 1 adds debug messages, for example why the version of the remote cluster couldn't be detected. 2 adds trace messages, including the ones of the kubernetes client library.
• Quiet (KUBERLR_QUIET): when true, only warnings and errors are shown and the download progress bar is hidden. Handy for scripts and CI.
• Color (KUBERLR_COLOR): auto (default) uses colors only when stderr is a terminal and the NO_COLOR environment variable is not set. always and never force the behaviour.

The kuberlr sub-commands also accept the -v/--verbosity, -q/--quiet and --color flags, which take precedence over the configuration file and the environment. The kubectl wrapper does not, because kubectl has flags with the same names: use the environment variables or the configuration file there.

The progress bar is never shown when stderr is not a terminal, e.g. when the output is redirected to a file.

Configuration

The behaviour of kuberlr can be adjusted by creating a configuration file in one of these locations:

1. /usr/etc/kuberlr.conf: this is the location used by distributions like openSUSE to handle the split between /etc and /usr/etc. You can find more details here.
2. /etc/kuberlr.conf
3. $HOME/.kuberlr/kuberlr.conf
4. $KUBERLR_CFG

The configuration files are read in the order written above and merged together. Configuration files can override the values defined by the previous ones, or provide new ones.

The configuration file is written using the TOML format:

# Allow the download of missing kubectl binaries from kubernetes' upstream mirror
AllowDownload = true

# When NO compatible local kubectl is found, opt-in to using the NEWEST local kubectl
# instead of failing in these cases:
#   1) downloads are disabled (AllowDownload = false), OR
#   2) a download attempt fails (e.g., no network, blocked mirror).
#
# Default: false
# WARNING: this may run a kubectl that is outside Kubernetes' version skew policy
# and could be incompatible with your API server. Use with care.
UseLatestIfNoCompatible = false

# Directory where kubectl binaries are made accessible to all the users of the system
# Default "/usr/bin"
SystemPath = "/usr/bin"

# Timeout (sec) for requests made against the kubernetes API
# Default 5
Timeout = 5

# URL of the upstream mirror where kubectl binaries can be downloaded from
# Default "https://dl.k8s.io"
KubeMirrorUrl = "https://dl.k8s.io"

# How much kuberlr says on stderr: 0 (default), 1 (debug) or 2 (trace)
Verbosity = 0

# Only show warnings and errors, hide the download progress bar
# Default false
Quiet = false

# When to use colors: "auto" (default), "always" or "never"
Color = "auto"

The behaviour can also be adjusted by using environment variables matching the config file:

Key	Default	ENV	Description
AllowDownload	true	KUBERLR_ALLOWDOWNLOAD	Whether kuberlr may download a compatible kubectl from the upstream mirror.
UseLatestIfNoCompatible	false	KUBERLR_USELATESTIFNOCOMPATIBLE	When no compatible local kubectl is found, use the newest local kubectl instead of failing if downloads are disabled or the download attempt fails.
SystemPath	/usr/bin	KUBERLR_SYSTEMPATH	Additional directory to scan for system-wide kubectl binaries.
KubeMirrorUrl	https://dl.k8s.io	KUBERLR_KUBEMIRRORURL	Custom upstream mirror for downloads.
Timeout	5	KUBERLR_TIMEOUT	Timeout (seconds) for contacting the API server to detect version.
Verbosity	0	KUBERLR_VERBOSITY	How much kuberlr says on stderr: 0, 1 (debug) or 2 (trace). See Output.
Quiet	false	KUBERLR_QUIET	Only show warnings and errors, hide the download progress bar.
Color	auto	KUBERLR_COLOR	When to use colors: auto, always or never.