CDK Tips

Introduction

CDK and Minishift are a nice, portable way of running a single-node OCP/OKD cluster on your development workstation. I collected some useful tips for an easier life with them.

What I assume

I assume that you are:

• proficient in using the bash shell
• understand the basic of container mechanics such as:
  • what is an image
  • why do you need a registry
• know how to work with containers using docker or podman
• have used CDK or Minishift before

Various CDK (Minishift) Tips

First off: I shall drop the Minishift term here (unless where absolutely necessary) and regard CDK as an alias for Minishift.

Configuration / Start Up

Useful Startup Options

Sometimes you need to figure out what the hell went wrong and why CDK is acting up on you. Try these:

$ cdk start --alsologtostderr --show-libmachine-logs -v 3

This will bump the log level up to debug and be very noisy on the console, but at least you'll get some info about what's cooking.

I personally tend to use these a lot when I'm playing around with some low-level settings and I need to see which components are affected by my changes.

Offline Use

Because I spend way too much time in airplanes and other transport modalities where internet is more or less absent, I've had a jolly fun time trying to make CDK work offline.

One of these days I'll write a howto on getting CDK to work fully offline, with a local registry VM, a Gogs instance, and working DNS resolution.

Until then, there are some simple tricks that should work, provided you have all the images you need in local cache and all the hostnames you need in /etc/hosts.

Some of the interesting options:

--skip-registration

(CDK only) This will not attempt to register your VM in the Red Hat Customer Portal.

--skip-registry-check

This will skip the test for online registry availability, but will fail horribly somewhere down the line, unless you're certain you really have all the platform images for your version of OCP/OKD.

--skip-startup-checks

This will skip all other start-up checks (such as image versions, oc client availability, etc.) not only the online ones. Make sure your CDK is in good shape by running the startup checks at least once before starting to skip them.

Have a look at cdk config to see the list of individual checks you can skip (look for the skip-check pattern):

$ cdk config | grep skip-check
 * skip-check-deprecation
 * skip-check-kvm-driver
 * skip-check-xhyve-driver
 * skip-check-hyperv-driver
 * skip-check-iso-url
 * skip-check-vm-driver
 * skip-check-vbox-installed
 * skip-check-openshift-version
 * skip-check-openshift-release
 * skip-check-clusterup-flags
 * skip-check-instance-ip
 * skip-check-network-host
 * skip-check-network-ping
 * skip-check-network-http
 * skip-check-storage-mount
 * skip-check-storage-usage
 * skip-check-nameservers

NOTE: Unfortunately, although they are listed as configuration options in cdk config output, these are ignored if you set them using cdk config set.

Administration

CDK Administration

TBD.

Cluster Administration

Sometimes you need to do something as system:admin, not just any cluster admin.

For example: in the CDK 3.6 / OCP 3.11 combo, the admin addon fails for some reason, which means you end up with no remote cluster admin capability. What now?

• First, get a shell in the origin container inside the boot2docker VM:

$ cdk ssh
Last login: Fri Nov 16 06:25:09 2018 from gateway
[docker@minishift ~]$ docker exec -it origin sh
sh-4.2#

• Notice that the oc command is not configured to load auth data by default:

sh-4.2# oc whoami
error: Missing or incomplete configuration info.  Please login or point to an existing, complete config file:

  1. Via the command-line flag --config
  2. Via the KUBECONFIG environment variable
  3. In your home directory as ~/.kube/config

To view or setup config directly use the 'config' command.

• Next, run the oc command with the --config option, telling it where to find the kubeconfig file with system:admin credentials:

sh-4.2# oc --config=./openshift.local.config/master/admin.kubeconfig whoami
system:admin

• Then, depending on how long you intend to spend in the VM, alias the oc command:

sh-4.2# alias oc="oc --config=./openshift.local.config/master/admin.kubeconfig"

Alternatively, you could set the KUBECONFIG env variable, of course.

• Then simply do your stuff!

sh-4.2# oc describe clusterrolebinding cluster-admin
Name:         cluster-admin
Labels:       kubernetes.io/bootstrapping=rbac-defaults
Annotations:  rbac.authorization.kubernetes.io/autoupdate=true
Role:
  Kind:  ClusterRole
  Name:  cluster-admin
Subjects:
  Kind   Name            Namespace
  ----   ----            ---------
  Group  system:masters  
sh-4.2# oc get users
NAME        UID                                    FULL NAME   IDENTITIES
admin       6f98e840-e990-11e8-8676-16ea592bfedd               anypassword:admin
developer   d482a716-e98f-11e8-8676-16ea592bfedd               anypassword:developer
sh-4.2# oc adm policy add-cluster-role-to-user cluster-admin admin
cluster role "cluster-admin" added: "admin"

Local Shell Environment

Bash Completion for CDK

Enabling

Running cdk completion bash will output a bash completion recipe.

You can then place it into /etc/bash_completion.d (or wherever your shell expects it).

(hint: on macOS, if using MacPorts, this is /opt/local/etc/bash_completion.d)

CDK vs Minishift

One caveat is that if you're using cdk rather than minishift, you'll have to replace (almost) everything in that file that says minishift, with cdk (because --minishift-home is a valid option for both).

So, a shortcut to getting it to work with cdk in a single pipeline would be:

$ cdk completion bash | sed 's/minishift/cdk/g; s/-cdk-/-minishift-/g' > cdk
$ sudo mv cdk /etc/bash_completion.d/
$ . /etc/bash_completion.d/cdk
$ cdk <TAB><TAB>
addons      config      delete      docker-env  image       logs        openshift   setup-cdk   start       stop        
completion  console     dns         hostfolder  ip          oc-env      profile     ssh         status      version     

Voila!