Page tree

Michael O'Brien

Skip to end of metadata
Go to start of metadata

Kubernetes is the de-facto docker container orchestrator and control plane.  As of Aug 2019 VMWare is installing kubernetes as the control plane for VSphere.  Amazon EKS running fargate is the state of the art for serverless container managed services.  You can also run your own kubernetes cluster on any cloud or on premises systems using Rancher RKE. Kubernetes

KUBERNETES-1 - Getting issue details... STATUS

Quickstart


Installing kubectl - alternate

Kubectl will come with docker desktop - however if you are running on a constrained environment and can only reference a remote kubernetes cluster - install kubectl and helm manually

Manual kubectl install - windows

curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.18.0/bin/windows/amd64/kubectl.exe
# add the exe to your path



Get your public and private keys on the Ubuntu 16.04 VM.

Adjust authorized_keys with your public key if not already - aws has it, openstack may not

get rke script from jira, gerrit or by cloning https://github.com/obrienlabs/magellan.git for https://github.com/obrienlabs/magellan/blob/master/kubernetes/rke_setup.sh or directly https://github.com/rancher/rke/releases


Versions

Kubernetes 1.16 is out https://kubernetes.io/blog/2019/09/18/kubernetes-1-16-release-announcement/ - as of 20200511 Docker Desktop 2.3.0.2 supports Kubernetes 1.16.5

InstallerKubernetesHelmDockerGoReleased
OKD 3/4




RKE 1.1.31.18.3



RKE 1.0.81.17.53.2.119.0.3.81.13.9202005
RKE 0.121.14.62.14.319.03.2

Docker Desktop OSX 2.2.0.51.15.53.1.019.03.51.13.8
Docker Desktop OSX 2.3.0.21.16.53.2.119.03.8
20200511

RKE Installations

Verify RKE ports in https://rancher.com/docs/rke/latest/en/os/

Manual Installation of RKE - Rancher Kubernetes Engine - see  Kubernetes RKE Cluster on 4 Intel NUC machines with 64G RAM

or use my automated script Quickstart


Install Docker

sudo curl https://releases.rancher.com/install-docker/19.03.sh | sh
sudo usermod -aG docker <user>


Single Node Kubernetes cluster running RKE on AWS EC2 with Helm


Private SSH key

scp your public key to the box - ideally to ~/.ssh and chmod 400 it - make sure you add your key to authorized_keys

Elastic Reserved IP

get a VIP or EIP and assign this to your VM

generate cluster.yml - optional

cluster.yml will generated by the script rke_setup.sh

azure config - no need to hand build the yml
Watch the path of your 2 keys
Also don't add an "addon" until you have one of the config job will fail

ubuntu@a-rke:~$ rke config --name cluster.yml

# use the updated Kubernetes 1.14.6 cluster.yml in the rke_setup.sh script

Setup SSH key access

see Developer Guide#Linux-Ubuntu16.04/18.04

# on your laptop/where your cert is
# chmod 777 your cert before you scp it over
scp ~/wse/onap_rsa ubuntu@kub0:~/

# on the host
sudo mkdir ~/.ssh
sudo cp onap_rsa ~/.ssh
sudo chmod 400 ~/.ssh/onap_rsa
sudo chown ubuntu:ubuntu ~/.ssh/onap_rsa 

# on the target - add the public key to authorized_keys if not already associated with the VM
$ cat ~/.ssh/onap_rsa.pub
ssh-rsa AAAAB3N......trics
ubuntu@ubuntu:~$ sudo vi ~/.ssh/authorized_keys

# login from another host
$ ssh -i ~/.ssh/onap_rsa ubuntu@192.168.20.137
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-62-generic x86_64)
Last login: Mon Dec 30 08:14:43 2019 from 192.168.20.137

# back on the target - check for the session
ubuntu@ubuntu:~$ who
ubuntu   tty7         2019-11-01 04:47 (:0)
ubuntu   pts/1        2019-12-30 08:17 (192.168.20.137)

# fix the VM if required
sudo nano /etc/apt/sources.list
# remove any "deb cdrom:


Disable Password Authentication

see Developer Guide#Linux-Ubuntu16.04/18.04

ubuntu@ubuntu:~$ sudo vi /etc/ssh/sshd_config 
PasswordAuthentication no


Install Kubernetes RKE script

# this test on a VMWare VM on OSX
git clone --recurse-submodules https://github.com/obrienlabs/magellan.git
cd magellan/kubernetes
#chmod 777 rke_setup.sh

amdocs@obriensystemsu0:~/magellan/kubernetes$ sudo ./rke_setup.sh -b master -s 172.16.173.130 -e obl -k onap_rsa -l ubuntu
please supply your ssh key as provided by the -k keyname - it must be be chmod 400 and chown user:user in ~/.ssh/
The RKE version specific cluster.yaml is already integrated in this script for 0.2.8 no need for below generation...
rke config --name cluster.yml
specifically
9address: 172.16.173.130
user: ubuntu
ssh_key_path: /home/ubuntu/.ssh/onap_rsa
Installing on 172.16.173.130 for master: RKE: 0.2.8 Kubectl: 1.14.6 Helm: 2.14.3 Docker: 19.03.2 username: ubuntu
Install docker - If you must install as non-root - comment out the docker install below - run it separately, run the user mod, logout/login and continue this script
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 15429  100 15429    0     0  22864      0 --:--:-- --:--:-- --:--:-- 22891
+ sh -c apt-get update
Hit:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB]                                                          
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [109 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB]                                     
Fetched 325 kB in 0s (487 kB/s)                                    
Reading package lists... Done
+ sh -c apt-get install -y -q apt-transport-https ca-certificates curl software-properties-common
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  libcurl3-gnutls python3-software-properties software-properties-gtk
The following packages will be upgraded:
  apt-transport-https ca-certificates curl libcurl3-gnutls python3-software-properties software-properties-common software-properties-gtk
7 upgraded, 0 newly installed, 0 to remove and 597 not upgraded.
Need to get 593 kB of archives.
After this operation, 55.3 kB disk space will be freed.
Get:1 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 curl amd64 7.47.0-1ubuntu2.14 [139 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libcurl3-gnutls amd64 7.47.0-1ubuntu2.14 [184 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 apt-transport-https amd64 1.2.32 [26.5 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 ca-certificates all 20170717~16.04.2 [167 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 software-properties-common all 0.96.20.9 [9,452 B]
Get:6 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 software-properties-gtk all 0.96.20.9 [47.2 kB]
Get:7 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 python3-software-properties all 0.96.20.9 [20.1 kB]
Fetched 593 kB in 1s (436 kB/s)
Preconfiguring packages ...
(Reading database ... 182650 files and directories currently installed.)
Preparing to unpack .../curl_7.47.0-1ubuntu2.14_amd64.deb ...
Unpacking curl (7.47.0-1ubuntu2.14) over (7.47.0-1ubuntu2.2) ...
Preparing to unpack .../libcurl3-gnutls_7.47.0-1ubuntu2.14_amd64.deb ...
Unpacking libcurl3-gnutls:amd64 (7.47.0-1ubuntu2.14) over (7.47.0-1ubuntu2.2) ...
Preparing to unpack .../apt-transport-https_1.2.32_amd64.deb ...
Unpacking apt-transport-https (1.2.32) over (1.2.24) ...
Preparing to unpack .../ca-certificates_20170717~16.04.2_all.deb ...
Unpacking ca-certificates (20170717~16.04.2) over (20160104ubuntu1) ...
Preparing to unpack .../software-properties-common_0.96.20.9_all.deb ...
Unpacking software-properties-common (0.96.20.9) over (0.96.20.7) ...
Preparing to unpack .../software-properties-gtk_0.96.20.9_all.deb ...
Unpacking software-properties-gtk (0.96.20.9) over (0.96.20.7) ...
Preparing to unpack .../python3-software-properties_0.96.20.9_all.deb ...
Unpacking python3-software-properties (0.96.20.9) over (0.96.20.7) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Processing triggers for dbus (1.10.6-1ubuntu3.3) ...
Processing triggers for hicolor-icon-theme (0.15-0ubuntu1) ...
Processing triggers for shared-mime-info (1.5-2ubuntu0.1) ...
Processing triggers for gnome-menus (3.13.3-6ubuntu3.1) ...
Processing triggers for desktop-file-utils (0.22-1ubuntu5.1) ...
Processing triggers for bamfdaemon (0.5.3~bzr0+16.04.20160824-0ubuntu1) ...
Rebuilding /usr/share/applications/bamf-2.index...
Processing triggers for mime-support (3.59ubuntu1) ...
Setting up libcurl3-gnutls:amd64 (7.47.0-1ubuntu2.14) ...
Setting up curl (7.47.0-1ubuntu2.14) ...
Setting up apt-transport-https (1.2.32) ...
Setting up ca-certificates (20170717~16.04.2) ...
Setting up python3-software-properties (0.96.20.9) ...
Setting up software-properties-common (0.96.20.9) ...
Setting up software-properties-gtk (0.96.20.9) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Processing triggers for ca-certificates (20170717~16.04.2) ...
Updating certificates in /etc/ssl/certs...
17 added, 42 removed; done.
Running hooks in /etc/ca-certificates/update.d...

Removing debian:WoSign.pem
done.
done.
+ curl -fsSl https://download.docker.com/linux/ubuntu/gpg
+ sh -c apt-key add -
OK
+ sh -c add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable"
+ [ ubuntu = debian ]
+ sh -c apt-get update
Get:1 https://download.docker.com/linux/ubuntu xenial InRelease [66.2 kB]
...                                     

+ sh -c apt-get install -y -q docker-ce=5:19.03.2~3-0~ubuntu-xenial
...
1 upgraded, 6 newly installed, 0 to remove and 596 not upgraded.
Need to get 87.8 MB of archives.
After this operation, 390 MB of additional disk space will be used.
Get:1 https://download.docker.com/linux/ubuntu xenial/stable amd64 containerd.io amd64 1.2.6-3 [22.6 MB]
...
Processing triggers for ureadahead (0.100.0-19) ...
+ sh -c docker version
Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfc
 Built:             Thu Aug 29 05:28:19 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.8
  Git commit:       6a30dfc
  Built:            Thu Aug 29 05:26:54 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

If you would like to use Docker as a non-root user, you should now consider
adding your user to the "docker" group with something like:
  sudo usermod -aG docker your-user
Remember that you will have to log out and back in for this to take effect!
...
Install RKE
--2019-09-25 21:03:10--  https://github.com/rancher/rke/releases/download/v0.2.8/rke_linux-amd64
Resolving github.com (github.com)... 140.82.113.3
Connecting to github.com (github.com)|140.82.113.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/108337180/839f6f80-c343-11e9-9c3c-49c76b856e47?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190926%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190926T020311Z&X-Amz-Expires=300&X-Amz-Signature=5a94946dcef52d35177ee4b2eba8cb8e5cf58c0f9251cf41ce8a8bf96e06ce00&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Drke_linux-amd64&response-content-type=application%2Foctet-stream [following]
--2019-09-25 21:03:11--  https://github-production-release-asset-2e65be.s3.amazonaws.com/108337180/839f6f80-c343-11e9-9c3c-49c76b856e47?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190926%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190926T020311Z&X-Amz-Expires=300&X-Amz-Signature=5a94946dcef52d35177ee4b2eba8cb8e5cf58c0f9251cf41ce8a8bf96e06ce00&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Drke_linux-amd64&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.128.11
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.128.11|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 40394065 (39M) [application/octet-stream]
Saving to: ‘rke_linux-amd64’

rke_linux-amd64                                        100%[============================================================================================================================>]  38.52M  15.5MB/s    in 2.5s    

2019-09-25 21:03:13 (15.5 MB/s) - ‘rke_linux-amd64’ saved [40394065/40394065]

Install make - required for beijing+ - installed via yum groupinstall Development Tools in RHEL
E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 41.1M  100 41.1M    0     0  26.5M      0  0:00:01  0:00:01 --:--:-- 26.5M
--2019-09-25 21:03:15--  http://storage.googleapis.com/kubernetes-helm/helm-v2.14.3-linux-amd64.tar.gz
Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.1.16, 2607:f8b0:400b:801::2010
Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.1.16|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 26533763 (25M) [application/x-tar]
Saving to: ‘helm-v2.14.3-linux-amd64.tar.gz’

helm-v2.14.3-linux-amd64.tar.gz                        100%[============================================================================================================================>]  25.30M  38.1MB/s    in 0.7s    

2019-09-25 21:03:16 (38.1 MB/s) - ‘helm-v2.14.3-linux-amd64.tar.gz’ saved [26533763/26533763]

linux-amd64/
linux-amd64/helm
linux-amd64/README.md
linux-amd64/LICENSE
linux-amd64/tiller
Bringing RKE up - using supplied cluster.yml
INFO[0000] Initiating Kubernetes cluster                
INFO[0000] [dialer] Setup tunnel for host [172.16.173.130] 
INFO[0000] [state] Pulling image [rancher/rke-tools:v0.1.42] on host [172.16.173.130] 
INFO[0005] [state] Successfully pulled image [rancher/rke-tools:v0.1.42] on host [172.16.173.130] 
INFO[0006] [state] Successfully started [cluster-state-deployer] container on host [172.16.173.130] 
INFO[0006] [certificates] Generating CA kubernetes certificates 
INFO[0006] [certificates] Generating Kubernetes API server aggregation layer requestheader client CA certificates 
INFO[0006] [certificates] Generating Kubernetes API server certificates 
INFO[0006] [certificates] Generating Service account token key 
INFO[0006] [certificates] Generating Kubernetes API server proxy client certificates 
INFO[0007] [certificates] Generating etcd-172.16.173.130 certificate and key 
INFO[0007] [certificates] Generating Kube Controller certificates 
INFO[0007] [certificates] Generating Kube Scheduler certificates 
INFO[0007] [certificates] Generating Kube Proxy certificates 
INFO[0007] [certificates] Generating Node certificate   
INFO[0007] [certificates] Generating admin certificates and kubeconfig 
INFO[0008] Successfully Deployed state file at [./cluster.rkestate] 
INFO[0008] Building Kubernetes cluster                  
...
INFO[0111] Finished building Kubernetes cluster successfully 
wait 2 extra min for the cluster
1 more min

copy kube_config_cluter.yaml generated - to ~/.kube/config
Verify all pods up on the kubernetes system - will return localhost:8080 until a host is added
kubectl get pods --all-namespaces
NAMESPACE       NAME                                      READY   STATUS      RESTARTS   AGE
ingress-nginx   default-http-backend-5954bd5d8c-tqmrq     1/1     Running     0          2m19s
ingress-nginx   nginx-ingress-controller-qfr48            1/1     Running     0          2m17s
kube-system     canal-fdp4m                               2/2     Running     0          2m34s
kube-system     coredns-autoscaler-5d5d49b8ff-tnnjh       1/1     Running     0          2m27s
kube-system     coredns-bdffbc666-zmqzg                   1/1     Running     0          2m28s
kube-system     metrics-server-7f6bd4c888-6xqkp           1/1     Running     0          2m22s
kube-system     rke-coredns-addon-deploy-job-jbnhb        0/1     Completed   0          2m30s
kube-system     rke-ingress-controller-deploy-job-m9xdp   0/1     Completed   0          2m20s
kube-system     rke-metrics-addon-deploy-job-pbt2t        0/1     Completed   0          2m25s
kube-system     rke-network-plugin-deploy-job-5z2vg       0/1     Completed   0          2m43s
install tiller/helm
serviceaccount/tiller created
clusterrolebinding.rbac.authorization.k8s.io/tiller created
Creating /home/amdocs/.helm 
Creating /home/amdocs/.helm/repository 
Creating /home/amdocs/.helm/repository/cache 
Creating /home/amdocs/.helm/repository/local 
Creating /home/amdocs/.helm/plugins 
Creating /home/amdocs/.helm/starters 
Creating /home/amdocs/.helm/cache/archive 
Creating /home/amdocs/.helm/repository/repositories.yaml 
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com 
Adding local repo with URL: http://127.0.0.1:8879/charts 
$HELM_HOME has been configured at /home/amdocs/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.

Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Waiting for deployment "tiller-deploy" rollout to finish: 0 of 1 updated replicas are available...
deployment "tiller-deploy" successfully rolled out
upgrade server side of helm in kubernetes
Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
sleep 30
$HELM_HOME has been configured at /home/amdocs/.helm.

Tiller (the Helm server-side component) has been upgraded to the current version.
sleep 30
verify both versions are the same below
Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
start helm server
sleep 30
Regenerating index. This may take a moment.
Now serving you on 127.0.0.1:8879
add local helm repo
"local" has been added to your repositories
NAME  	URL                                             
stable	https://kubernetes-charts.storage.googleapis.com
local 	http://127.0.0.1:8879                           
To enable grafana dashboard - do this after running cd.sh which brings up onap - or you may get a 302xx port conflict
kubectl expose -n kube-system deployment monitoring-grafana --type=LoadBalancer --name monitoring-grafana-client
to get the nodeport for a specific VM running grafana
kubectl get services --all-namespaces | grep graf
Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfc
 Built:             Thu Aug 29 05:28:19 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.8
  Git commit:       6a30dfc
  Built:            Thu Aug 29 05:26:54 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.6", GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:49Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.6", GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:16Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
NAMESPACE       NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE
default         kubernetes             ClusterIP   10.43.0.1      <none>        443/TCP                  5m3s
ingress-nginx   default-http-backend   ClusterIP   10.43.151.5    <none>        80/TCP                   4m9s
kube-system     kube-dns               ClusterIP   10.43.0.10     <none>        53/UDP,53/TCP,9153/TCP   4m18s
kube-system     metrics-server         ClusterIP   10.43.71.16    <none>        443/TCP                  4m13s
kube-system     tiller-deploy          ClusterIP   10.43.194.75   <none>        44134/TCP                108s
NAMESPACE       NAME                                      READY   STATUS      RESTARTS   AGE
ingress-nginx   default-http-backend-5954bd5d8c-tqmrq     1/1     Running     0          4m9s
ingress-nginx   nginx-ingress-controller-qfr48            1/1     Running     0          4m7s
kube-system     canal-fdp4m                               2/2     Running     0          4m24s
kube-system     coredns-autoscaler-5d5d49b8ff-tnnjh       1/1     Running     0          4m17s
kube-system     coredns-bdffbc666-zmqzg                   1/1     Running     0          4m18s
kube-system     metrics-server-7f6bd4c888-6xqkp           1/1     Running     0          4m12s
kube-system     rke-coredns-addon-deploy-job-jbnhb        0/1     Completed   0          4m20s
kube-system     rke-ingress-controller-deploy-job-m9xdp   0/1     Completed   0          4m10s
kube-system     rke-metrics-addon-deploy-job-pbt2t        0/1     Completed   0          4m15s
kube-system     rke-network-plugin-deploy-job-5z2vg       0/1     Completed   0          4m33s
kube-system     tiller-deploy-7f4d76c4b6-nnts8            1/1     Running     0          108s
finished!
amdocs@obriensystemsu0:~/magellan/kubernetes$ 


Manual Installation of Kubernetes via RKE on Ubuntu 16.04 VM - optional

Determine RKE and Docker versions

Don't just use the latest docker version - check the RKE release page to get the version pair - 0.1.15/17.03 and 0.1.16/18.06 - see https://github.com/docker/docker-ce/releases - currently https://github.com/docker/docker-ce/releases/tag/v18.06.3-ce

ubuntu@a-rke:~$ sudo curl https://releases.rancher.com/install-docker/18.06.sh | sh
ubuntu@a-rke:~$ sudo usermod -aG docker ubuntu
ubuntu@a-rke:~$ sudo docker version
Client:
 Version:           18.06.3-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        d7080c1
 Built:             Wed Feb 20 02:27:18 2019

# install RKE
sudo wget https://github.com/rancher/rke/releases/download/v0.1.16/rke_linux-amd64
mv rke_linux-amd64 rke
sudo mv ./rke /usr/local/bin/rke

ubuntu@a-rke:~$ rke --version
rke version v0.1.16

Kubernetes HA Cluster Production Installation


Docker Desktop - Default single node Kubernetes cluster

Docker Desktop 2.0.0.3 comes with Kubernetes v1.10.11 on top of Docker 18.09.2 - I would recommend moving up the far more recent Kubernetes v1.14.3 on top of Docker 19.03.1 as part of Docker Desktop 2.1.0.1.

Kubernetes 1.10Kubernetes 1.14

Docker desktop comes by default with a kubernetes cluster (you must enable it)

Enable kubernetes 

// In windowsmichaelobrien@biometrics MINGW64 ~
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

michaelobrien@biometrics MINGW64 ~
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c8e1c1dc6b93 docker/kube-compose-controller "/compose-controller…" About a minute ago Up About a minute k8s_compose_compose-74649b4db6-fn6bt_docker_6af81994-caac-11e9-9b7e-00155d663100_0
60df4dcc5951 docker/kube-compose-api-server "/api-server --kubec…" About a minute ago Up About a minute k8s_compose_compose-api-7564f85bcf-pzzst_docker_6ae014d2-caac-11e9-9b7e-00155d663100_0
a21462e44609 k8s.gcr.io/pause-amd64:3.1 "/pause" About a minute ago Up About a minute k8s_POD_compose-74649b4db6-fn6bt_docker_6af81994-caac-11e9-9b7e-00155d663100_0
bce704c0c0ed k8s.gcr.io/pause-amd64:3.1 "/pause" About a minute ago Up About a minute k8s_POD_compose-api-7564f85bcf-pzzst_docker_6ae014d2-caac-11e9-9b7e-00155d663100_0
b3080360fa2b k8s.gcr.io/k8s-dns-sidecar-amd64 "/sidecar --v=2 --lo…" About a minute ago Up About a minute k8s_sidecar_kube-dns-86f4d74b45-8b7n6_kube-system_530cdf8d-caac-11e9-9b7e-00155d663100_0
59863c770eb1 k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64 "/dnsmasq-nanny -v=2…" About a minute ago Up About a minute k8s_dnsmasq_kube-dns-86f4d74b45-8b7n6_kube-system_530cdf8d-caac-11e9-9b7e-00155d663100_0
bdde470b375d k8s.gcr.io/k8s-dns-kube-dns-amd64 "/kube-dns --domain=…" About a minute ago Up About a minute k8s_kubedns_kube-dns-86f4d74b45-8b7n6_kube-system_530cdf8d-caac-11e9-9b7e-00155d663100_0
398c7f8c6e0d k8s.gcr.io/kube-proxy-amd64 "/usr/local/bin/kube…" About a minute ago Up About a minute k8s_kube-proxy_kube-proxy-nzm9j_kube-system_5309f3c0-caac-11e9-9b7e-00155d663100_0
76992079974c k8s.gcr.io/pause-amd64:3.1 "/pause" 2 minutes ago Up About a minute k8s_POD_kube-dns-86f4d74b45-8b7n6_kube-system_530cdf8d-caac-11e9-9b7e-00155d663100_0
3421d48f9150 k8s.gcr.io/pause-amd64:3.1 "/pause" 2 minutes ago Up About a minute k8s_POD_kube-proxy-nzm9j_kube-system_5309f3c0-caac-11e9-9b7e-00155d663100_0
c23b1b374fd7 e851a7aeb6e8 "kube-apiserver --ad…" 2 minutes ago Up 2 minutes k8s_kube-apiserver_kube-apiserver-docker-for-desktop_kube-system_bb0ce6461863dda427ec695afd7382b1_1
dd1a5c5b954e k8s.gcr.io/etcd-amd64 "etcd --client-cert-…" 2 minutes ago Up 2 minutes k8s_etcd_etcd-docker-for-desktop_kube-system_48668e6f8eb2c5de8ec8f4109bcc57cc_0
dce159bcbfc3 k8s.gcr.io/kube-scheduler-amd64 "kube-scheduler --le…" 2 minutes ago Up 2 minutes k8s_kube-scheduler_kube-scheduler-docker-for-desktop_kube-system_ecf299f4fa454da5ab299dffcd70c70f_0
ae2aeb910a75 k8s.gcr.io/kube-controller-manager-amd64 "kube-controller-man…" 2 minutes ago Up 2 minutes k8s_kube-controller-manager_kube-controller-manager-docker-for-desktop_kube-system_14d6eb408e956ff69623d89a5202834b_0
73f818622deb k8s.gcr.io/pause-amd64:3.1 "/pause" 3 minutes ago Up 2 minutes k8s_POD_etcd-docker-for-desktop_kube-system_48668e6f8eb2c5de8ec8f4109bcc57cc_0
c8fe8b3804e2 k8s.gcr.io/pause-amd64:3.1 "/pause" 3 minutes ago Up 2 minutes k8s_POD_kube-apiserver-docker-for-desktop_kube-system_bb0ce6461863dda427ec695afd7382b1_0
c7b5de703df4 k8s.gcr.io/pause-amd64:3.1 "/pause" 3 minutes ago Up 3 minutes k8s_POD_kube-controller-manager-docker-for-desktop_kube-system_14d6eb408e956ff69623d89a5202834b_0
81f2d85cee89 k8s.gcr.io/pause-amd64:3.1 "/pause" 3 minutes ago Up 2 minutes k8s_POD_kube-scheduler-docker-for-desktop_kube-system_ecf299f4fa454da5ab299dffcd70c70f_0

michaelobrien@biometrics MINGW64 ~
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
docker-for-desktop Ready master 2m v1.10.11

michaelobrien@biometrics MINGW64 ~
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
docker compose-74649b4db6-fn6bt 1/1 Running 0 1m
docker compose-api-7564f85bcf-pzzst 1/1 Running 0 1m
kube-system etcd-docker-for-desktop 1/1 Running 0 1m
kube-system kube-apiserver-docker-for-desktop 1/1 Running 1 1m
kube-system kube-controller-manager-docker-for-desktop 1/1 Running 0 1m
kube-system kube-dns-86f4d74b45-8b7n6 3/3 Running 0 2m
kube-system kube-proxy-nzm9j 1/1 Running 0 2m
kube-system kube-scheduler-docker-for-desktop 1/1 Running 0 1m

michaelobrien@biometrics MINGW64 ~
$ kubectl get services --all-namespaces
NAMESPACE     NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
default       kubernetes    ClusterIP   10.96.0.1        <none>        443/TCP         9m
docker        compose-api   ClusterIP   10.108.148.162   <none>        443/TCP         8m
kube-system   kube-dns      ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP   9m


Kubernetes 1.14 based
PS C:\Windows\system32> docker ps -a
CONTAINER ID        IMAGE                            COMMAND                  CREATED              STATUS              PORTS               NAMES
8b13d62e7bfc        docker/kube-compose-controller   "/compose-controller…"   About a minute ago   Up About a minute                       k8s_compose_compose-6c67d745f6-4b4bn_docker_cf2269f0-cc2e-11e9-a3cc-00155d663102_0
59d0faa98d85        docker/kube-compose-api-server   "/api-server --kubec…"   About a minute ago   Up About a minute                       k8s_compose_compose-api-57ff65b8c7-rdlh9_docker_cf1b94c4-cc2e-11e9-a3cc-00155d663102_0
398a3b5e96f9        k8s.gcr.io/pause:3.1             "/pause"                 About a minute ago   Up About a minute                       k8s_POD_compose-6c67d745f6-4b4bn_docker_cf2269f0-cc2e-11e9-a3cc-00155d663102_0
8107237c9e58        k8s.gcr.io/pause:3.1             "/pause"                 About a minute ago   Up About a minute                       k8s_POD_compose-api-57ff65b8c7-rdlh9_docker_cf1b94c4-cc2e-11e9-a3cc-00155d663102_0
78a359d43285        eb516548c180                     "/coredns -conf /etc…"   2 minutes ago        Up 2 minutes                            k8s_coredns_coredns-fb8b8dccf-6qqnh_kube-system_a33262a3-cc2e-11e9-a3cc-00155d663102_0
427183ed6d57        eb516548c180                     "/coredns -conf /etc…"   2 minutes ago        Up 2 minutes                            k8s_coredns_coredns-fb8b8dccf-ltbgd_kube-system_a3312bf3-cc2e-11e9-a3cc-00155d663102_0
2c60fe972a24        004666307c5b                     "/usr/local/bin/kube…"   2 minutes ago        Up 2 minutes                            k8s_kube-proxy_kube-proxy-qrjvf_kube-system_a30ee329-cc2e-11e9-a3cc-00155d663102_0
60d5e4a4fb17        k8s.gcr.io/pause:3.1             "/pause"                 2 minutes ago        Up 2 minutes                            k8s_POD_coredns-fb8b8dccf-6qqnh_kube-system_a33262a3-cc2e-11e9-a3cc-00155d663102_0
77bddf0e283b        k8s.gcr.io/pause:3.1             "/pause"                 2 minutes ago        Up 2 minutes                            k8s_POD_coredns-fb8b8dccf-ltbgd_kube-system_a3312bf3-cc2e-11e9-a3cc-00155d663102_0
aeebbfadf9c5        k8s.gcr.io/pause:3.1             "/pause"                 2 minutes ago        Up 2 minutes                            k8s_POD_kube-proxy-qrjvf_kube-system_a30ee329-cc2e-11e9-a3cc-00155d663102_0
91e4d986093e        9946f563237c                     "kube-apiserver --ad…"   2 minutes ago        Up 2 minutes                            k8s_kube-apiserver_kube-apiserver-docker-desktop_kube-system_7c4f3d43558e9fadf2d2b323b2e78235_0
1fcac02063de        2c4adeb21b4f                     "etcd --advertise-cl…"   2 minutes ago        Up 2 minutes                            k8s_etcd_etcd-docker-desktop_kube-system_3773efb8e009876ddfa2c10173dba95e_0
7893ab856a39        ac2ce44462bc                     "kube-controller-man…"   2 minutes ago        Up 2 minutes                            k8s_kube-controller-manager_kube-controller-manager-docker-desktop_kube-system_9c58c6d32bd3a2d42b8b10905b8e8f54_0
4303da6a46a5        953364a3ae7a                     "kube-scheduler --bi…"   2 minutes ago        Up 2 minutes                            k8s_kube-scheduler_kube-scheduler-docker-desktop_kube-system_124f5bab49bf26c80b1c1be19641c3e8_0
01d222f23f98        k8s.gcr.io/pause:3.1             "/pause"                 2 minutes ago        Up 2 minutes                            k8s_POD_kube-scheduler-docker-desktop_kube-system_124f5bab49bf26c80b1c1be19641c3e8_0
0dcc4e343bc0        k8s.gcr.io/pause:3.1             "/pause"                 2 minutes ago        Up 2 minutes                            k8s_POD_kube-controller-manager-docker-desktop_kube-system_9c58c6d32bd3a2d42b8b10905b8e8f54_0
ca16b0d85cda        k8s.gcr.io/pause:3.1             "/pause"                 2 minutes ago        Up 2 minutes                            k8s_POD_kube-apiserver-docker-desktop_kube-system_7c4f3d43558e9fadf2d2b323b2e78235_0
4c921ad85555        k8s.gcr.io/pause:3.1             "/pause"                 2 minutes ago        Up 2 minutes                            k8s_POD_etcd-docker-desktop_kube-system_3773efb8e009876ddfa2c10173dba95e_0
PS C:\Windows\system32> kubectl get services --all-namespaces
NAMESPACE     NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
default       kubernetes    ClusterIP   10.96.0.1        <none>        443/TCP                  2m43s
docker        compose-api   ClusterIP   10.100.135.112   <none>        443/TCP                  81s
kube-system   kube-dns      ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP   2m42s
PS C:\Windows\system32> kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0", GitTreeState:"clean", BuildDate:"2019-06-06T01:44:30Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0", GitTreeState:"clean", BuildDate:"2019-06-06T01:36:19Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
PS C:\Windows\system32> kubectl get nodes
NAME             STATUS   ROLES    AGE    VERSION
docker-desktop   Ready    master   3m7s   v1.14.3
PS C:\Windows\system32> kubectl get pods --all-namespaces
NAMESPACE     NAME                                     READY   STATUS    RESTARTS   AGE
docker        compose-6c67d745f6-4b4bn                 1/1     Running   0          107s
docker        compose-api-57ff65b8c7-rdlh9             1/1     Running   0          107s
kube-system   coredns-fb8b8dccf-6qqnh                  1/1     Running   0          3m
kube-system   coredns-fb8b8dccf-ltbgd                  1/1     Running   0          3m1s
kube-system   etcd-docker-desktop                      1/1     Running   0          2m13s
kube-system   kube-apiserver-docker-desktop            1/1     Running   0          111s
kube-system   kube-controller-manager-docker-desktop   1/1     Running   0          108s
kube-system   kube-proxy-qrjvf                         1/1     Running   0          3m1s
kube-system   kube-scheduler-docker-desktop            1/1     Running   0          111s


Helm needs to be installed

Multi Node Kubernetes cluster running RKE on VMWare Workstation

Exposing RKE ports on the VMs

Enable the following ports through the NAT or via VMware VMWare#PortForwardingonVMwareFusion

https://rancher.com/docs/rancher/v2.x/en/installation/requirements/ports/#commonly-used-ports

portuse
22

80

443

2376

2379

2380

3389

6443

8472

9099

10250

10254

30000-

Turn off windows firewall and test SSH access through the NAT

sudo vi /Library/Preferences/VMware\ Fusion/vmnet2/nat.conf

[incomingtcp]
# The format and example are as follows:
#<external port number> = <VM's IP address>:<VM's port number>
#8080 = 172.16.3.128:80
443 = 192.168.199.128:443 #443
10250 = 192.168.199.128:10250
30000 = 192.168.199.128:30000
2380 = 192.168.199.128:2380
2023 = 192.168.199.129:22 #22
8472 = 192.168.199.128:8472
2022 = 192.168.199.128:22 #22
9099 = 192.168.199.128:9099
10254 = 192.168.199.128:10254
2379 = 192.168.199.128:2379
6443 = 192.168.199.128:6443
30001 = 192.168.199.128:30001
3389 = 192.168.199.128:3389
2376 = 192.168.199.128:2376
80 = 192.168.199.128:80 #80

[incomingudp]
# UDP port forwarding example
#6000 = 172.16.3.0:6001
30000 = 192.168.199.128:30000
8472 = 192.168.199.128:8472
30001 = 192.168.199.128:30001

sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --stop
sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --start

Restart the VM network via vm-cli in VMWare#PortForwardingonVMwareFusion

Helm Charts

Install helm

Install Helm on OSX

https://helm.sh/docs/intro/install/

Install Brew 

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

biometric:wse_helm michaelobrien$ brew install helm
==> Downloading https://homebrew.bintray.com/bottles/helm-3.0.3.catalina.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/59/5987c80ea21063f3c26a799889ad3e0b35c73275bd3579e5a1f6785d6f3f43d5?__gda__=exp=1581208465~hmac=fcf13391d90275fbcd6c015d86b3a0ec9abeb02aa1583eb3dca3f653da1aa281&response-content-disposition=attachment%3Bfilename%3D%22helm-3.0.3.catali
######################################################################## 100.0%
==> Pouring helm-3.0.3.catalina.bottle.tar.gz
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completions have been installed to:
  /usr/local/share/zsh/site-functions
==> Summary
  /usr/local/Cellar/helm/3.0.3: 7 files, 40.6MB

biometric:wse_helm michaelobrien$ helm version
version.BuildInfo{Version:"v3.0.3", GitCommit:"ac925eb7279f4a6955df663a0128044a8a6b7593", GitTreeState:"clean", GoVersion:"go1.13.7"}

biometric:wse_go michaelobrien$ kubectl get services --all-namespaces
default       tomcat-dev                 LoadBalancer   10.103.194.12    localhost     80:32305/TCP                 75s

biometric:wse_go michaelobrien$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                     READY   STATUS    RESTARTS   AGE
default       tomcat-dev-76d87c8fb6-9xjr6              1/1     Running   0          106s

biometric:wse_go michaelobrien$ helm list
NAME      	NAMESPACE	REVISION	UPDATED                             	STATUS  	CHART       	APP VERSION
tomcat-dev	default  	1       	2020-02-16 11:15:14.196403 -0500 EST	deployed	tomcat-0.4.1	7.0 

Upgrade Helm using Brew

brew upgrade helm

==> Upgrading 1 outdated package:
helm 3.0.3 -> 3.2.1
==> Upgrading helm 3.0.3 -> 3.2.1

biometric:wse_helm $ helm version
version.BuildInfo{Version:"v3.2.1", GitCommit:"fe51cd1e31e6a202cba7dead9552a6d418ded79a", GitTreeState:"clean", GoVersion:"go1.13.10"} 


Override values.yaml parameters in the default helm chart

root@tomcat-dev-76d87c8fb6-7nxjx:/usr/local/tomcat/logs# curl --head http://127.0.0.1:8080/sample
HTTP/1.1 302 Found
Server: Apache-Coyote/1.1
Location: /sample/
Transfer-Encoding: chunked
Date: Sun, 16 Feb 2020 18:15:28 GMT

root@tomcat-dev-76d87c8fb6-7nxjx:/usr/local/tomcat/logs# curl --head http://127.0.0.1:8080/sample/
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Accept-Ranges: bytes
ETag: W/"636-1185812788000"
Last-Modified: Mon, 30 Jul 2007 16:26:28 GMT
Content-Type: text/html
Content-Length: 636
Date: Sun, 16 Feb 2020 18:16:02 GMT

curl http://192.168.20.144:80/sample/
192.168.65.3 - - [16/Feb/2020:18:21:18 +0000] "GET /sample/ HTTP/1.1" 200 636

change the port from 80 to 31111
helm upgrade tomcat-dev stable/tomcat --set service.externalPort=31111
default       tomcat-dev    LoadBalancer   10.97.22.152   localhost     31111:31962/TCP          26m

n> curl http://192.168.20.144:31111/sample/
StatusCode        : 200
StatusDescription : OK


Install Helm on Windows 10

https://github.com/helm/helm/releases/tag/v3.2.0

extract the zip and add the exe to your classpath
F:\opt\helm32\helm.exe

$ helm version
version.BuildInfo{Version:"v3.2.0", GitCommit:"e11b7ce3b12db2941e90399e874513fbd24bcb71", GitTreeState:"clean", GoVersion:"go1.13.10"}



Install Tomcat from a stable Helm chart

Follow https://github.com/helm/charts/tree/master/stable/tomcat

biometric:wse_go michaelobrien$ helm repo add stable https://kubernetes-charts.storage.googleapis.com
"stable" has been added to your repositories

biometric:wse_go michaelobrien$ helm install tomcat-dev stable/tomcat
NAME: tomcat-dev
LAST DEPLOYED: Sun Feb 16 11:15:14 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Get the application URL by running these commands:
     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
           You can watch the status of by running 'kubectl get svc -w tomcat-dev'
  export SERVICE_IP=$(kubectl get svc --namespace default tomcat-dev -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  echo http://$SERVICE_IP:
biometric:wse_go michaelobrien$ export SERVICE_IP=$(kubectl get svc --namespace default tomcat-dev -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
biometric:wse_go michaelobrien$ echo http://$SERVICE_IP:
http://localhost:


Create your own Helm charts

https://helm.sh/docs/using_helm/#quickstart-guide  - this example contains a default nginx server

Create Helm Chart

ubuntu@ip-172-31-81-46:~/obrienlabs$ sudo helm create difference-nbi
Creating difference-nbi
ubuntu@ip-172-31-81-46:~/obrienlabs$ ls difference-nbi/
Chart.yaml  charts  templates  values.yaml

Package Helm Chart

ubuntu@ip-172-31-81-46:~/obrienlabs/difference-nbi$ cd ..
ubuntu@ip-172-31-81-46:~/obrienlabs$ sudo helm package difference-nbi
Successfully packaged chart and saved it to: /home/ubuntu/obrienlabs/difference-nbi-0.1.0.tgz

Install Helm Chart

ubuntu@ip-172-31-81-46:~/obrienlabs$ helm install difference-nbi --name difference-nbi
NAME:   difference-nbi
LAST DEPLOYED: Mon Jun 10 19:00:59 2019
ubuntu@ip-172-31-81-46:~/obrienlabs$ kubectl get pods --all-namespaces
NAMESPACE       NAME                                      READY   STATUS      RESTARTS   AGE
default         difference-nbi-5fc754f69-hqkr2            1/1     Running     0          16s
ubuntu@ip-172-31-81-46:~/obrienlabs$ kubectl get services --all-namespaces
NAMESPACE       NAME                                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                  AGE
default         difference-nbi                         ClusterIP   10.43.82.98     <none>        80/TCP                                   75s
ubuntu@ip-172-31-81-46:~/charts/stable$ sudo helm ls
NAME          	REVISION	UPDATED                 	STATUS  	CHART                       	APP VERSION  	NAMESPACE
difference-nbi	1       	Mon Jun 10 19:00:59 2019	DEPLOYED	difference-nbi-0.1.0        	1.0          	default   

ubuntu@ubuntu:~$ kubectl port-forward difference-nbi-74955fd75b-9kgb2 8180:80
Forwarding from 127.0.0.1:8180 -> 80
Forwarding from [::1]:8180 -> 80

SCP Helm Chart and git commit it

obrienbiometrics:difference-kubernetes michaelobrien$ scp -rp ubuntu@rke.obrienlabs.cloud:~/obrienlabs/* .
Chart.yaml                                                                                                                                                     100%  110     1.4KB/s   00:00    
.helmignore                                                                                                                                                    100%  342     4.5KB/s   00:00    
values.yaml                                                                                                                                                    100% 1070    13.7KB/s   00:00    
service.yaml                                                                                                                                                   100%  611     7.1KB/s   00:00    
deployment.yaml                                                                                                                                                100% 1581    12.7KB/s   00:00    
ingress.yaml                                                                                                                                                   100% 1070    13.2KB/s   00:00    
_helpers.tpl                                                                                                                                                   100% 1066    14.3KB/s   00:00    
test-connection.yaml                                                                                                                                           100%  585     9.9KB/s   00:00    
NOTES.txt                                                                                                                                                      100% 1513    20.9KB/s   00:00    


Upgrade Helm Chart

Use helm hooks to key into any part of the lifecycle.

Uninstall Helm Chart


Installations

Docker is the lowest layer in our Docker | Kubernetes | Helm orchestration stack.  On Ubuntu installation is a couple lines of code, however on Windows installation is a bit more involved.  Docker desktop for windows or OSX comes with a kubernetes stack out of the box.

Docker Installation

see Docker Development Guide

Docker Installation on Ubuntu

Docker Installation on OSX

VMware Fusion and Docker can co-exist on OSX

Docker Installation on ARM Raspberry PI 4

Raspberry PI#DockerSetup

Docker Installation on Windows

Note: until VMware changes their control plane to kubernetes - Windows installations of Docker require Hyper-v which is incompatible with VMware Workstation.

Docker Installation on Windows non-Admin accounts

If you wish to run docker from a non-admin account - do the following first.

Install docker desktop from the admin account.

Add the non-admin account to the docker-users group.

Startup docker.



If you attempt to download docker image layers from a docker repository (as part of running a container) - and you are running behind a docker proxy - set the proxy in docker preferences first.


Restart docker after any configuration change.

Verify you can startup a simple container with no file system shares.

Enable the built in Kubernetes cluster.

Startup a tomcat container to verify docker desktop.

Override any firewall rules blocking port access.

Enabled file sharing so we can use persistent volumes from Docker or Kubernetes charts.

Enable file sharing through your firewall for docker containers.

open port 445 on 10.0.75.1 as per https://docs.docker.com/docker-for-windows/#firewall-rules-for-shared-drives

Upgrading inside a firewall

An error occurred while sending the request.
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at LightweightInstaller.DownloadStep.<DoAsync>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at LightweightInstaller.InstallWorkflow.<ProcessAsync>d__23.MoveNext()


Verify versions

PS C:\Users\michaelobrien> docker version
Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfc
 Built:             Thu Aug 29 05:26:49 2019
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.8
  Git commit:       6a30dfc
  Built:            Thu Aug 29 05:32:21 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
PS C:\Users\michaelobrien> docker-compose version
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.0.2q  20 Nov 2018



Kubernetes Native Applications - Kubernetes Operator

Operators enable us to develop for Kubernetes.

https://kubernetes.io/docs/concepts/extend-kubernetes/operator/

Existing at https://operatorhub.io/

Writing your own at https://kudo.dev/ | https://book.kubebuilder.io/ | https://github.com/operator-framework/getting-started
https://github.com/operator-framework/operator-sdk-samples/tree/master/go/memcached-operator/

https://medium.com/@mtreacher/writing-a-kubernetes-operator-a9b86f19bfb9

https://medium.com/@cloudark/kubernetes-operator-faq-e018132c6ea2

https://enterprisersproject.com/article/2019/2/kubernetes-operators-plain-english

https://kubernetes.io/docs/concepts/extend-kubernetes/operator/#what-s-next

Golang based Kubernetes Operators

see Go Lang Developer Guide

see https://medium.com/@mtreacher/writing-a-kubernetes-operator-a9b86f19bfb9



Containerizing Applications

https://blogs.oracle.com/javamagazine/containerizing-apps-with-jlink

Kubernetes Frameworks Plugins and Tools


Persistent Volumes

use GlusterFS - 

Open Policy Agent

JSON based CNCF project run as a sidecar container or cluster vm DaemonSet

https://www.openpolicyagent.org/docs/v0.12.2/kubernetes-admission-control/

Kubectl Command Reference

see Kubernetes Cheetsheet https://kubernetes.io/docs/reference/kubectl/cheatsheet/

Switching context from Azure AKS back to local Docker Desktop Kubernetes

Connecting to an Azure AKS instance at the same time as running your own developer kubernetes install

biometric:reference-nbi michaelobrien$ kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://kubernetes.docker.internal:6443
  name: docker-desktop
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://obl-dev-....5.hcp.eastus.azmk8s.io:443
  name: obl-dev
contexts:
- context:
    cluster: docker-desktop
    user: docker-desktop
  name: docker-desktop
- context:
    cluster: docker-desktop
    user: docker-desktop
  name: docker-for-desktop
- context:
    cluster: obl-dev
    user: clusterUser_obl_dev_aks_obl-dev
  name: obl-dev
current-context: obl-dev
kind: Config
preferences: {}
users:
- name: clusterUser_obl_dev_aks_obl-dev
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
    token: 3a0ee7e0....126fe
- name: docker-desktop
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
biometric:reference-nbi michaelobrien$ kubectl config current-context
obl-dev
biometric:reference-nbi michaelobrien$ kubectl config get-contexts
CURRENT   NAME                 CLUSTER          AUTHINFO                          NAMESPACE
          docker-desktop       docker-desktop   docker-desktop                    
          docker-for-desktop   docker-desktop   docker-desktop                    
*         obl-dev              obl-dev          clusterUser_obl_dev_aks_obl-dev   
biometric:reference-nbi michaelobrien$ kubectl config use-context docker-desktop
Switched to context "docker-desktop".
biometric:reference-nbi michaelobrien$ kubectl config get-contexts
CURRENT   NAME                 CLUSTER          AUTHINFO                          NAMESPACE
*         docker-desktop       docker-desktop   docker-desktop                    
          docker-for-desktop   docker-desktop   docker-desktop                    
          obl-dev              obl-dev          clusterUser_obl_dev_aks_obl-dev   
biometric:reference-nbi michaelobrien$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                     READY   STATUS    RESTARTS   AGE
default       kafka-0                                  1/1     Running   1          62d
default       kafka-1                                  1/1     Running   1          62d
default       kafka-2                                  1/1     Running   1          62d
default       kafka-zookeeper-0                        1/1     Running   0          62d
default       kafka-zookeeper-1                        1/1     Running   0          62d
default       kafka-zookeeper-2                        1/1     Running   0          62d
default       testclient                               1/1     Running   0          62d
default       tomcat-dev-76d87c8fb6-9xjr6              1/1     Running   0          54d
docker        compose-7b7c5cbbcc-6nhng                 1/1     Running   0          62d
docker        compose-api-dbbf7c5db-2lsq2              1/1     Running   0          62d
kube-system   coredns-5c98db65d4-rbggz                 1/1     Running   1          62d
kube-system   coredns-5c98db65d4-txftp                 1/1     Running   1          62d
kube-system   etcd-docker-desktop                      1/1     Running   0          62d
kube-system   kube-apiserver-docker-desktop            1/1     Running   0          62d
kube-system   kube-controller-manager-docker-desktop   1/1     Running   0          62d
kube-system   kube-proxy-7brgm                         1/1     Running   0          62d
kube-system   kube-scheduler-docker-desktop            1/1     Running   0          62d
kube-system   storage-provisioner                      1/1     Running   1          7d1h


Jiras

https://github.com/kubernetes/kubernetes/issues/83253

Links

https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/

https://coreos.com/blog/rkt-and-kubernetes.html










  • No labels