Kubernetes Developer Guide | Helm Development Guide | Reference Architecture
todo: https://www.terraform.io/docs/providers/helm/index.html
Helm V2
We can run collocated helm installs by moving the helm binary to helm3 and installing helm2
helm v3 changes from v2 https://helm.sh/docs/faq/
brew uninstall helm brew uninstall helm@2 brew install helm@2 helm version vi ~/.bash_profile add export PATH="/usr/local/opt/helm@2/bin:$PATH" source ~/.bash_profile helm version # enable tiller in minikube kubectl -n kube-system create serviceaccount tiller kubectl create clusterrolebinding tiller --clusterrole=cluster-admin --serviceaccount=kube-system:tiller helm init --service-account tiller kubectl -n kube-system rollout status deploy/tiller-deploy sudo helm init helm serve & helm repo add local http://127.0.0.1:8879 helm repo list
Helm V3
For kubernetes installation/development see Kubernetes Developer Guide
see also Asynchronous Messaging using Kafka#KafkaOperatorCharts
- REF-3Getting issue details... STATUS
Install helm
Install Helm on OSX
https://helm.sh/docs/intro/install/
Install Brew
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" biometric:wse_helm michaelobrien$ brew install helm ==> Downloading https://homebrew.bintray.com/bottles/helm-3.0.3.catalina.bottle.tar.gz ==> Downloading from https://akamai.bintray.com/59/5987c80ea21063f3c26a799889ad3e0b35c73275bd3579e5a1f6785d6f3f43d5?__gda__=exp=1581208465~hmac=fcf13391d90275fbcd6c015d86b3a0ec9abeb02aa1583eb3dca3f653da1aa281&response-content-disposition=attachment%3Bfilename%3D%22helm-3.0.3.catali ######################################################################## 100.0% ==> Pouring helm-3.0.3.catalina.bottle.tar.gz ==> Caveats Bash completion has been installed to: /usr/local/etc/bash_completion.d zsh completions have been installed to: /usr/local/share/zsh/site-functions ==> Summary /usr/local/Cellar/helm/3.0.3: 7 files, 40.6MB biometric:wse_helm michaelobrien$ helm version version.BuildInfo{Version:"v3.0.3", GitCommit:"ac925eb7279f4a6955df663a0128044a8a6b7593", GitTreeState:"clean", GoVersion:"go1.13.7"} biometric:wse_go michaelobrien$ kubectl get services --all-namespaces default tomcat-dev LoadBalancer 10.103.194.12 localhost 80:32305/TCP 75s biometric:wse_go michaelobrien$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE default tomcat-dev-76d87c8fb6-9xjr6 1/1 Running 0 106s biometric:wse_go michaelobrien$ helm list NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION tomcat-dev default 1 2020-02-16 11:15:14.196403 -0500 EST deployed tomcat-0.4.1 7.0
Upgrade Helm using Brew
brew upgrade helm ==> Upgrading 1 outdated package: helm 3.0.3 -> 3.2.1 ==> Upgrading helm 3.0.3 -> 3.2.1 biometric:wse_helm $ helm version version.BuildInfo{Version:"v3.2.1", GitCommit:"fe51cd1e31e6a202cba7dead9552a6d418ded79a", GitTreeState:"clean", GoVersion:"go1.13.10"}
Override values.yaml parameters in the default helm chart
root@tomcat-dev-76d87c8fb6-7nxjx:/usr/local/tomcat/logs# curl --head http://127.0.0.1:8080/sample HTTP/1.1 302 Found Server: Apache-Coyote/1.1 Location: /sample/ Transfer-Encoding: chunked Date: Sun, 16 Feb 2020 18:15:28 GMT root@tomcat-dev-76d87c8fb6-7nxjx:/usr/local/tomcat/logs# curl --head http://127.0.0.1:8080/sample/ HTTP/1.1 200 OK Server: Apache-Coyote/1.1 Accept-Ranges: bytes ETag: W/"636-1185812788000" Last-Modified: Mon, 30 Jul 2007 16:26:28 GMT Content-Type: text/html Content-Length: 636 Date: Sun, 16 Feb 2020 18:16:02 GMT curl http://192.168.20.144:80/sample/ 192.168.65.3 - - [16/Feb/2020:18:21:18 +0000] "GET /sample/ HTTP/1.1" 200 636 change the port from 80 to 31111 helm upgrade tomcat-dev stable/tomcat --set service.externalPort=31111 default tomcat-dev LoadBalancer 10.97.22.152 localhost 31111:31962/TCP 26m n> curl http://192.168.20.144:31111/sample/ StatusCode : 200 StatusDescription : OK
Install Helm on Windows 10
https://github.com/helm/helm/releases/tag/v3.2.0
extract the zip and add the exe to your classpath F:\opt\helm32\helm.exe $ helm version version.BuildInfo{Version:"v3.2.0", GitCommit:"e11b7ce3b12db2941e90399e874513fbd24bcb71", GitTreeState:"clean", GoVersion:"go1.13.10"}
Helm Chart Examples
Install Tomcat from a stable Helm chart
Follow https://github.com/helm/charts/tree/master/stable/tomcat
biometric:wse_go michaelobrien$ helm repo add stable https://kubernetes-charts.storage.googleapis.com "stable" has been added to your repositories biometric:wse_go michaelobrien$ helm install tomcat-dev stable/tomcat NAME: tomcat-dev LAST DEPLOYED: Sun Feb 16 11:15:14 2020 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: 1. Get the application URL by running these commands: NOTE: It may take a few minutes for the LoadBalancer IP to be available. You can watch the status of by running 'kubectl get svc -w tomcat-dev' export SERVICE_IP=$(kubectl get svc --namespace default tomcat-dev -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') echo http://$SERVICE_IP: biometric:wse_go michaelobrien$ export SERVICE_IP=$(kubectl get svc --namespace default tomcat-dev -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') biometric:wse_go michaelobrien$ echo http://$SERVICE_IP: http://localhost:
Your own Helm charts
https://helm.sh/docs/using_helm/#quickstart-guide - this example contains a default nginx server
Helm Lifecycle
Create Helm Chart
ubuntu@ip-172-31-81-46:~/obrienlabs$ sudo helm create difference-nbi Creating difference-nbi ubuntu@ip-172-31-81-46:~/obrienlabs$ ls difference-nbi/ Chart.yaml charts templates values.yaml
Package Helm Chart
ubuntu@ip-172-31-81-46:~/obrienlabs/difference-nbi$ cd .. ubuntu@ip-172-31-81-46:~/obrienlabs$ sudo helm package difference-nbi Successfully packaged chart and saved it to: /home/ubuntu/obrienlabs/difference-nbi-0.1.0.tgz package to local chart repo
Install Helm Chart
ubuntu@ip-172-31-81-46:~/obrienlabs$ helm install difference-nbi --name difference-nbi NAME: difference-nbi LAST DEPLOYED: Mon Jun 10 19:00:59 2019 ubuntu@ip-172-31-81-46:~/obrienlabs$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE default difference-nbi-5fc754f69-hqkr2 1/1 Running 0 16s ubuntu@ip-172-31-81-46:~/obrienlabs$ kubectl get services --all-namespaces NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default difference-nbi ClusterIP 10.43.82.98 <none> 80/TCP 75s ubuntu@ip-172-31-81-46:~/charts/stable$ sudo helm ls NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE difference-nbi 1 Mon Jun 10 19:00:59 2019 DEPLOYED difference-nbi-0.1.0 1.0 default ubuntu@ubuntu:~$ kubectl port-forward difference-nbi-74955fd75b-9kgb2 8180:80 Forwarding from 127.0.0.1:8180 -> 80 Forwarding from [::1]:8180 -> 80
SCP Helm Chart and git commit it
obrienbiometrics:difference-kubernetes michaelobrien$ scp -rp ubuntu@rke.obrienlabs.cloud:~/obrienlabs/* . Chart.yaml 100% 110 1.4KB/s 00:00 .helmignore 100% 342 4.5KB/s 00:00 values.yaml 100% 1070 13.7KB/s 00:00 service.yaml 100% 611 7.1KB/s 00:00 deployment.yaml 100% 1581 12.7KB/s 00:00 ingress.yaml 100% 1070 13.2KB/s 00:00 _helpers.tpl 100% 1066 14.3KB/s 00:00 test-connection.yaml 100% 585 9.9KB/s 00:00 NOTES.txt 100% 1513 20.9KB/s 00:00
Upgrade Helm Chart
Use helm hooks to key into any part of the lifecycle.
ONAP references to helm upgrade
https://gitlab.com/obriensystems/oom/-/blob/master/kubernetes/helm/plugins/deploy/deploy.sh
https://git.onap.org/oom/tree/kubernetes/helm/plugins/deploy/deploy.sh#n184
https://docs.onap.org/en/elalto/submodules/oom.git/docs/oom_quickstart_guide.html
Issues with pod deletion - see https://github.com/obrienlabs/onap-root/blob/master/cd.sh#L63
Uninstall/delete Helm Chart
delete the chart, delete handing pods, optionally delete the pvc then pv. sudo helm delete kafka sudo helm del --purge kafka kubectl delete pvc/datadir-kafka-0 or delete all in order kubectl delete pvc --all kubectl delete pv --all
Helm Backup Plugin
see https://github.com/maorfr/helm-backup from https://v2.helm.sh/docs/related/
sudo helm plugin install https://github.com/maorfr/helm-backup helm install --namespace kafka --name kafka kafka/ sudo helm backup kafka helm delete --purge kafka sudo helm backup kafka --restore biometric:incubator michaelobrien$ kubectl get pods --all-namespaces | grep kafka kafka kafka-0 1/1 Running 1 2m53s kafka kafka-1 1/1 Running 0 80s kafka kafka-2 1/1 Running 0 47s kafka kafka-zookeeper-0 1/1 Running 1 2m53s kafka kafka-zookeeper-1 1/1 Running 0 2m17s kafka kafka-zookeeper-2 1/1 Running 0 113s
Kubectl apply from Helm get/pull backup
There are cases where a deployed helm chart can be backed up to a single yaml using "helm get". In order to redeploy this yaml without doing a helm install - the yaml must be edited to remove the initial values.yaml extract at the top before the first apiVersion line.
biometric:incubator michaelobrien$ sudo helm get kafka > kafka_get.yaml biometric:incubator michaelobrien$ helm delete --purge kafka biometric:incubator michaelobrien$ vi kafka_get.yaml remove the following to avoid error: error parsing kafka_get.yaml: error converting YAML to JSON: yaml: line 7: could not find expected ':' REVISION: 1 RELEASED: Thu Jul 23 14:26:00 2020 CHART: kafka-0.21.2 USER-SUPPLIED VALUES: {} ... to HOOKS: biometric:incubator michaelobrien$ kubectl apply -f kafka_get.yaml pod/kafka-test-topic-create-consume-produce created poddisruptionbudget.policy/kafka-zookeeper created configmap/kafka-zookeeper created service/kafka-zookeeper-headless created service/kafka-zookeeper created service/kafka created service/kafka-headless created statefulset.apps/kafka-zookeeper created statefulset.apps/kafka created biometric:incubator michaelobrien$ kubectl get pods --all-namespaces | grep kafka default kafka-0 1/1 Running 1 4m58s default kafka-1 1/1 Running 0 3m37s default kafka-2 1/1 Running 0 3m5s default kafka-test-topic-create-consume-produce 0/1 Error 0 4m58s default kafka-zookeeper-0 1/1 Running 1 4m58s default kafka-zookeeper-1 1/1 Running 0 4m24s default kafka-zookeeper-2 1/1 Running 0 3m58s biometric:incubator michaelobrien$ kubectl delete -f kafka_get.yaml pod "kafka-test-topic-create-consume-produce" deleted poddisruptionbudget.policy "kafka-zookeeper" deleted configmap "kafka-zookeeper" deleted service "kafka-zookeeper-headless" deleted service "kafka-zookeeper" deleted service "kafka" deleted service "kafka-headless" deleted statefulset.apps "kafka-zookeeper" deleted statefulset.apps "kafka" deleted
1 Comment
Michael O'Brien
For deployments
pre kubernetes 1.16
apiVersion: extensions/v1beta1
kubernetes 1.16+
apiVersion: apps/v1
to avoid
biometric:reference-helm michaelobrien$ helm install --name reference-nbi reference-nbi-0.1.0.tgz
Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"
with apps/v1
biometric:reference-helm michaelobrien$ helm install --name reference-nbi reference-nbi-0.1.0.tgz
NAME: reference-nbi
LAST DEPLOYED: Wed Jul 15 21:18:54 2020
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
reference-nbi 0/1 1 0 0s
default reference-nbi-6bf49d969d-wrhrh 1/1 Running 0 33s