Infrastructure

Kubernetes Cluster on VMWare Workstation 15 across multiple physical machines

Issue: running a Kubernetes cluster across multiple VMs across multiple physical machines - sharing ports and the network

VMware Fusion Pro - OSX

Verify VMX capability

sysctl -a | grep -E --color 'machdep.cpu.features|VMX'
machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C

Check Side channels

https://kb.vmware.com/s/article/79832

vm | settings | advanced

Port Forwarding on VMware Fusion

Via VMware Fusion 11.5 gui

in Preferences | Network - add a new network - add a port forward rule.  Switch the VM to use this network.  Restart VM, Rewrite port forward rule with new IP

// test connection

biometric:reference user$ ssh user@192.168.20.156 -p 2022
Last login: Tue May 19 15:53:06 2020 from 192.168.58.1
user@obriensystemsu0:~$ ifconfig
ens33 Link encap:Ethernet HWaddr 00:0c:29:ab:19:b7 
inet addr:192.168.75.128 Bcast:192.168.75.255 Mask:255.255.255.0

Via command line

see RKE port exposure in Kubernetes Developer Guide#ExposingRKEportsontheVMs

sudo vi /Library/Preferences/VMware\ Fusion/vmnet2/nat.conf 
sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --stop
sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --start

Issues

Workstation Pro 16 and Nested Virtualization on Windows 10 2004

Some VMs no longer allow the VT-x bit set - fixing

in powershell/reboot
bcdedit /set hypervisorlaunchtype off

Workstation Pro 16 and Hyper-V

It seems workstation and hyper-v still dont work together but out of the box on a new thinkpad P17 - after installing docker desktop to use WSL - VMware still cannot do nested virtualization.

getting

https://communities.vmware.com/thread/643805

https://docs.vmware.com/en/VMware-Workstation-Pro/16.0/workstation-pro-16-user-guide.pdf

VBS and Hyper-V = slower

16.1

Solution

for "Virtualized Intel VT-x/EPT is not supported on this platform."

Installing Windows Hypervisor Platform (keeping hyper-v off as it is not required anymore for Docker Desktop) - in windows features on/off near the bottom

https://docs.microsoft.com/en-us/virtualization/api/

did not work, this works https://communities.vmware.com/thread/634674

20211218: Verified VMware 16.1.0 on Windows 10 pro for workstations 21H1 19043.1415 128G RAM Xeon W-10855M

1 - turn off hyper-v and containers

2 - reboot

3 - run in admin powershell

bcedit /set hypervisorlaunchtype off

4 - reboot

5 - don't let docker for desktop upgrade reinstall

pre 202112 procedure....
flip from auto to in an admin powershell and reboot
bcdedit /set hypervisorlaunchtype off
install wsl2
and
PS C:\Windows\system32> wsl --set-default-version 2
For information on key differences with WSL 2 please visit https://aka.ms/wsl2

breaks docker

Workstation and Docker Desktop - Hyper-v Incompatibility 

For at least until late 2020 - hyper-v is required for docker but VMware needs this windows feature disabled = https://kb.vmware.com/s/article/76918

turn off both hyper-v and containers in windows features

OSX can run both concurrently with no issues.

Workstation 15.0 no longer launches under Windows 10 Prof Nov 2019 Release

If you get the following dialog running VMWare workstation as of Nov 2019

Run the following registry edit to allow Workstation 15 to run again on Windows 10.

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags]
"{3d9912c3-cb54-4f34-ab71-1d429553bf96}"=dword:00000077
"{66f21bbc-149a-411b-8e11-880af7c1266c}"=dword:00000077

Running Workstation 12.5 (from 15) - requires credential guard disable

https://docs.microsoft.com/en-us/windows/security/identity-protection/credential-guard/credential-guard-manage

VMWare Fusion 11 on OSX 10.15 Issues

https://www.huibdijkstra.nl/how-to-set-up-a-osx-mojave-vm-in-vmware-fusion/