Nested ESXi for NSX Lab
In my last lab build for NSX I came across some issues that were very time consuming to diagnose as they only show up when you start working with nsx
I built the nested esx lab in esxi5.1 with vsphere 6 esxi and vsca. See the diagrams in other post for details
1. Create first host, without configuring networking, power off and clone additional hosts
Check that hardware virtualization has been enabled in the vm
2. Install vmware tools for nested ESXi
For esxi 6.0 the tools for nested hosts is integrated into ESXi
For 5.5 hosts download the tools,
Upload the vib using scp to your host and run
esxcli software vib install -v /<path>/esx-tools-for-esxi-9.7.2-0.0.5911061.i386.vib
Avoid mac and duplication
vCenter doesn’t mind the duplicate esxi mac addresses, but nsx controller deployments will fail.
Run the following on the nested host
# esxcli system settings advanced set -o /Net/FollowHardwareMac -i 1
Avoid uuid duplication (important on cloned esxi hosts)
Delete the line entry /system/uuid in /etc/vmware/esx.conf and run /sbin/auto-backup.sh to save it persistently. This ensure a new system uuid gets generated at boot time
Set nested host nic to e10003 – avoid packet loss dup!
Packet loss dup! packets and failing controllers in nsx when using jumbo frames
Issues with controller deploys, pings to and from the controllers show intermittent timeout and packets loss,
You should change the nic for the nested vms to e1000e, can’t remember where I discovered this…
Power off a the virtual machine
Log in as root to the terminal of the VMware ESX host.
Locate and open the virtual machine configuration file (.vmx)
For each network interface you wish to change to e1000e, modify the virtualDev line:
ethernetN.virtualDev = “e1000” to ethernetN.virtualDev = “e1000e”
Reload the vm configuration
# vim-cmd vmsvc/getallvms
In this example, the Vmid is 2848.
Reload the .vmx file using this command:
# vim-cmd vmsvc/reload Vmid
# vim-cmd vmsvc/reload 2848
Set your domain controller as an ntp server
VCSA, NSX, Controller and Edge devices will all appreciate using NTP rather than host time, point everything to the NTP server using utc
Download the easyfix from https://support.microsoft.com/en-us/kb/816042 and run in on the dc.